Appendix A: Examination Procedures
Examiners should use these procedures to determine the quality and effectiveness of the institution's management of IT. Examiners should also use these procedures to measure the adequacy of the institution's ITRM process, including management awareness and participation, risk assessment, policies and procedures, reporting, ongoing monitoring, and follow-up.
These examination procedures (also known as the work program) are intended to assist examiners in determining the effectiveness of the institution's IT management process. Examiners may choose, however, to use only particular work steps of the following examination procedures based on the size, complexity, and nature of the institution's business. Examiners should use these procedures to measure the adequacy of the institution's cybersecurity risk management processes.
Objective 1: Determine the appropriate scope and objectives for the examination.
1. Review past reports for outstanding issues or previous problems. Consider the following:
- Regulatory reports of examination.
- Internal and external audit reports.
- Internal or independent tests or reviews of controls (e.g., penetration tests, business continuity reviews, and third-party management reviews).
- Regulatory and audit reports on service providers.
2. Review management's response to issues raised during, or since, the last examination. Consider the following:
- Adequacy and timing of corrective action.
- Resolution of root causes rather than just specific issues.
- Existence of any outstanding issues.
- Whether management has taken positive action toward correcting exceptions reported in audit and examination reports.
- Independent review of resolution and reporting of resolution to the audit committee.
3. Interview management and review responses to pre-examination information requests to identify changes to the technology infrastructure or new products and services that might increase the institution's risk. Consider the following:
- Products or services delivered to either internal or external users.
- Current network diagrams and data flow diagrams, including changes to configuration or components.
- Hardware and software inventories.
- Loss or addition of key personnel.
- Inventories of third-party providers and software vendors.
- Organizational charts that include reporting relationships between business units and control functions (e.g., enterprise risk management, ITRM, and internal audit).
- Credit or operating losses primarily attributable (or thought to be attributable) to IT (e.g., system problems, inadequate controls, improperly implemented changes to systems, and fraud resulting from cybersecurity attacks, such as account takeover).
- Changes to internal business processes.i. Internal reorganizations.
Objective 2: Determine whether the board of directors oversees and senior management appropriately establishes an effective governance structure that includes oversight of IT activities.
1. Review the institution's governance structure to determine the oversight of IT activities and verify that it includes the following:
- Board sets the tone and direction for the institution's use of technology.
- IT risks are adequately identified, measured, and mitigated.
- Board approval of the information security program and other IT-related policies.
- Board members are familiar with IT activities.
2. Review the activities performed by the board or a committee of the board to determine the effectiveness of IT oversight. Specifically, review whether the board or a committee of the board appropriately does the following:
- Reviews and approves an IT strategic plan that aligns with the overall business strategy and includes an information security strategy to safeguard against ongoing and emerging threats, including cybersecurity threats.
- Oversees the institution's adoption of effective IT governance processes.
- Oversees management processes for approving third-party providers that include an assessment of financial condition and IT security posture of the third party, including on cybersecurity.
- Has an oversight process that includes receiving updates on major projects, IT budgets, IT priorities, and overall IT performance; and has an approval process for critical projects and activities.
- Reviews the adequacy and allocation of IT resources in terms of funding and personnel.
- Approves a policy to escalate and report significant security incidents to the board, steering committee, government agencies, and law enforcement, as appropriate.
- Holds management accountable for the identification, measurement, and mitigation of IT risks.
- Provides for independent, comprehensive, and effective audit coverage of the IT program.
3. Determine whether the board does the following:
- Delegates monitoring for specific IT activities, as appropriate, to a steering committee.
- Provides a credible challenge to management decisions.
- Receives regular reports regarding operations.
- Directs management to maintain an institution-wide view of technology and the business processes supported by technology.
4. Review the membership list of board, steering committee, and/or relevant management committees established to review IT activities. Determine whether board, senior management, lines of business, audit, and IT personnel are represented appropriately, and whether regular meetings are held and minutes are maintained.
5. Review the minutes of the board of directors and relevant committee meetings for evidence of board support and supervision of IT activities.
6. If the board delegates certain activities regarding the oversight of IT to a committee, review the membership, responsibilities, and activities of the committee. Specifically, determine whether the committee does the following:
- Maintains a charter that defines its responsibilities.
- Has a defined mission to assist the board in IT oversight.
- Has decision-making authority.
- Receives appropriate management information from IT, lines of business, and external sources.
- Coordinates and monitors IT resources.
- Determines whether there is adequate training, including cybersecurity training, for institution staff.
- Reports to the board on the status of IT activities to enable the board to make decisions.
- Receives reports on IT to remain informed on risk.
- Is responsible for effective strategic IT planning, oversight of IT performance, and aligning IT with business needs.
7. Review the board of directors and management oversight program for IT. Determine whether the board has effective oversight of IT. Determine whether the board oversees and management implements the following:
- Processes and procedures that meet objectives of governing IT policies.
- Appropriate policies for information security, including cybersecurity risk management processes, and other relevant IT policies.
- Policies that result in compliance with applicable regulatory requirements.
- Controls over risks associated with system development and acquisition.
- Process for business continuity planning.
8. Review IT management and determine whether management performs the following:
- Implements effective IT governance and IT risk management processes, including those that relate to cybersecurity.
- Reviews, understands, approves, and provides for at least annual reviews of ITRM processes.
- Assesses the institution's inherent IT risks across lines of business and ensures IT risks are included in enterprise-wide risk assessments.
- Provides regular reports to the board on IT risks, IT strategies, and IT changes.
- Coordinates priorities between the IT department and lines of business.
- Establishes a formal process to obtain, analyze, and respond to information on threats and vulnerabilities by developing a repeatable threat intelligence and collaboration program.
- Ensures that hiring and training practices are governed by appropriate policies to maintain competent and trained staff.
9. Review the roles and responsibilities of all levels of management, including executive management, CIO or CTO, CISO, IT line management, and IT business unit management, to ensure that there is a clear delineation between management and oversight functions and operational duties.
10. Review the corporate and IT departmental organization charts to determine whether they show the following:
- IT management reports directly to senior management, with appropriate reporting directly to the board, as needed.
- The IT department's responsibilities are appropriately segregated from business processing activities.
11. Review the institution's structure to determine whether the board established the following:
- The organizational structure provides for effective IT support throughout the institution, from IT management up through senior management and the board.
- Defined roles and responsibilities for key IT positions, including executive management (CEO and COO, and often CIO or CTO), and CISO.
- An appropriate and effective executive management team or positions, such as CEO and COO, to assist in the oversight and management of IT.
- A defined and functioning role for the CIO or CTO to focus on strategic IT issues and the overall effectiveness of the IT function.
- A CISO or information security officer position responsible for the management and mitigation of information security risks.
- Involvement of frontline management in the IT oversight process.
- Integration of business line managers into the IT oversight process.
12. Determine whether the reporting structure ensures that the CISO has the appropriate authority to carry out its responsibilities and that there are no conflicts of interest in the ability of the CISO to make decisions in line with the risk appetite.
13. Determine management's need for, or effectiveness in, selecting and implementing appropriate EA and assess whether the EA program serves the institution's needs, complexity, and future technology plans.
Objective 3: As part of the ITRM structure, determine whether financial institution management has defined IT responsibilities and functions. Verify the existence of well-defined responsibilities and expectations between risk management and IT functional areas, such as information security, project management, business continuity, and information systems reporting.
1. Review the institution's established lines of authority for enforcing and monitoring controls.
2. Determine whether management has a board-approved written information security program and verify that it is maintained and updated according to regulatory requirements.
3. Determine whether the institution has a project management function appropriate for the complexity of the institution, and verify that this function contains the appropriate elements.
4. Determine whether the institution maintains an adequate and up-to-date enterprise-wide business continuity plan. Determine whether the board oversees implementation and approves policies related to business continuity planning.
5. Determine whether the institution has a well-defined role for the implementation and use of information systems reporting and that it produces accurate and useful reports. Determine the effectiveness of the reports used by senior management or relevant management committees to supervise and monitor the following IT functions:
- Management reports that provide the status of software development and maintenance activities.
- Performance and problem reports prepared by internal user groups.
- System use and planning reports prepared by operating managers.
- Internal and external audit reports of IT activities.
6. Review information systems reports for management, and determine whether they provide the information necessary to help manage the institution effectively. Determine the following:
- The information systems reports facilitate the management of the business.
- The process and results are effective.
- Data and information provided to the board and senior management allow them to make strategic decisions.
- The information systems reports provide key risk and performance trends, indicators, and performance against risk tolerances.
- The institution has effective controls procedures in place to ensure that information is correct and relevant.
- The systems and reporting meet the five elements of effective reporting: timeliness, accuracy, consistency, completeness, and relevance.
- The information systems reports are appropriate for the size and complexity of the institution.
Objective 4: Determine the adequacy of the institution's IT operations planning and investment. Assess the adequacy of the risk assessment and the overall alignment with the institution's business strategy, including planning for IT resources and budgeting.
1. Determine whether the board oversees and management considers the following when formulating the institution's overall business strategy:
- Risk assessment, priority, and mitigation across the institution.
- IT strategic plans.
- Major projects in process or planned.
- Third-party relationships, including the third party's current and future plans (e.g., changes in strategy and products offered) and service or security issues that may affect the institution.
- Staffing levels sufficient to complete tasks as scheduled.
- IT operating costs.
- IT contingency planning and business recovery.
2. Review the strategic plan for IT activities. Determine whether the goals and objectives are consistent with the institution's overall business strategy. Document significant changes made since the previous examination that affect (or any planned changes that may affect) the institution's organizational structure, hardware or software configuration, and overall operational goals. Determine the following:
- Business needs are realistic.
- IT has the ability to meet business needs.
- The plan addresses long-term (three- to five-year horizon) goals and allocation of resources.
- The plan incorporates the entire IT environment.
- The plan lists strategic initiatives and considers all necessary factors around those initiatives.
- The plan includes tactical plans to achieve strategic goals.
- The plan explains trends and issues of potential impact.
- The plan incorporates clearly defined goals and metrics.
- The planning process adjusts for new or changing risks.
- IT Management participates in the development of the IT strategic plan.
- There is review of and credible challenge to the plan.
3. Determine whether the institution has adequate tactical and operational IT plans to support the larger IT strategic plan.
4. Determine whether the board or board committee reviews and approves the following:
- Information security risk assessment, including cybersecurity.
- Short- and long-term IT tactical, operational, and strategic plans.
- Resource allocation (e.g., major hardware or software acquisition and project priorities).
- Reported status of major projects.
- IT budgets and current operating cost and the allocation of IT resources.
5. Determine the effectiveness of management's process to fund IT resources to meet the current operational needs of the institution. Assess whether management considers the following IT resources:
- Operating software.
- Application software.
6. Determine whether the board reviews management's budget plans. Determine the effectiveness of the budget process to estimate and control the institution's activities.
7. If the institution uses third-party providers, determine whether management:
- Verifies that the third-party providers can continue to support current contract requirements and future changes (e.g., that the third party has a satisfactory financial condition).
- Has a process to assess whether a third party's actions may negatively affect the institution (e.g., a review of the third-party plans to continue offering the necessary products or services contracted by the institution).
- Has an effective ongoing monitoring process of its third-party providers.
Objective 5: Along with the IT audit and compliance departments, the HR department can serve as an influencing function for IT. Determine the adequacy of the institution's HR function to ensure its ability to attract and retain a competent workforce.
1. Determine the institution's ability to attract and retain a competent workforce and the ability of HR management to effectively meet the requirements for IT and the lines of business that IT supports.
2. Identify key IT positions, review biographical data (e.g., résumés and training and development records), and determine the following:
- Job descriptions are reasonable and represent actual practice.
- Employees have appropriate qualifications.
- Staffing levels are appropriate.a. There are provisions for management succession that provide for an acceptable transition in the event of the loss of a key IT manager or staff member.
- Backup personnel are identified and trained.
3. Review and evaluate written job descriptions to ensure that management performs the following:
- Clearly defines the authority, responsibility, and technical skills required.
- Maintains updated job descriptions in writing.
4. Determine whether the HR function has processes for compensation planning, performance reviews, knowledge transfer mechanisms, training, and mentoring.
5. Determine whether the financial institution has a process to ensure that staff has the requisite expertise to fulfill its roles. Review the adequacy of the process.
6. Determine the adequacy of the institution's training programs. Determine whether the institution has or supports the following:
- Internal or external training programs.
- Certification programs.
- Training processes that support the goals and objectives of the institution.
7. Review turnover rates of IT staff and discuss staffing and retention issues with IT management, or review turnover rates of IT management and discuss with senior management. Identify root causes of any staffing or expertise shortages, including compensation plans or other retention practices.
8. If IT staff members have duties in other departments, determine the following:
- Management is aware of the potential conflicts such duties may cause.
- Conflicting duties are subject to appropriate supervision and compensating controls.
Objective 6: Evaluate management's review and oversight of IT controls, including the other influencing functions of IT audit and compliance.
1. Consult with the examiner reviewing audit or IT audit to determine the adequacy of IT audit coverage and management's responsiveness to identified weaknesses.
2. Determine whether the board provides for the necessary expertise in the audit department and that audit coverage is comprehensive, timely, and independent. Assess whether the board requires audit reporting directly to the board or a designated committee.
3. Determine whether the board, or its committee, has appropriate oversight of audit through the following:
- Audit risk assessment and audit plan.
- Audit review activities.
- Audit reports with identified weaknesses.
- Management's responses and corrective actions to audit issues.
- Updates on any audit concerns and the status of issues.
4. Determine whether the board, or a board committee, is responsible for overseeing performance and compensation for the audit department.
5. Determine whether the compliance function has involvement in the institution's review oversight process, and assess the adequacy of its involvement.
6. Determine whether compliance staff reviews new products, systems, applications, or changes to evaluate compliance with applicable laws and regulations.
Objective 7: Determine whether the institution's risk management program facilitates effective risk identification and measurement and provides support for risk decisions within ITRM.
1. Determine whether the institution has a risk management program and whether the program includes an integrated approach for enterprise-wide risk management, including identification, measurement, mitigation, monitoring, and reporting of risk. If applicable, determine whether the structure conforms to regulatory requirements.
2. Determine the effectiveness of the risk management program by reviewing whether it receives appropriate direction and support from the board and senior management.
3. Determine the following:
- The board of directors has defined its risk appetite and the institution's risk tolerance levels.
- The board of directors has applied sufficient resources to achieve its risk appetite and remain within the institution's risk tolerance levels.
- Management has committed to support the board's risk decisions.
4. Determine whether the institution maintains a risk assessment process to perform the following:
- Identify risks and threats from both internal and external sources.
- Develop or update policies within the risk management function to guide risk measurement activities.
- Ensure the existence of a process to promote sound understanding and analysis of threats, events, assets, and controls.
- Maintain processes within the risk management function to help make risk mitigation decisions.
- Determine the entities that should have involvement in that decision-making process.
- Ensure that the board and management understand the risk categories.
Objective 8: Determine whether the board of directors oversees and senior management proactively mitigates operational risk.
1. Review the institution's management of operational risk, and verify that the risk management process includes aspects of operational risk across the institution, including the following:
- Back-office operations and transaction processing.
- Customer service.
- Systems development and support.
- Internal controls and processes.
- Capacity planning.
2. Determine whether the institution's management of operational risk incorporates an enterprise-wide view of IT and business processes that are supported by technology.
3. Assess whether IT management maintains an active role in the institution's strategic planning to align IT with established business goals and strategies. Assess whether effective IT controls exist throughout the institution, either through direct oversight or by holding lines of business accountable for IT-related controls.
4. Determine whether IT management participates in the enterprise-wide risk management process to identify and measure risk from the use of IT, support decisions on how to mitigate the risks, implement the mitigation decisions, and monitor and report on the resulting outcomes.
Objective 9: Determine whether management implements an ITRM process that supports the overall enterprise-wide risk management process.
1. Review the role of IT management in the risk management process and identify whether it is supportive and collaborative to the overall process.
2. Determine whether the ITRM process includes the following:
- A risk identification process to identify risks to information assets within the institution and information assets controlled by third-party providers.
- A risk measurement process using an evidence-based approach to measure the level of risk and determine if it is in line with the board's risk appetite.
- A risk mitigation process to ensure that management mitigates the risks to an acceptable residual risk level.
- A risk monitoring and reporting process to monitor changing risk levels and report the results of the process to the board and senior management.
3. Determine whether the ITRM process includes the following:
- Is regularly updated with a frequency appropriate for the pace of change.
- Aligns IT and business objectives.
- Has formality appropriate to the complexity of the institution.
- Considers the overall IT environment, regardless of the design and management of the IT environment.
Objective 10: Determine whether the institution maintains a risk identification process that is coordinated and consistent across the enterprise.
1. Determine whether the institution has a comprehensive IT risk identification process that includes the identification of cybersecurity risks. Specifically, determine whether management performs the following:
- Maintains an inventory of assets, event classes, threats, and existing controls.
- Participates in an information sharing forum (such as FS-ISAC).
- Has a process to identify internal and external threats.
- Considers existing controls-including governance of controls, their limitations, and their effectiveness-in a comprehensive control assessment.
- Has a risk identification process that is formal yet flexible enough to adapt to changes in the IT environment.
- Incorporates a measurement and assessment of outsourced relationships in the risk identification process.
- Considers the information security risk assessments completed in accordance with the Information Security Standards in management oversight of IT operations.
2. Determine whether the institution's risk identification process includes the ongoing collection of information on the IT environment, including the following:
- IT systems inventories.
- IT strategic plans.
- Interconnectivity documentation.
- Information flow diagrams.
- Business continuity and disaster recovery plans.
- Third-party management program.
- Call center data.
- Department self-assessments.
- IT audit findings.
- Threat intelligence information.
Objective 11: Determine whether institution management maintains a risk measurement process that is coordinated and consistent across the enterprise.
1. Determine whether management's risk measurement process includes the determination of risk factors (such as adverse events, threats, and controls) and the affected assets. Determine whether management develops inventories of those risk factors. Specifically, determine whether management does the following in the risk measurement process:
- Identifies reasonable threats to financial institution assets.
- Performs a threat analysis.
- Estimates the probability of occurrence of adverse events.
- Determines the potential impacts of events and threats, internal and external.
- Analyzes the institution's technical and organizational vulnerabilities.
- Measures risk through qualitative, quantitative, or hybrid measurement approaches.
- Measures and assesses risks posed by third-party relationships.
- Considers the information security risk assessments completed in accordance with the Information Security Standards in management oversight of IT operations.
- Risk ranks information assets according to a rigorous and consistent methodology.
2. Determine whether the risk measurement process is comprehensive and includes the following types of risks that affect the institution:
- Security breaches.
- System failures.
- External or insider events.
- Development and acquisition issues.
- Capacity planning issues.
- Third-party provider issues.
3. Identify whether the institution has a proactive process in place to effectively update its measurement of risk before implementing system changes, rolling out new products or services, or confronting new external conditions.
Objective 12: Determine whether financial institution management effectively implements satisfactory risk mitigation practices.
1. Determine whether the institution has processes within enterprise-wide risk management to assist IT management in making risk mitigation decisions, and determine which entities should be involved in the decision-making process.
2. Determine whether management has adequate methods and tools, including control self-assessments and scenario analysis, to evaluate controls for effectiveness against identified threats.
3. Determine whether the ITRM process addresses risks with an effective IT control structure in the institution's IT environment and through conformance with external legal and regulatory requirements.
4. Determine whether IT management has developed adequate policies, standards, and procedures to manage the risk from technology and that they are current, documented, and appropriately communicated. Policies, standards, and procedures should address the following:
- Risk assessment.
- Personnel administration.
- Development and acquisition, including secure development.
- Computer operations.
- Third-party risk management.
- Computer and information security, including cybersecurity.
- Business continuity and resilience planning.
- IT audit.
5. Determine whether management has effective hiring and training practices that include the following:
- Performing appropriate background checks on new staff, contractors, and third-party provider personnel, as necessary.
- Confirming identity.
- Obtaining character references.
- Requiring periodic acknowledgement of acceptable-use policies.
- Obtaining signed confidentiality and nondisclosure agreements.
- Providing information security awareness and training programs.
6. Determine whether the board has appropriate oversight and management has appropriate responsibility for the implementation of the institution's information security program.
7. For the information security program, verify that the board is responsible for the following:
- Overseeing the development, implementation, and maintenance of the program.
- Assigning specific responsibility for its implementation.
- Providing management with guidance and reviewing the effectiveness of management's actions.
- Annually reviewing and approving a formal, written information security program.
- Overseeing management steps to safeguard the information assets of the bank and its customers.
- Annually reviewing management's report on the status of the bank's actions to achieve or maintain compliance with the Information Security Standards.
8. Determine whether, as part of the institution's information security program, the board of directors oversees and management establishes a control structure that is intended to specifically address cybersecurity risks and includes the following:
- Developing and implementing processes to identify, protect against, detect, respond to, and recover from security events and incidents.
- Developing, implementing, and periodically testing incident response procedures.
- Using a threat intelligence and collaboration process to identify and respond to information on threats and vulnerabilities.
- Including information security risks when developing, implementing, or updating products.
- Assigning "business owner" responsibility in product development or update processes.
- Performing penetration tests before launching new or making significant changes to existing Internet- and client-facing applications and remediating findings from the tests.
- Conducting initial due diligence and ongoing monitoring to fully understand the connections and mitigating controls in place between the financial institution and its third-party providers.
- Implementing a governance process to establish, monitor, maintain, and test controls to mitigate interconnectivity risk.
- Developing a policy for escalating and reporting security incidents to the board, government agencies, law enforcement, and the institution's primary federal and state regulators based on thresholds defined by the financial institution.
9. Determine whether the board of directors approved policies and management established and implemented policies, procedures, and responsibilities for an enterprise-wide business continuity program, including the following:
- Annual review and approval of the business continuity program by the board of directors.
- Management responsibility to document, maintain, and test the plan and backup systems periodically according to risk.
- Annual reports by management of the results of the business continuity and disaster recovery tests to the board of directors.
10. Determine whether management assesses and mitigates the operational risks associated with the development or acquisition of software. Appropriate management of the risks should include the following:
- Policies documenting risk management controls for the development and acquisition of systems.
- System development life cycle or similar methodology based on the complexity and type of development performed.
- Tests of new technology, systems, and products before deployment to validate functionality, controls, and interoperability.
- Penetration tests of new or updated applications, particularly for Internet- or client-facing applications, to detect and correct security flaws.
11. Review major acquisitions of hardware and software to determine if the acquisitions are within the limits approved by the board of directors.
12. Determine whether management is aware of and mitigates operational risks associated with IT operations, including the following:
- Data center or computer operations.
- Network services.
- Distributed computing.
- Desktop computing.
- Change management.
- Project management.
- Resource management.
- Contingency and resiliency planning.
13. Review the financial institution's insurance program and determine whether it is commensurate with the size, complexity, risks, and mitigation strategy of the institution. Determine the adequacy of insurance coverage (if applicable) for the following:
- Employee fidelity.
- IT equipment and facilities.
- Media reconstruction.
- Extra expenses, including backup site expenses.
- E-banking activities.
- Business interruption.
- Valuable papers and records.
- Errors and omissions.
- Items in transit.
- Other probable risks (unique or specific risks for a particular institution).
14. Review the financial institution's third-party management program to ascertain the extent and effectiveness of the oversight by the board of directors and management of risks involved in the financial institution's outsourced relationships. An effective third-party management program should incorporate the following:
- A framework for management to identify, measure, mitigate, and monitor the risks associated with third-party relationships.
- Board oversight and senior management development and implementation of enterprise-wide policies to govern the third-party management program.
- A review process of third-party providers to ensure that each relationship supports the institution's overall business objectives and strategic plans.
- Evaluation of prospective third-party providers based on the scope and criticality of services provided.
- Tailoring of the monitoring program based on the initial and ongoing risk assessment of the third party and the services provided.
15. As part of the examiner's review of the institution's third-party management program, analyze the third party's financial condition and note any potential weaknesses, including measures to improve those weaknesses.
16. When reviewing information provided by the institution's third-party providers, determine whether the third-party provider enables adequate financial institution client access to relevant information. Consider the following:
- The third party's method of communication with financial institution clients.
- Timeliness of third-party reporting to financial institution clients.
- Quality of financial information, as determined by internal or external auditor reports.
17. When reviewing information provided by the institution's third-party providers, determine the adequacy of third-party provider audit reports in terms of scope, independence, expertise, frequency, and corrective actions taken on identified issues. Work with the examiner reviewing the third-party management program to determine its adequacy.
18. When reviewing information provided by the institution's third-party providers, determine the quality of management's follow-up and resolution of customer concerns and problems with its third-party providers.
Objective 13: Determine whether IT management develops satisfactory measures for defining and monitoring metrics, performance benchmarks, service level agreements, compliance with policies, effectiveness of controls, and quality assurance and control. Determine whether management developed satisfactory reporting of ITRM activities.
1. Determine whether management develops and uses metrics to help assess the overall IT environment. Determine whether the metrics used and the frequency and monitoring of those metrics are useful to direct management's attention to emerging issues. Additionally, determine whether necessary metrics or summary reports of metrics are provided to the board.
2. Determine whether there are established performance benchmarks and standards for the IT function and whether they serve to help management identify problem areas, particularly in system or data center availability, operating conditions, response times, and error rates.
3. Review whether the institution has formal service level agreements with all of its third-party providers. Determine whether the agreements provide the institution with assurance of continued service.
4. Determine the effectiveness of management's communication and monitoring of IT policy compliance across the institution.
5. Determine whether management has an adequate method of testing the effectiveness of control design and implementation and whether management and the board appropriately monitor risk mitigation activities. Determine whether management considers all forms of controls, including governance of controls, their limitations, and their effectiveness in a comprehensive control assessment.
6. Determine whether management has QA and QC procedures defined for significant IT activities and whether those procedures are performed internally or externally. Specifically, review whether management:
- Has a process to assist it in determining whether products or services meet specified requirements (QA).
- Has procedures to ensure that a product or application adheres to a defined set of quality criteria to meet end-user requirements (QC),
- Performs tests associated with QA and QC independent of the programming function, and whether the QA and QC procedures incorporate user acceptance testing programs.
- Receives effective reports on the results of QA and QC testing.
7. Review the monitoring and reporting specific to the institution's ITRM activities. Specifically, determine whether the institution has developed the following:
- A process to adequately identify and monitor relevant external threats and vulnerabilities.
- Effective risk monitoring that provides tangible feedback on the quality of the implementation of controls and risk mitigation strategies.
- A reporting process that assembles and reports IT risk-related information in a timely, complete, transparent, and relevant manner.
- Appropriate escalation procedures in place depending on the content of the reporting.
Objective 14: Discuss corrective action and communicate findings.
1. Review preliminary conclusions with the examiner-in-charge (EIC) regarding:
- Violations of laws and regulations.
- Significant issues warranting inclusion as matters requiring attention or recommendations in the report of examination.
- Proposed Uniform Rating System for Information Technology management component rating and the potential impact of the examiner's conclusions on composite or other component IT ratings.
- Potential impact of the examiner's conclusions on the institution's risk assessment.
2. Discuss findings with management and obtain proposed corrective action for significant deficiencies.
3. Document conclusions in a memorandum to the EIC that provides report-ready comments for all relevant sections of the report of examination and guidance to future examiners.
4. Organize work papers to ensure clear support for significant findings by examination objective.
Appendix B: Glossary