Execution, Evaluation, Independent Assessment, and Reporting of Test Results
Once testing strategies and test plans are developed, the following procedures should be implemented as part of the overall testing policy:
Execution and Documentation
Testing requires centralized coordination by the BCP coordinator or team. The team or coordinator is responsible for overseeing the accomplishment of targeted objectives and ensuring the test results are appropriately documented.
Generally, it is advisable to have the maximum number of personnel involved in implementing the BCP to also participate in the test. Management should also rotate personnel periodically during the testing process to reduce dependence on specific individuals who may leave the organization or may not be available during a disaster. This participation increases awareness and ownership in achieving successful BCP implementation.
Once the tests are executed, test results should be properly documented and include the following, at a minimum:
- Test dates and locations;
- An executive summary detailing a comparison between the test objectives and test results;
- Material deviations from the test plans, including whether intended participation levels were achieved;
- Problems identified during testing; and
- An evaluation by a qualified independent party.
Once tests have been executed and documented, test results should be evaluated to ensure that test objectives are achieved and that business continuity successes, failures, and lessons learned are thoroughly analyzed. Business lines and support function management should review test results to validate whether test procedures were effectively completed and adequately documented. Finally, test results, including quantitative metrics, such as achieving RTOs and RPOs, should be used to determine the effectiveness of the institution's BCP. If test objectives were not achieved, business line and support function management should identify necessary corrective measures and determine whether a follow-up test should be conducted prior to the next regularly scheduled exercise. Exceptions to this process should be documented and approved by senior management.
Institutions are expected to evaluate testing across business lines and support functions in order to validate the BCP. An analysis of tests completed over a period of time should be conducted to determine whether the institution is capable of achieving its overall business continuity objectives.
Key tests should be observed, verified, and evaluated by independent parties. This provides assurance to the board and other stakeholders of the validity of the testing process and the accuracy of test results. This independent assessment is typically conducted by internal audit, although it can be performed by other qualified third parties. An effective practice is to include a review by both business line and IT auditors. This review should include an assessment of the testing scope and objectives, written test plans, testing methods and schedules, and communication of test results and recommendations to the board. The analysis of underlying assumptions and the results of modeling and simulation techniques should also be independently assessed to assure the board and other stakeholders of their reasonableness and validity. In addition, the board should receive and review audit reports addressing the effectiveness of the institution's process for identifying and correcting areas of weakness, and audit recommendations should be monitored to ensure that they are implemented in a timely manner.
Reporting Test Results
Test results, gaps between the BCP and the actual test results, and the resolution of any problems should be reported to several audiences, including the board and senior management, business line management, risk management, IT management, and other stakeholders. A management assessment of the institution's ability to meet its continuity objectives and testing program requirements should be provided to the board at least annually. The assessment should contain sufficient information so that the board can determine if the BCP meets the objectives established by the BIA. In addition, business lines and support functions should identify the validity of the test data processed, any untested aspects of production operations, and the need for additional tests. The board should receive reports more frequently when test results for critical business lines indicate an inability to meet continuity objectives.
Updating Business Continuity Plan and Test Program