Welcome » IT Booklets » Information Security » Security Monitoring » Condition Monitoring » Independent Tests
Independent tests include penetration tests, audits, and
assessments. Independence provides credibility to the test results.
To be considered independent, testing personnel should not be
responsible for the design, installation, maintenance, and
operation of the tested system, or the policies and procedures that
guide its operation. The reports generated from the tests
should be prepared by individuals who also are independent of the
design, installation, maintenance, and operation of the tested
Penetration tests, audits, and assessments can use the same set
of tools in their methodologies. The nature of the tests,
however, is decidedly different. Additionally, the
definitions of penetration test and assessment, in particular, are
not universally held and have changed over time.
Penetration Tests. A
penetration test subjects a system to the real-world attacks
selected and conducted by the testing personnel. The benefit
of a penetration test is that it identifies the extent to which a
system can be compromised before the attack is identified and
assesses the response mechanism's effectiveness. Because a
penetration test seldom is a comprehensive test of the system's
security, it should be combined with other monitoring to validate
the effectiveness of the security process.
compares current practices against a set of standards.
Industry groups or institution management may create those
standards. Institution management is responsible for
demonstrating that the standards it adopts are appropriate for the
Assessments. An assessment
is a study to locate security vulnerabilities and identify
corrective actions. An assessment differs from an audit by
not having a set of standards to test against. It differs
from a penetration test by providing the tester with full access to
the systems being tested. Assessments may be focused on the
security process or the information system. They may also
focus on different aspects of the information system, such as one
or more hosts or networks.
Management is responsible for considering the following key
factors in developing and implementing independent tests:
testing is frequently only as good as the personnel performing and
supervising the test. Management is responsible for reviewing
the qualifications of the testing personnel to satisfy itself that
the capabilities of the testing personnel are adequate to support
the test objectives.
Scope. The tests and
methods utilized should be sufficient to validate the effectiveness
of the security process in identifying and appropriately
controlling security risks.
Management is responsible for considering whom to inform within the
institution about the timing and nature of the tests. The need for
protection of institution systems and the potential for disruptive
false alarms must be balanced against the need to test personnel
reactions to unexpected activities.
Data Integrity, Confidentiality,
and Availability. Management is responsible for
carefully controlling information security tests to limit the risks
to data integrity, confidentiality, and system availability.
Because testing may uncover nonpublic customer information,
appropriate safeguards to protect the information must be in
place. Contracts with third parties to provide testing
services should require that the third parties implement
appropriate measures to meet the objectives of the 501(b)
guidelines. Management is responsible for ensuring that
employee and contract personnel who perform the tests or have
access to the test results have passed appropriate background
checks, and that contract personnel are appropriately bonded.
Because certain tests may pose more risk to system availability
than other tests, management is responsible for considering whether
to require the personnel performing those tests to maintain logs of
their testing actions. Those logs can be helpful should the
systems react in an unexpected manner.
Confidentiality of Test Plans
and Data. Since knowledge of test planning and results
may facilitate a security breach, institutions should carefully
limit the distribution of their testing information.
Management is responsible for clearly identifying the individuals
responsible for protecting the data and providing guidance for that
protection, while making the results available in a useable form to
those who are responsible for following up on the tests. Management
also should consider requiring contractors to sign nondisclosure
agreements and to return to the institution information they
obtained in their testing.
frequency of testing should be determined by the institution's risk
assessment. High-risk systems should be subject to an
independent test at least once a year. Additionally, firewall
policies and other policies addressing access control between the
financial institution's network and other networks should be
audited and verified at least quarterly.The quarterly
auditing and verification need not be by an independent source.
See NIST Special Publication 800-41. Factors that may
increase the frequency of testing include the extent of changes to
network configuration, significant changes in potential attacker
profiles and techniques, and the results of other testing.
Independent testing of a proxy system is generally not effective in
validating the effectiveness of a security process. Proxy
testing, by its nature, does not test the operational system's
policies and procedures, or its integration with other
systems. It also does not test the reaction of personnel to
unusual events. Proxy testing may be the best choice,
however, when management is unable to test the operational system
without creating excessive risk.