Measuring the effectiveness of cybersecurity processes. IS metrics. Part 3

Ruslan Rakhmetov, Security Vision

In the previous article we looked at NIST SP 800-55, which is dedicated to recommendations for measuring cybersecurity effectiveness, and in today's article we will look at the international standard ISO/IEC 27004:2016, which is also dedicated to measuring, evaluating and assessing the effectiveness of information security management systems (ISMS) and describes how to build a process for measuring IS effectiveness in organisations.

ISO/IEC 27004:2016 ‘Monitoring, measurement, analysis and evaluation’ is the successor to the first version of this document, which was released in 2009. This document is logically related to the ISO 27000 series of standards, uses the same terminology and is used to verify that an ISMS meets the performance assessment requirements as specified in clause 9.1 of ISO/IEC 27001:2013. Specifically, this clause requires organisations to assess the effectiveness of the information security function and the built ISMS by defining the objects of measurement (including IS processes and measures), selecting methods for assessing and evaluating effectiveness, and approving timelines and responsibilities while maintaining documented results of the assessment.

The purpose of evaluating the effectiveness of the ISMS is to verify that the processes and activities implemented actually lead to fulfilment of the requirements of ISO 27001. The main tasks in evaluating the effectiveness of an ISMS are to collect relevant information, to process and analyse it correctly and to ensure that the results are reproducible. The benefits of implementing ISMS performance assessment processes are increased transparency of the ISMS (identifying incorrect, ineffective or not implemented protection measures), providing a quantitative assessment of the progress of ISMS effectiveness and IS processes, documenting compliance with ISO 27001 requirements, and supporting risk-based decision-making processes, including for justifying the allocation of IS budgets.

ISO/IEC 27004:2016 suggests that monitoring of the following activities, processes and systems should be used to provide relevant information suitable for subsequent analysis:

1. Implementation of ISMS processes

2. Management of cyber incidents

3. Vulnerability management

4. Configuration management

5. Awareness programmes

6. IS event collection

7. IS auditing

8. Cyber risk assessment and handling processes

9. Third party cyber risk management

10. Business continuity management

11. Physical security process management

12. IT infrastructure monitoring.

ISO/IEC 27004:2016 specifies the following ISMS processes that can be analysed to measure IS effectiveness:

1. Planning

2. Support from the company's management

3. Risk management

4. IS policy management

5. Resource management

6. Information sharing

7. Management evaluation

8. Documentation

9. Auditing.

The paper also suggests the following roles that can be assigned to the responsible persons of the company as part of the process of measuring the effectiveness of the ISMS:

1. Customer - requests information on the effectiveness of the ISMS and information protection measures, including the PPEs

2. Planner - matches the data received from the ISMS or the PPE with the parameters used to evaluate the effectiveness of the ISMS, creating a measurement model

3. Reviewer - verifies the correctness of the measurement model and the relevance of the mapping of data to the parameters

4. Information Owner - transfers data from the components of the ISMS or MIS under his/her responsibility

5. Information collector - responsible for collecting, recording, and storing data for the measurement

6. Analyst - analyses the collected data

7. Communicator - communicates the results of the analysis to interested authorised persons.

The ISO/IEC 27004:2016 standard proposes the following process-based PDCA approach for evaluating the effectiveness of an ISMS:

1. Defining the input data to be used for evaluation and analysis; this uses a high-level description of the ISMS and the company's business processes.

2. Creation and maintenance of measurable parameters that are transferred from the ISMS components, from the PPE, from employees as a result of surveys, e.g. IS events, scan reports, statistics on passed training, statistics on IS incidents, results of audits, results of business continuity and recovery training alarms.

3. Launching procedures to assess the effectiveness of ISMS, which includes notification of the involved persons and departments, selection and configuration of systems for data collection, setting up rules for checking the received data for correctness, data analysis and reporting in the specified format, including reports, dashboards, diagrams.

4. Monitoring and measurement, which includes manual and/or automated data collection, checking the data received for correctness, secure storage.

5. Analysis of results and interpretation of the analysis results, which should reflect ISMS deficiencies and deviations between expected and actual measurement results of ISMS parameters.

6. Evaluating the effectiveness of the ISMS and IS processes based on the analysis of the results obtained.

7. Revise and improve the processes of monitoring, measuring, analysing, evaluating the ISMS based on the feedback received, analysis of lessons learned, benchmark data, revision of procedures for evaluating the effectiveness of the ISMS.

8. Retention of reporting documents, transferring them for review to interested authorised persons.

Annex ‘B’ of ISO/IEC 27004:2016 provides an example of a set of directly applicable IS metrics that can be used to evaluate the effectiveness of an ISMS in accordance with the requirements specified in ISO 27001:

1. Ratio of resources utilised to resources allocated to the ISMS within the budget period.

2. The ratio of IS policies revised during the period to the total number of IS policies.

3. Ratio of meetings held with top management on IS issues to the number of planned meetings.

4. Number of cyber risks not handled during the time period.

5. Ratio of IS audits performed to planned audits.

6. Ratio of activities completed on time, with the specified quality and within the planned costs to the total number of planned activities.

7. Total damage from IS incidents that occurred over a period of time.

8. Ratio of the number of IS incidents that resulted in review and remediation of ISMS processes to the total number of cyber incidents.

9. The ratio of unrealised corrective actions to the total number of planned corrective actions to correct ISMS processes.

10. Ratio of the number of employees who have undergone IS training to the total number of employees who should have undergone IS training within a given period of time.

11. Percentage of employees who correctly answered the test questions on awareness-training materials.

12. Percentage of employees who clicked on a phishing link as part of a training mailing.

13. Percentage of passwords to employee accounts that comply with the organisation's password complexity rules and password rotation policies.

14. Percentage of passwords that are bruteforce-crackable in 4 hours.

15. Ratio of the number of critical IT systems where user access rights are periodically reviewed to the total number of critical IT systems.

16. Number of physical security incidents in which unauthorised access to IT systems was gained.

17. Lag of the date of completed maintenance of IT systems from the scheduled date.

18. Ratio of the number of installed/updated applications/systems, the installation/updating of which was in compliance with the regulations for making changes to the IT systems configuration, to the total number of installed/updated applications/systems.

19. Ratio of the number of IS incidents caused by malware to the total number of detected and blocked malware attacks.

20. Ratio of the number of hosts in the company's network with anti-virus signature databases outdated by 1 week or more to the total number of devices in the network.

21. The number of rules on border firewalls with zero ‘hits’ (i.e., not used once) over the time period.

22. Ratio of analysed log files from critical IT systems to the total number of available log files from critical IT systems over the time period.

23. Ratio of the number of devices in the company network whose configuration meets IS requirements to the total number of devices.

24. Ratio of the number of critical IT systems for which penetration testing or vulnerability scanning has been performed since the date of the last major software/OS update to the total number of critical IT systems.

25. Ratio of the number of IT systems for which penetration testing or vulnerability scanning was conducted within the last year or within the last quarter to the total number of IT systems.

26. CVSS scores and number of vulnerable systems.

27. Ratio of third-party contracts with IS requirements to total number of contracts.

28. Number of IS incidents that took longer to process than the normative time for this type of cyber incident.

29. Number and type of IS incidents detected during the time period.

30. Ratio of the number of ISMS audits performed by independent external auditors to the number of planned audits.