The foundation of what we do in InfoSec is all based on risk. How we select controls to reduce the likelihood and impact of threats and vulnerabilities stems from our risk assessments. Every compliance standard mandates one form of risk assessment be done as a part of an organization’s security program. But not all risk assessments are done the same way nor do they produce the same results.
There are many risk assessment standards for organizations to choose from, including OCTAVE, FAIR, ISO 27005:2011, FMEA, and NIST SP 800-30. Some are quantitative, based on real numbers and data with results often expressed in potential dollars lost, while others are qualitative, based on expert opinion and expressed in colors or high-medium-low ratings. Some risk assessments are built from hundreds of hours of observations and measurements. Others are constructed from calibrated opinion surveys of subject matter experts. More than a few risk assessments are presented with incomplete results, merely focusing on threats and vulnerabilities while ignoring impacts or likelihoods.
Until recent history, auditors and regulators have mostly graded risk assessments on effort, not effectiveness or completeness. The common audit question is “Did you do a risk assessment?” can be satisfactorily answered with a simple affirmative. In 2015, I presented1 research at the Society of Information Risk Analysts (SIRA) conference that showed that only 29% of organizations assessing the security of third parties asked if there was a risk assessment process in place. Only 13% wanted to know any details on that process.
Weak risk assessments have gotten a pass up until now, but that may be changing. On April 12, 2017, the FTC issued a Warning Letter to Abbot Labs for a number of security failings in their medical devices. A key cause that was singled out was poor risk assessment. They noted:
“Your firm identified the hardcoded universal unlock code as a risk control measure for emergent communication. However, you failed to identify this risk control also as a hazard. Therefore, you failed to properly estimate and evaluate the risk associated with the hardcoded universal lock code in the design of your High Voltage devices.” 2
As well as:
“Your firm’s updated Cybersecurity Risk Assessments, (b)(4) Cybersecurity Risk Assessment, (b)(4), Revision A, April 2, 2015 and Merlin@home Product Security Risk Assessment, (b)(4), Revision B, May 21, 2014 failed to accurately incorporate the third party report’s findings into its security risk ratings, causing your post-mitigation risk estimations to be acceptable, when, according to the report, several risks were not adequately controlled.”
What better way to diagnose a failed security program than to point at an inferior assessment of risk? If an organization omits or misjudges a critical risk, then the decisions that flow from that finding will be incorrect.
A problem with standardizing risk assessment is that the measurement of relevant risk is going to vary significantly from organization to organization, with different priorities, trade-offs, and tolerances affecting the analysis. However, the question remains: can your risk assessment withstand outside scrutiny? If you get unlucky and hacked, how is your organization’s risk assessment going to fare when regulators and lawyers scrutinize it page by page?
Your strategy should be to develop a risk assessment that appears reasonable and appropriate to hazards, threats, and potential impacts on your systems. The FTC came down hard on Abbot Labs because they manufacture medical devices, so impacts can include loss of life and breach of medical privacy. A more thorough risk assessment would be expected in this environment rather than of an IT office equipment vendor.
A defensible risk assessment should be:
- Standardized. The method should be as formal as possible so that given the same data, someone could reproduce the same results. The same method should be used for the same type of risk assessment.
- Relevant. The right risk modeling technique should be chosen for your organization’s industry, possible impacts, and threat environment. It should also be current—the standard is to perform them at least once a year.
- Explicit. Assumptions, trade-offs, estimates, and conclusions should be clearly documented. An auditor or regulator should be able to trace your line of reasoning in decisions made.
A risk assessment must also be read and used to manage the risk it identifies. A thick, beautifully detailed risk report that sits on the shelf is not only useless, but a clear indicator of negligence. Imagine a regulator asking you, “You knew about this risk, but why didn’t you do anything about it?” Acting on a risk assessment also means verifying that the risk was reduced by an active risk management process. In the same letter to Abbot Labs, the FTC also reprimanded them by saying:
“Your firm conducted a risk assessment and a corrective action outside of your CAPA system. Your firm did not confirm all required corrective and preventive actions were completed, including a full root cause investigation and the identification of actions to correct and prevent recurrence of potential cybersecurity vulnerabilities, as required by your CAPA procedures.”
No one wants to have these things said about their security program. I see this as a warning shot across the bow for all organizations: clean up your risk assessment processes and make sure you act on the results.
