Over the past 11 years, I’ve done hundreds of audits for organizations of all sizes around the world. I specialize in audits for SSAE 16/18 (SOC1 and SOC2),1 Sarbanes Oxley,2 and PCI DSS.3 I’ve seen a lot of audit failures, and there are some common themes to them from which other companies can learn. My work has been primarily in the software, cloud, telecom, and manufacturing industries, but these concepts apply to any industry.
1. Poor prioritization from the top
I can predict how an audit will go by looking first to the tone at the top. If management hasn’t bought into the importance of compliance, then the people implementing and working on the controls won’t, either. Management attitude establishes priority for the entire organization, which drives resources and participation.
Compliance should be incorporated into the culture of an organization. Management attention gives teeth to a policy, which makes controls stick and reduces the chance it’ll be ignored. Take security awareness training, for example. A company with poor compliance priorities can miss its deadline for security awareness training with the employees who think they’re exempt (like executives or road warriors). These untrained individuals are not only a compliance miss, they may pose a security risk if they click on that phishing link or divulge sensitive information in a social engineering attack. A successful company will revoke access to the network and facilities once the deadline passes. These organizations tend to achieve 100% compliance on the training. Such compliance measures send a clear message about what’s really important.
2. Lack of documentation
Organizations generally do a good job of implementing controls and managing risk. Where many fall down is in not taking credit for the hard work they are doing. As auditors, we are trained to follow four key tenets of establishing evidence: inquiry (asking about a control), inspection (reviewing documentation supporting a control), observation (watching the control occur), and re-performance (repeating the control ourselves). Without documentation on a control or records regarding the performance of that control, we must rely on inquiry alone. However, inquiry is considered the weakest form of audit evidence, and under many compliance frameworks, isn’t alone sufficient. Without proof, we will assume that process is either not operating or being performed inconsistently. In addition, new or revised audit standards are becoming increasingly rigorous regarding the emphasis on specific documentation.
Most of the findings auditors identify stem from documentation failures. Luckily, this is easy to correct: companies should document what they’re doing in written policies, ensure everyone is trained in the proper procedures, and create a paper trail of the performance of the controls.
3. Human error compounded by too many manual processes
Consider this scenario: It’s four o’clock on a Friday afternoon. A sysadmin needs to disable a login for an employee’s last day. But he wants to head out for the weekend. What are the odds that he’ll remember to do this come Monday morning? Especially considering what Monday mornings can look like in IT? Leaving a terminated account live is a significant audit finding as well as a security risk.
A less error-prone process would be to connect the user authentication system to Human Resources and/or Payroll. When someone stops getting paid, their account is automatically locked. It reduces the chance of someone having to run a checklist and forget a vital step or miss a deadline.
4. Weak or missing risk assessment
Many organizations don’t understand or are intimidated by the phrase “risk assessment.” Consequently, they don’t do one, or they do an incomplete job. Most audit standards require a risk-based approach so that controls are focused on reducing the highest risks. Without a good risk assessment, organizations will waste resources on controls that don’t address highest risk. This means missing or skimping on vital controls that turn into audit findings or create unnecessary exposure for an organization.
While there is no definitive guide to which risk assessment is best, the COSO standard,4 which comes from the Committee of Sponsoring Organizations of the Treadway Commission, is a worthwhile starting point. Most audit standards, like COBIT5(Control Objectives for Information and Related Technologies), are built upon COSO. If that’s too much, organizations can start simply by putting themselves in their customers’ shoes: what would a customer want to avoid happening when working with your organization? This approach can provide information on the highest-risk scenarios. After the risk assessment, companies should mitigate high risks through controls or outsource the handling to a third party.
5. Internal assessment too self-congratulatory
A poor internal assessment trips up many organizations going into an external audit for the first time. People will naturally try to see themselves and others in the best light—it’s human nature. In the business world, this often means internal assessors overlook important shortcomings.
For example, an organization will walk through a PCI DSS self-assessment6 and give itself credit for adequately performing all 200+ controls. I’ll walk in and ask for detail on the same controls, and the organization will realize they aren’t actually implemented fully. Independence and objectivity are critical parts of an audit. Another problem stems from a poor risk assessment (see previous failure). If the organization has built its controls around a misaligned risk assessment, then all the internal assessment is going to do is reaffirm the original bad decision.
The answer is simple: develop a proper independent internal audit program—one that has a different reporting structure than the security and IT teams—or hire an independent assessor. Even a contracted consultant can fulfill this role, as long as he or she is segregated from the implementation of the controls.
6. Misunderstanding that some audits are ongoing not point-in-time
Some audits, like the PCI DSS, are a point-in-time audit, where an auditor visits once per year to review controls and documentation. However, many other audit standards, like SSAE 16/18 and Sarbanes-Oxley, cover a period of time during which your controls need to operate consistently during that entire timeframe. Sometimes organizations will not realize this distinction and cease control work right after the auditor walks out the door. When I show up again later, they’re confused. We did this once; why are you asking for it again? Then, these gaps in control activity become audit findings. As mentioned above in item 2, it’s also critical that companies retain all process documentation for the entire audit period.
Of course, there’s a bigger problem here. If an organization is only focused on making the auditor happy when he or she shows up, then it is ignoring the point of security: managing the risk. Controls implemented to reduce risk should never be point-in-time affairs, but an ongoing part of organizational culture. After all, the whole idea is to have a more secure organization—audits notwithstanding.
