To quote the great master Yoda, “Pass on what you have learned. Strength, mastery. But weakness, folly, failure also. Yes, failure most of all. The greatest teacher, failure is.” Taking his advice to heart, we asked security leaders about their past failures and the lessons they want to pass on to others. I’ll start off with one of my own failures.
Never Assume Things are Working Right
For me, I made the mistake of not focusing enough on the basics, such as asset discovery, when I began building a new security program. This has become an important mantra for me, as I’ve written about in To Protect Your Network, You Must First Know Your Network.
The specific failure is how one of my largest, eventually most successful security programs nearly went right off the rails in the first six months. I made the amateur mistake of taking what I was being told by staff at face value instead of double-checking it myself. I knew that some developers had access to customer data, which I shut down as soon as possible. I had been told this meant their access to the data was revoked, but within a few months, my folly was revealed. Database dumps of customer data were still sitting around on developer workstations, unencrypted and uninventoried. Worse, network access control rules restricting developer access to customer data had some notable gaps in coverage for some subnets.
What followed was a panicked stampede of laptop file inventory scans and a full network rules audit. Eventually, I added Data Leak Prevention (DLP) tools as well as automated network access list audits. No, you can’t check everything yourself, but when it’s something as important as critical data asset inventory and access controls, you need to know the ground truth.
Get Help, Especially in the Beginning
Erik Pierson, Director of Information Security at Slalom Consulting, said he regretted "Not bringing in outside consulting help early on to help set objectives and plan our program. We eventually got to where we needed to be, but it took longer than I had hoped."
We eventually got to where we needed to be, but it took longer than I had hoped.
You never want to go it alone in cyber-security. It’s a hard enough job with a fully-functional team backing you up. And in security, governance is just as important as technology. Consulting help in setting up a security program for the first time can save a lot of frustration and dead ends. Choosing realistic objectives and measures, matching controls to risk, and laying out the roadmap are all things a consultant can help with. Having an outside perspective is also useful just to have a fresh set of eyes on the problem. One thing to remember though, you can use consultants and advisors to help you build a strategy. However, just as the risk burden is yours alone, so ultimately is ownership of the strategy to manage it.
Assume Breach Means Detection and Response
Mike Hamilton, Founder and CISO of CI Security and former Chief Information Security Officer for the City of Seattle said, “If I could do it all over again, I’d have much more of an emphasis on detection and rapid response. The realization that preventative controls will fail against a determined adversary must be coupled with a focus on monitoring the network and hunting for aberrational behavior. In this way, risk is managed by minimizing the impact of what is now a foreseeable event.”
I’d have much more of an emphasis on detection and rapid response.
We’ve talked about planning for the eventuality that attackers are going to get through your defenses, otherwise known as the assume breach principle. Mike brings up the implication of that principle: you need to be watching and ready to respond when it happens. In the old paradigm of the supposed “impenetrable perimeter,” monitoring and response was secondary and therefore not very robust.
However, given the state of technology and security, it’s a better investment to build near real-time detection capability and have trained, capable incident responders ready to contain and neutralize any attacks. A step further is to have teams constantly scanning and reviewing internal logs and assets for signs of existing intrusion.
Have Understanding Beyond Technology and Security
Mary Gardner, CISO of F5 and former CISO of Seattle Children’s Hospital says, "I would have loved to have better understood more about the politics of the environments before I started.” While this may not have not have changed her decisions about what should have been done, she adds, “understanding the politics/culture/power dynamics means you can move through projects and decisions in a much better manner.” She notes that “Politics influences a lot of what InfoSec can do,” and her experiences at different orgainzations over the years gave her a lot of new insights on how to manage them.
Understanding the politics/culture/power dynamics [of your environment] means you can move through projects and decisions in a much better manner.
Ultimately, it is the obligation of the security team to understand the organization, not the other way around. CISOs must speak the language of the business and its culture. Without this, a lot of time can be wasted on fruitless efforts to mitigate risk and implement control processes. This can mean taking the time in the beginning to get the lay of the land and learning to speak to the organization in a manner that is most familiar. Some CISOs would prefer to rush in and implement controls, but Mary cautions that listening and understanding is a better first move.
What’s Your Fail?
No doubt there are things that have happened in your career that you wish you could do over. Nobody gets it right the first time, and there is always room for improvement. I encourage you to reflect upon your own past and pass on what you have learned. After all, there is a reason why we call these security practices.
