One of the pillars of DevOps is—according to its founders—automation. Along with automation naturally comes orchestration, which as you might guess is automation but at a higher level of abstraction.
Where automation focuses on using scripts and APIs to codify some task, like “add a rule to the firewall,” orchestration encompasses processes that span internal demesnes, like “enable access to this app from the public Internet.”
Both are equally important to digital transformation efforts being undertaken by most organizations today. That’s because apps are key to digital transformation, and with every app comes a commensurate amount of responsibility (a.k.a. work) across IT to not only get it into production, but also keep it available, fast, and secure. That means deploying, configuring, and managing network and app services.
Now, even with security budgets rising over the past few years, there remains a significant talent shortage in both security and cloud arenas. That means that even if it were feasible to address growing workloads—spoiler alert, it’s not—it’s unlikely that an organization would be able to find and train the personnel needed to keep up with the fantastical growth occurring today. So, scaling operations to keep up with all the apps and those critical services is a significant challenge.
It turns out that operational scale is one of the driving forces behind the increasing adoption of automation and orchestration in enterprises today. That’s not just me talking, that’s the conclusion of over 2100 IT professionals that responded to our State of Application Delivery survey. Of all the options available, scalability and OpEx reduction were the top two. The often-cited mantra of “improving time to market”? A distant third.
Organizations are adopting frameworks, tools, and technologies that enable automation. Globally, organizations employ an average of 1.86 frameworks to realize the scale and speed necessary to compete in today’s digital economy. And the more apps a business manages the more frameworks it tends to use. The market is already seeing its targets slowly solidify, as some frameworks tend to be more popular with different constituencies within IT. There’s also merges and acquisitions to take into consideration that bring different technologies and architectures into the fold. These factors mean multiple frameworks are often in use. A non-trivial 12% of respondents in our survey indicate they’re currently using four or more frameworks today.
Now, usually these data points would lead naturally into a discussion of SecOps—the portmanteau that brings Security and DevOps together under a single, automated umbrella. And while that’s an important topic, it’s not the topic we’re going to segue into today because there’s another, more nefarious issue growing with all this adoption of automation and orchestration in the enterprise: risk.
Automation Risks
There are a number of container and automation frameworks out there that seek to make scale as effortless as the click of a mouse. Some of them are rising quickly due to the excitement over containers, like Mesos and Kubernetes. Others have been around for a while—think Puppet, Chef, VMware, Cisco, and OpenStack.
The thing we sometimes overlook is that these tools are software. They’re apps, like any other. And as such, they’re bound to exhibit the same characteristics of other apps, namely security risks.
Consider this dive into Mesos and its rather laissez-faire authentication requirements:
The thing is, the default upon installation of Mesos at this time is to have no—I repeat no—authentication for frameworks, and no limitation on what those frameworks can see. If you use command line switches, you can modify this default behavior, but the default is still to be wide open where frameworks are involved.
https://devops.com/mesos-security-awareness-considerations/
Lest you think I’m picking on Mesos, I’m not. Similar issues are cropping up in other container orchestration frameworks, as well. And I suspect that if we dig into others, we’d find even more tidbits that would give security professionals palpitations. The point is not to pick on frameworks, but rather to ensure that you’re aware of the risks posed by operational apps inside the data center. Because one of the drawbacks of automation is that it’s hard to pull back on the reins once it kicks off. An errant command can propagate faster than mosquitos in a Florida swamp. Lacking controls on frameworks designed to scale and manage the infrastructure necessary for critical apps, the potential for significant damage to be wrought is huge.
After all, if I can take out one server with an API call, how many can I take out with a system that automatically propagates that call? And what else can I do? Force system updates? Change access rules?
As noted in the discussion on Mesos, there are options to enable controls and reduce the risk of an internal framework accidentally (or intentionally) being used to wreak havoc on infrastructure. As is likely true for most frameworks and systems being put into place to automate and orchestrate critical IT systems. But they often require action on the part of implementers. The key, then, is to be aware of the use of these frameworks and to audit their implementations and ensure proper controls have in fact been enabled in the first place.
The propagation of automation and orchestration frameworks to realize the economy of scale necessary to survive a digital transformation is inevitable. Along with them comes ——a familiar set of risks that security professionals need to not only be aware of, but also take the initiative to ensure they are addressed before they become an existential threat.
Set aside some time to inventory the frameworks in use today in your environment and understand how they’re being used as well as what (if any) security controls are being employed to ensure that risk doesn’t become an existential threat.
