Maslow’s Hierarchy of Automation Needs
If there’s anything hotter than containers right now, it’s probably something based on containers, like serverless. In addition to being the “cool” kid on the block right now, it’s also the focus of just how important automation and orchestration* is to modern applications.
Just about every day brings some new component to container architectures and, with it, the appropriate level of automation. If something can be automated in a container world, it will. A plethora of APIs and a reliance on configuration/template-based artifacts drive the near-continuous evolution of automation within container-based environments. If the API economy is important to business’ digital transformation, then the other API economy is critical to IT’s digital transformation.
Container environments are no exception, and as they continue to make their way into production environments, it becomes increasingly important for upstream (N-S) services to not only be able to deliver the apps being scaled and served from within these highly volatile environments, but to integrate with them at the automation and orchestration layers.
That’s because some of the layers of this “stack” of technology are more volatile than others. Container creation and destruction, for example, happens on a far more regular basis than that of whole clusters being created and/or destroyed. Indeed, Data Dog HQ’s Docker Adoption report notes that “as of March 2017, roughly 40 percent of Datadog customers running Docker were also running Kubernetes, Mesos, Amazon ECS, Google Container Engine, or another orchestrator.” Automation/orchestration is critical to container success, particularly at its core. That means that as we’re moving along, it is imperative that some automation needs are met as quickly (and thoroughly) as possible, because they’re the foundation for everything else.
This is true for cloud environments, as well. It isn’t just volatility that drives the need for automation and orchestration, after all, it’s also agility. The complex “self-service” model associated with cloud computing demands automation to support the notion of cloud computing’s elastic, on-demand architectures within a utility computing model. There is an identifiable set of automation needs common to both cloud and containers that lay the foundation for understanding its value to the business.
Much like Maslow’s Hierarchy of Needs, the Hierarchy of Automation Needs builds upon the premise that there are “deficiency” needs which must be met in order to progress upwards toward higher-order needs that enable growth.
One must satisfy lower level deficit needs before progressing on to meet higher level growth needs. When a deficit need has been satisfied it will go away, and our activities become habitually directed towards meeting the next set of needs that we have yet to satisfy. These then become our salient needs. However, growth needs continue to be felt and may even become stronger once they have been engaged. Once these growth needs have been reasonably satisfied, one may be able to reach the highest level called self-actualization. -- https://www.simplypsychology.org/maslow.html
At its core, this particular Hierarchy of Automation Needs focuses on foundational needs around scaling applications and their composite services. As the very talented F5 engineer who first presented this concept explains: “The automation at the bottom is most valuable because it happens the most and is crucial to an application staying online. The automation at the top is least valuable because it happens less frequently (maybe only "once") and is happening at the same time that many folks will be doing manual things already.”
These are the “basic” needs of automating the creation/destruction of containers/virtual machines and apps is critical to not just scale but efficacy of scale. That efficacy is important to constraining costs and alleviating the burden of scale traditionally laid on (costly) manual operations. Given the volume and frequency with which such events occur today, automation is a requirement, not a nice to have. It is a deficiency need, which means if it is not met, there’s little reason to worry about higher-order, less frequently occurring automation.
This is especially true for containers, and seems validated by Data Dog again when it notes “containers have an average lifespan of 2.5 days, while across all companies, traditional and cloud-based VMs have an average lifespan of 23 days.” This is, it seems, impacted by automation. “In organizations running Docker with an orchestrator, the typical lifetime of a container is less than one day. At organizations that run Docker without orchestration, the average container exists for 5.5 days.”
The ability to automate the creation/destruction of containers and subsequently apps is imperative to realizing the speed and agility of scale required for organizations to reap the benefits of containers both in traditional and cloud-based environments.
The higher order automation needs are pretty much routing needs and fall into what are growth needs. Growth needs are sought once basic (deficiency) needs have been met. Cluster creation/destruction and routing changes happen infrequently and only become valuable once the app is in place and able to scale. This becomes imperative once the environment migrates from dev/test into a production environment and is relied upon to deliver business value by serving up applications. After all, routing to an app that can’t respond is like giving a hug to a starving man. The warm fuzzy feeling doesn’t address the basic need for food. A 2016 Mesosphere survey of its users found that 62% of them were already using containers in production environments. A 2017 Portworx survey at container-focused conference DockerCon found that 67.2% of its respondents were using containers in production. Which means the routing needs are quickly becoming important, at least to the subset that is container-adopting organizations.
The ultimate goal of business’ self-actualization – growth – cannot be achieved until the entire hierarchy is fulfilled; that is, until the entire “stack” is automated and orchestrated. So it should be a given that those who primarily live on the N-S data plane will orchestrate from the bottom up, enabling scaling needs of the environment first and working up to the routing needs, where the transition from N-S to E-W traffic takes place. This is evident as well in the continued development of the more tightly coupled ecosystem around container orchestrators themselves, such as Kubernetes. One of the more recent developments has been focused on the “routing” needs with the inclusion of ingress controllers that route at layer 7 (URI / API) to ensure the N-S transition to E-W services takes place seamlessly. That component, too, must eventually be integrated (automated) to satisfy routing needs and continue to propel organizations up the hierarchy to realize growth.
Translating this to container-specific constructs enables us to map the container environment automation needs to those services in the network that support its need to scale to realize growth. Both the container and network/application service constructs require automation. Creating a hundred containers to scale an app without simultaneously automating the services responsible for managing them during delivery yields unsatisfied needs. These networking constructs can exist as either native container constructs or as existing app services adapted to fit natively into the container environment. Implementation details will vary based on architecture and operational requirements, though the existence of both sets of constructs is necessary to satisfy basic (scale) and growth (routing) needs to realize success.
In all cases, the success of containers and cloud are heavily reliant on their ecosystems and that means reliance on the Other API economy to enable the automation and orchestration critical to business growth through digital transformation initiatives.
* I find it frustrating that the concepts of automation and orchestration seem to be conflated within the container world, juxtaposing one for the other. But for now, let’s just ignore that and I’ll try to keep the pedant inside where it belongs. Inside my head. Screaming. Desperately.