The Surprising Truth about Digital Transformation: Cloud Chaos
This is the second blog in a series on the challenges arising from digital transformation.
Cloud Chaos.
Some clever engineer is right now remarking that this phrase is redundant. Cloud is chaos, after all, with its lack of governance and laissez-faire approach to securing apps tossed into its fluffy white depths without a well-defined schedule.
Let us pause here to note that the chaos associated with cloud is most often a people and process problem, not a technical one.
That’s not to say there aren’t technical –or at least architectural – challenges associated with cloud. Some of them are related to the complexities of virtual networking. But most are architectural. One of those architectural challenges is related to very the nature of cloud with its per-application focus.
If you think about it, public cloud and its deployment model is considered the ideal (the ‘form of cloud’ as Platonists might call it) form of cloud. That means private cloud, too, should follow its precepts and exhibit the same characteristics.
That means APIs and self-service provisioning. Dashboards and web-based consoles. But it also means an architectural shift to a deployment that is purely focused on a single application.
Per-app architectures are the norm in cloud. Services are deployed and configured to support one app at a time, and create a singular data path from user to endpoint that traverse those services. This stands in stark contrast to traditional data center network design in which a significant number of network and application service infrastructure are shared.
It is the notion of ‘shared infrastructure’ that becomes challenging in a DevOps and Agile development world that is as per-app focused as public cloud.
Traditional, shared infrastructure, networks result in ‘one true path from the door to the app’. Challenges arising from this approach in today’s fast-paced, digital world are:
- Infrastructure version alignment. One app might require features provided by an upgrade or patch while another depends on the existing version. Shared infrastructure does not generally allow for diversity of versions on the same system, and thus can impede one application in order to favor another.
- Conflicting deployment schedules. When shared resources are impacted, enterprises begin a bargaining phase in which application stakeholders try to resolve conflicts in deployment schedules. The process can be long (and is always painful) and no one wins when a new app or update is held back because of scheduling conflicts.
- Blast radius. It goes without saying that if my app causes a shared system to fail, every app relying on that system fails. The blast radius for shared infrastructure is huge, and it is a significant factor in the reluctance to provide greater access to the deployment pipeline to those outside IT.
- Troubleshooting. Trying to paw through aggregated logs is painful. Very successful companies have been built on nothing more than the ability to mine massive logs and disaggregate the data back into per-application flows. Troubleshooting shared systems is complicated and time consuming and has a negative impact on the mean time to resolution, a critical KPI for modern IT.
In the cloud, none of this is typically a problem. There is one path per app, because apps are generally deployed on their own schedule in their own environment - often by different teams.
To save our sanity (and our data center) we need to adopt this approach on-premises, whether in a private cloud or not.
You are still going to have shared network services. Firewalls, IPS/IDS, DDoS, and user-inspection is very much neutral with respect to applications and can (and should) be applied on a corporate-wide basis. For everything else, there’s per-app (app-specific, if you prefer) services and infrastructure. This logical separation preserves the stability and security of the business while enabling the more volatile and perhaps unstable environment closer to the apps. It also preserves a data path for traditional (legacy and heritage) apps that don’t require the kind of speed and support needed for newer apps. The per-app network might be a private cloud, or multiple clusters of containers or some combination thereof.
The marriage of the two will result in a more stable, resilient network that is also flexible and fast.
A per-app approach to the network has a variety of benefits in addition to mitigating the issues arising from a shared approach coupled with modern app architectures and delivery models:
- Individual pipelines and schedules. I can schedule my own deployments when they’re ready because they impact no app or infrastructure but mine. This offers a flexibility to support the varying schedules you have to support in a modern data center.
- Restricted blast radius. If my deployment causes a crash on my dedicated systems, its all on me. It doesn’t impact any other apps. This dedicated, per-app data path approach further makes it easier to troubleshoot as I can be confident that logs and error messages on all systems belong to me and my application.
- Scale is not restricted to platform resources. In a highly volatile, dynamic world, it’s nigh unto impossible to know when the next rush of visitors will come or whether some content will go viral or, Heaven forbid, you’re being attacked. Whatever the cause, when demand spikes for apps on shared systems (hardware or software), scale is restricted to the resources available on the platform. Using a per-app architecture mitigates this constraint and enables individual components to scale out as necessary to meet demand.
- Supports true infrastructure as code model.Shared systems use shared configurations, even if it’s just a base configuration with per-app policies. That model can cause chaos when attempting to implement an infrastructure as code model, especially when multiple apps are trying to update at the same time. Changes may impact other apps without warning, or commit and clone actions may fail thanks to others locking resources when you need them. A per-app architecture ensures that deployment artifacts belong to one app only and can be managed properly by the app owners (whether dev, DevOps, NetOps, or ).
While a per-app architecture won’t solve every problem that arises from digital transformation, it can go along way toward taming the chaos arising from cloud. It offers greater opportunities for standardizing security as well as supporting the multi-cloud environments that most businesses are becoming.
Stay tuned for the next post in this series, in which we’ll dig into how you can deal with those that are Skipping Security thanks to go to market pressures arising from digital transformation.