Time for NetOps to Embrace Continuous Deployment

Continuous deployment doesn’t have to mean every change, every time, right away. But it does have to start somewhere.

CI/CD (Continuous Integration/Continuous Delivery) is the domain of developers. It’s the overarching model for improving speed to delivery – which really means “ready for deployment” for those used to seeing it in a more active, user-related context.

That doesn’t mean that NetOps can ignore CI/CD. On the contrary. But it doesn’t mean that NetOps has to maintain a 1:1 deployment to delivery ratio, either. The frequency with which apps are “ready for deployment”, especially if your app developers have perfected the art of continuous delivery, might be overwhelming, particularly if NetOps hasn’t perfected the art of continuous deployment quite yet.  But there is room to accommodate a greater frequency of deployment that in some cases will certainly taste like continuous deployment to developers and the business. 

The secret lies in the difference between “minor changes” and “major changes” to an app.

The NetDevOps Fall 2016 survey found that minor changes were being pushed into production with a great deal more frequency than we are often led to believe. Nearly 51% of respondents are pushing minor changes into production more than once a day. More than one in three were pushing major changes between 1 to 5 per time month, with another third confessing they made major changes less than once a month.

Unfortunately, the definition of “major” and “minor” changes is not quantified, but as is usually the case, such relative descriptors are often peculiar not just to an industry or organization, but often to each application. Still, it seems a good number – a little more than half – are deploying major changes in production on a very regular basis.

Which means that in the big scheme of things, we’re taking small steps in the direction of continuous deployment. The thing is, deploying a shiny new application is not the same thing as deploying updates to an existing application. Even if we’re adding new network or app services, it’s still way different on the “potential for disruption” scale than the deployment of a brand new application.

So let’s say for now, shiny new apps are going to require more coordination and planning. We’re still left with what are certainly the majority of apps in production as being potential targets for continuous deployment. One of the things we can do encourage that is to put some context around “minor” versus “major” changes with respect to its impact on production environments.

Minor changes might be restricted to those that impact the app, internally. Changes in logic, for example. They might include patches or upgrades to the app stack (web server, app server, etc…) as well as those to its supporting app infrastructure. Basically, minor changes impact only the app. They require no changes to the network or app services that actually deliver and secure that app in its typical use.

Major changes, then, would be those that impact anything in the network or app services supporting the app. Major changes might require tweaks to a delivery policy (turn on compression, would ya?) or the insertion of a new service, such as URL filtering or request inspection to prevent exploitation of a recently discovered vulnerability. These are major changes in that they impact a broader set of production systems that may or may not in turn impact other applications.

It might even be the case that you work out a more nuanced scoring system that takes into consideration the impact on external services. A new service is likely more disruptive than a tweak to an existing service. A new policy requires more attention to implementation than an existing one with a small update.

Once you’ve agreed on the “system” you’re going to use, it becomes a lot easier to agree to enable minor changes on-demand. Essentially you’re moving into continuous deployment by enabling app delivery to flow into production as long as it’s considered to have minimal potential impact if something goes wrong. Minor changes, minor risk. At least that’s the assumption.

What that does is enable minor technical changes that just might be major business changes to get into the hands of users faster. Because those are often held up in large, traditional organizations. Even a small change to the user interface might be held up for weeks – or more – due to scheduling conflicts and decisions made by change control boards based on project priorities or politics. Even though that small change might represent a better user experience that results in higher conversion or customer retention rates. If that small change impacts only the app, i.e. it’s a “minor” change, are the politics of IT really a reason to hold it up?

Continuous deployment isn’t just technically enabling automated pushes into production, it’s also about tearing down traditional constructs that hold up the processes determining when applications and updates make it into production (that’s culture).

By establishing a common criteria for differentiating between “minor” and “major” updates, between those that have the potential to impact systems outside the app and those that don’t, organizations can enable continuous deployment for some changes without incurring additional risk or destabilizing production. And it just might mean a positive impact to the business, which means everyone wins.