What Your Pipelines Say about your Organizational Structure

One of the "laws" associated with DevOps belongs to a programmer named Melvin Conway. His law was introduced in 1967 and simply states "organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations".

I have emphasized the word "systems" because too often Conway's Law is applied only to apps. To software systems. But the reality is that "systems" encompass everything from apps to the integrated pipelines that deliver and deploy them. Your pipelines.

When we talk about embracing DevOps methodologies on the deployment side of the house (that's production) we need to also be aware of associated principles like Conway's Law. Because that law applies to the design of deployment pipelines just as it does the apps it pushes to market.

You may recall from our 2019 State of Application Services two questions. The first asked about the organizational structure of IT - combined operations, single-function, or cross-functional teams. We found that nearly half (46%) of the more than 2000 respondents were organized as single-function teams. Combined operations came in a respectable second at 37%. Cross-functional teams are less common, with only 15% of respondents operating in such structures.

Now that's important as we shift to examine the state of deployment pipeline automation. Because the results are definitely a reflection of the domination of single-function teams.

The data here is showing an inconsistent automation effort that isn't going to help realize the goal of nearly half (48%) of respondents with respect to digital transformation - that of deploying apps faster and more frequently. The disparity between these established domains within IT is existential proof that Conway's Law applies to any system, anywhere.

Digging deeper we find that just over one in ten (11%) have automated only one of these domains. One in four (25%) have managed to automate two or three domains. And just over one in five (21%) have automated all four primary domains necessary to complete a fully automated deployment pipeline.

A significant percentage - 42% - have automated exactly ZERO of these domains.

The inequity of automation across the deployment pipeline is an important piece of the time to value puzzle because each time you encounter a manual process in the pipeline you incur a delay. That delay slows down time to value (or time to market, if you prefer). The differences in rates shows the impact of organizations remaining in a "single function team structure" because we’re seeing individual "silos" automated with little or no concern for how it interacts with the rest of the pipeline.

This is why combined operations or cross-functional team structures are preferred. Because the communication channels between IT domains becomes a peer process that better facilitates the design of a system that spans multiple concerns. When a pipeline (system) is designed by a team, they are able to take into consideration the pipeline as a whole rather than as its composite pieces.

The work of automation itself might be (and probably should be) handled by domain experts. But the overall design and integration methods (APIs) need to be designed in an open, collaborative team Otherwise we end up with automation silos that may or may not serve the business goal of getting to market faster and more frequently.

So if you're running into delays or detours in your attempts to automate production pipelines, take a step back and consider the organizational structures in place and how they impact the design of that pipeline. You may find that before you can effectively automate anything you need to push for more effective organizational structures to support the effort.