What’s the Measure of a Maturity Model?

In the Cloud Ops line of business at Pythian, a key area of concern for our team is maturity, and over the course of this year I’ve been asked to put thought and effort into the development of a maturity model. Digging into this topic, I’ve become intimately familiar with the Capability Maturity Model and its application to the iterative development of business processes. It attempts to evaluate and define the level of maturity of a given process based on some subjective criteria, which are supposed to represent the characteristics of a particular level of maturity. These levels are generally defined as “ad-hoc”, “repeatable”, “defined”, “managed”, and “optimized”, and they represent various levels of formalism that may not be necessary for the successful execution of the process in question.

What is Being Measured?

A capability is a thing that we bring to bear in order to achieve a particular business outcome or objective. Generally, this will take the form of some formal business process, and certainly the most common application of the CMM is towards the formal refinement of business processes in general. For me, this brings into question the value of such an evaluation; because the business should really only care about the outcome of the process, and not necessarily be hung up on the characteristics of the process itself. I found myself asking what we are trying to achieve with such a model.

When we address the formalism, it’s clear that the characteristic levels of maturity are ultimately intended to improve the outcome of the process, but it’s a statement of adherence to principles that should result in improved outcomes rather than a direct measure of the result it achieves. In its application we make the critical assumption that when we attain a specific level of process maturity, our outcomes are accordingly improved. How can we make this statement when our model is not measuring the outcome? The very framework of it quickly becomes dogmatic, and its value to the business completely subjective.

The above concern is not to say that a CMM is not a valuable tool, or that the qualitative characterization of our processes should not be undertaken. Let’s take it for what it is though, and that’s a guideline for the iterative improvement of a capability rather than a practical measure of its value to the organization. With this perspective, we may be prompted to ask some different questions about our processes. Rather than “How mature is our process?”, we may instead ask “How effective is our process?” and conduct ourselves accordingly. For example, the latter question will very likely lead us to measure our process outcomes directly.

Where Are the Results?

When we address process characteristics using the CMM, we find in the model a clear intention towards a scientific approach to process development. That is, the maturity of the process is evaluated in terms of characteristics that should result in increasingly repeatable, predictable, and measurable outcomes. Proponents of the scientific method however, would never tolerate a subjective mapping of process characteristics as a measure of its effectiveness. We can do better.

In my mind, the key here is to push towards our desired level of maturity right away rather than to step through the levels of formalism prescribed by the model. After all, why would we not set our target at the ultimate goal, which is an “optimized” level of maturity? This level incorporates the previous levels’ characteristics, but adds the incorporation of feedback into the process itself, which is without a doubt the secret sauce we need to achieve tangible improvement of outcomes. With this in mind, we should certainly incorporate the primary characteristics of each maturity level into the process, but let’s then measure the outcome of the process and begin optimizing from the first iteration.

Applying the Whole Model…

The main characteristics of maturity that the CMM prescribes are:

1. Repeatable ? This is the notion that a process can be repeated with some reasonable expectation of a consistent result. The intent is for the business itself to own the process without relying on a specific person’s skills, context or expertise for its execution. Should a person involved in the process leave the organization, the capability must remain with the business. A common approach to this requirement might be to require some level of skills certification, and to cross-train multiple resources in the execution of the process.
2. Defined ? The process is formalized in some way such that it can be iterated on in order to affect its ongoing improvement. Traditionally, this takes the form of business process documentation, however the modern approach would provide a repository and version controlled artifacts, including templates, documentation, automation, resources and tooling that defines and supports the process in a prescriptive manner. Building on the above objective of repeatability, this approach provides the business with a tangible focus for process development.
3. Managed ? Ongoing development of the process should be directed towards objective requirements that the process seeks to fulfill. As such, it behooves us to ensure that those requirements are identified and prioritized, and that the development of the process targets their fulfillment. These requirements should be defined in terms of the outcome of the process, rather than its characteristics, with the ultimate goal of providing a result that delivers some level of repeatable value to the business.
4. Optimized ? Our target level of maturity incorporates the scientific method to the process in terms of an objective feedback mechanism. We measure how reliably our process produces the desired outcome, and then modify the methodology to optimize the result. By continuously measuring our results and refining the process with each iteration, we become more accurate and efficient at achieving our target outcome, and realize the ultimate goal of creating a process that delivers maximum value to the business.

…Gives A Measurable Result

When we aggregate these elements and apply them all at once, the CMM loses its hierarchical formality and becomes prescriptive. By shifting the focus from a formal evaluation of process characteristics to a measurement of their outcomes, we’re essentially applying the scientific method to our processes. There’s no reason to rest on our laurels if we haven’t implemented some level of feedback and optimization… having a “defined” process that isn’t also “managed” and “optimized” demonstrates no measurable progress towards the goal of delivering a specific outcome. We have accomplished nothing concrete until we implement all of the aspects of the maturity model in our process to some degree and can actually measure the outcome and work iteratively to improve it.

In my next blog, we’ll explore the relationship between business practices and capabilities, and talk about the kinds of abstracted requirements that we want to target in order to develop a repeatable process to deliver against diverse customer requirements.