Six Sigma, the CMM, and the CMMI

Six Sigma, the CMM, and the CMMI

Six Sigma employs a measurement driven approach to continuous improvement.  It starts with quantitative business goals providing direct value to the customer. Data analysis is used to identify specific processes with the greatest leverage on these goals. Critical inputs affecting process performance are identified. Goals are related to changes in process outputs. Improvements are implemented on a pilot basis. If measurements indicate goals have been achieved, improvements are institutionalized. Controlling critical inputs controls process performance to the new levels.

The acronym for the Six Sigma continuous improvement methodology is DMAIC (Define Measure Analyze Improve Control). The DMAIC method is used in conjunction with the Six Sigma toolkit, a more or less standard set of statistical analysis techniques.

In some sense Six Sigma is a bottoms up method of continuous improvement.  Individual processes are selected based on their ability to affect business results and provide visible value to the customer.  The connection between process performance measurement and business is always explicit, so it is comparatively easy to measure return on investment.

Unlike the model based approaches to process improvement such as the CMM and the CMMI, Six Sigma is not domain specific.  Six Sigma does not include any process models.  It is a strategy to improve the bottom-line coupled with a measurement driven method for continuous improvement.  Six Sigma evolved out of the manufacturing world, a world with complex proprietary processes.  Consequently Six Sigma emphasizes method and a first principles analysis of current processes to a level that is quite alien to the CMM and CMMI. This emphasis on method and first principles analysis can be a both strength and a weakness when Six Sigma is applied to software development.

In the software world, processes are comparatively simple.  Usually the number of steps is quite small.   There are no issues associated with the variability of raw materials.  Given reasonable working conditions the influence of environmental factors is typically negligible.  The cost of measurement is low and the accuracy of measurements can be quite high if the developer use a simple automated measurement framework, however it can sometimes be quite difficult to perform designed experiments.   There is no need to deal with sampled measurements. There is an extensive published literature on many software processes that has been incorporated into the maturity models.

Frequently Black Belts that have been trained in a typical Six Sigma curriculum are not aware of these differences because the standard training courses draw all their examples from manufacturing and services.  When they apply their new analytical skills to software problems, they have a tendency to re-discover well known relationships and to re-invent well known optimizations of thoroughly understood processes like inspections.  In contrast, one of the great strengths of model based approaches is that they do a good job of making everyone familiar with existing best practices.  Model based approaches provide a strategic vision that is missing from a standard Six Sigma approach.

Six Sigma’s emphasis on measurement and relevance to the bottom line provide a key advantage relative to the typical model based approach to software process improvement.  Although both the CMM and CMMI emphasize the importance of business results, the need to make the coupling between process performance and bottom line improvement explicit is not present to the degree that it is in Six Sigma.

In fact the situation is usually quite the contrary. Since the model based assessments provide a level rating, it is easy for organizations to defer putting a measurement program in place until they reach the higher maturity levels and avoid the issue of demonstrating improved business results as a natural outcome of effective processes.  The net result is that organizations make huge investment in model based improvement without quantifying their returns.  Since it is quite possible to comply with the goals of the CMM KPA’s or CMMI PA’s with inefficient processes, many organizations move up to level 3 without getting the expected bottom line improvements. 

A widely accepted cliché is that Six Sigma isn’t appropriate for software until an organization is level 4 or level 5.  The Six Sigma toolkit aligns nicely with the quantitative process management, product quality management, and process optimization practices associated with level 4 and 5. Unfortunately many organizations never even reach level 4 or 5 because the maturity models allow them to avoid a sharp focus on the bottom line through level 3.  This would not be an issue if an organization could move to level 4 in a year or two, but typical organizations reach the higher maturity levels so slowly, if ever, that they loose focus along the way.

CMMI recognizes these issues to some extent.  It adds a new PA called Measurement and Analysis to level 2 making it difficult to defer putting a measurement framework in place early on. This brings it somewhat closer into alignment with Six Sigma at the lower maturity levels. CMMI also allows for two improvement representations, staged and continuous, theoretically making it less likely for organizations to try and move in rigid lockstep with the staged representation. But CMMI still does not make it necessary to establish the coupling of process performance to bottom-line performance at the outset.  It still allows maturity levels to substitute for measured performance improvements.

Combining Six Sigma at the tactical level with model based improvement at the strategic level provides the best of both worlds.  Six Sigma analysis techniques can be used to establish priorities for selecting individual processes for improvement within CMMI’s continuous representation or at every level of CMM’s staged representation.  DMAIC provides a systematic method from getting and sustaining measurable improvements in individual processes.  The process maturity model provides a strategic framework for continuous improvement, a perspective on industry best practices, and a systematic approach to benchmarking.  This avoids excessive analysis, re-inventing the wheel, and inadvertent sub-optimization

Six Sigma moves the organization away from level oriented goals eliminating one of the most common failure modes of model based improvement.  Levels become by products of bottom-line improvements, not goals in and of themselves.  Six Sigma makes it easy to measure ROI and to control processes so that they continue to perform at improved levels. The net result is organization’s start to see bottom line results soon in the process, sponsorship improves, and long-term success is much more likely.

Six Sigma techniques can produce software products of predictable quality at a predictable cost that meets the business and customer objectives.  Six Sigma can be used by software organizations at any level of CMM maturity. Organizations that delay dealing with Six Sigma until CMM level 4 may miss the opportunity to make their CMM effort more likely to achieve measurable business results