Dec 19, 2017 - Full-Text Paper (PDF): Agile Software Development and CMMI: What We Do Not Know about Dancing with Elepha...
Agile Software Development and CMMI: What we do not know about Dancing with Elephants 1
Célio Santana , Cristine Gusmão1, Caryna Pinheiro 2 , Liana Soares1 ,Teresa 4 Maciel3 and Alexandre Vasconcelos 1
University of Pernambuco, Departament of Computer and Systems, Benfica St. 455, 50720-001 Recife, Pernambuco, Brazil 2 University of Calgary, Departament of Computer Science, Calgary, Canada 3 Federal Rural University of Pernambuco, Informatic and Statistic Departament, Dois Irmãos avenue. S/N, 52171-900 Recife, Pernambuco, Brazil 4 Federal University of Pernambuco, Informátic Centre, Prof Luiz Freire Avenue. S/N,50740-540 Recife, Pernambuco, Brazil [email protected]
, [email protected]
, [email protected]
, [email protected]
, [email protected]
, [email protected]
Abstract. In this paper we discuss how the merging of Agile Methodologies and Software Quality Models in same process today is ignoring many important aspects of both approaches. The inconsideration of these points results in a rigid integration of Agile and Quality Models that limits the full potential of their synergies. Ignoring such important items however does not necessarily means that they are not being utilized in the process, it normally indicates their utilization in an ad-hoc way. To explore this topic, we collected qualitative and quantitative data from literature and two Brazilian companies which work with agile and XP. Keywords: Capability Maturity Model Integration, CMMI, Agile Software Development.
1 Introduction Long before the agile paradigm, in fact since 1968, software engineers have tried to emulate traditional engineering in order to address quality problems with varying degrees of success. Since the ‘80s and ‘90s the emphasis has gradually started to shift from software product improvement to software process improvement , with the appearance of quality standards and guidelines such as: ISO 9001 , software process improvement, SPICE / ISO-15504  and capability maturity model integrated (CMM/CMMI) . The latter became the most recognized software quality model, turning into a mandatory requirement for bidders in the outsourcing of IT services. In 2001, with the creation of the Agile Manifesto , the Agile movement proposed a shift in the values of software development process from the mechanistic
(i.e., driven by process and rules of science) to the organic (i.e., driven by softer issues of people and their interactions) . Schuh  defines Agile development as a counter movement to 30 years of increasingly heavy-handed processes meant to refashion computer programming into software engineering, rendering it as manageable and predictable as any other engineering discipline. Boehm and Turner  define that agile on a practical perspective, emerged from a common discovery among practitioners such as that their practice had slowly drifted away from the traditional heavy document and process centered development approaches to more people-centered and less document-driven approaches. Despite the changes in traditional software engineering presented by the Agile methods, in 2001 one of the CMM developers, Mark Paulk, stated that CMM and an Agile method called Extreme Programming (XP)  contain ideas that can be synergistic . Since then, many papers were published adapting Agile to CMMI, following a similar line to that one proposed by Paulk. This paper provides, through the analysis of the quantitative and qualitative data gathered in literature and industry, a further contribution by bringing to attention the relevant disregarded elements when merging Agile and CMMI, as well as presents the consequences of implementing this incomplete merged methodology. After this introductory section, section 2 explores Agile Methods. In section 3, CMMI is explained; section 4 shows how CMMI and Agile are being merged today; section 5 shows what is fully or partially unknown about this merging and section 6 shows a summary and future works.
2 Agile Methodologies Although some of the agile methods have existed in one way or another for a decade or two, the term “agile method” was coined more recently, in February 2001, by seventeen of the leading developers and proponents of what were then known as the “Light” methodologies . These individuals met “to see whether there was anything in common among the various light methodologies” . The meeting resulted in four levels of agreement among the participants : 1st Level: There was a need for methods designed to respond to change during software projects. Thus, they adopted the term “Agile” to identify such methods. They agreed that the term “Light” was not appropriate because certain projects would not employ a “Light” methodology but could still require agility. 2nd Level: The four statements of the “Agile Manifesto” . These four statements capture the core values on which all of the Agile Methods are built, as well as the spirit in which they should be implemented. 3rd Level: The next level of agreement was on a set of 12 Agile Principles. In these statements, the values are fleshed out in more detail and given more concrete meaning. 4th Level: The final level of agreement, which was to be a more detailed with actual activities or tactics to running projects, was beyond their grasp at the time.
They were content to leave that fourth level for each agile method to define in its own way. Some examples of Agile techniques are Extreme Programming (XP) , Scrum , Crystal  and Lean .
3 Capability Maturity Model Integration The Capability Mature Model Integration (CMMI)  is defined as a process improvement maturity model for the development of products and services. It consists of best practices that address development activities that cover the product life cycle from conception through delivery and maintenance. There are 22 process areas. A process area is a cluster of related practices that, when implemented collectively, satisfies a set of goals considered important for making improvement in that area . Each process area can be summarized to illustrate their components, as shown in Figure 1.
Fig. 1. CMMI Process Areas and their Components .
CMMI enables one to approach process improvement and appraisals using two different representations: continuous and staged. The continuous representation enables an organization to select a process area (or group of process areas) and improve processes related to it. This representation uses six (0 to 5) capability levels to characterize improvement related to an individual process area. The staged representation uses predefined sets of process areas to define an improvement path for an organization. This improvement path is characterized by five (1 to 5) maturity levels. Each maturity level provides a set of process areas that characterize different organizational behaviors. Figure 2 shows a summary of the target profiles that must be achieved when using the continuous representation to be equivalent to maturity levels 2 through 5. Each
shaded area in the capability level columns represents a target profile that is equivalent to a maturity level. The following rules summarize equivalent staging: To achieve maturity level 2, all process areas assigned to maturity level 2 must achieve capability level 2 or higher. To achieve maturity level 3 or higher, all process areas assigned to that specific maturity levels and lowers must achieve capability level 3 or higher.
Fig. 2. Target Profiles and Equivalent Staging .
And a short description of each capability level follows: A capability level 0 process is defined as an “incomplete process”. A process that either is not performed or partially performed. One or more of the specific goals of the process area are not satisfied, and no generic goals exist for this level. A capability level 1 process is defined as a “performed process.” A performed process is a process that satisfies the specific goals of the process area. Although capability level 1 does not address all generic goals at capability 2. A capability level 2 or higher process is a performed (capability level 1) process that address all the generic goals for that capability level. The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) method is used for appraising organizations using CMMI, and one result of an
appraisal is a rating . If the continuous representation is used for an appraisal, the rating is a capability level profile. If the staged representation is used for an appraisal, the rating is a maturity level rating.
4 Agile and CMMI: What do we Know? 4.1 Mapping techniques of Agile Methods to CMMI Specific Practices The first mapping between Agile and CMM was conducted by Paulk  fitting XP to SW-CMM level 2. Since then, many reports following Paulk’s ideas were published [18, 19, 20, 21, 22, 23, 24, 25, 26]. These mappings refer to the low levels (two and three) of CMMI and CMM. This paper also presents the research conducted in two companies using Agile and following CMMI patterns. Their names must remain anonymous due to confidentiality reasons. Company 1, located in Recife, Brazil, has eleven years of experience in the market and has recently been evaluated for SCAMPI to CMMI level 3 and uses Scrum in sixteen projects. Company 2, also located in Recife, Brazil, is a small-sized company with nine years of experience in the market which will be evaluated at CMMI level 2 using XP in a pilot project. They will be referred in the following sections to provide an industry insight on the subjects. Both companies conducted a gap analyses to understand the misfits between CMMI and Agile. These gap analyses are also the mapping of each company about agile and its relationship with CMMI.
4.2 Traditional and Agile Software Process Improvement For Salo & Abrahamsson , “in traditional SPI approaches the promoter of SPI initiatives is the organizational level and the goals set in it. The organizational learning cycle addresses the setting of the improvement goals, and planning and analyzing the results of the SPI initiatives. The project learning cycle addresses the execution (e.g., piloting) and collecting feedback from SPI initiatives (i.e., projects) for organizational analysis”. Salo & Abrahamsson  further stated that there are two central differences between traditional and agile SPI that can be identified. Firstly, the origin of SPI goals is traditionally from the organizational level. However, in the agile project context the impelling power for SPI lies within individual project teams, and their experiences and learning throughout projects. Secondly, the process knowledge of project teams has traditionally been harvested from finished projects in order to enhance project work in the future. Instead, the immediate focus of agile project learning is on improving the performance of the ongoing project. The traditional and agile SPI approaches are not contradictory. Rather, both the approaches can be beneficial in integrating agile software development and
organizational learning. This, however, requires the integration of agile SPI mechanisms in the organizational learning cycle. The companies under study (company 1 and 2) structured their Software Engineering Process Group (SEPG) outside agile teams, following the traditional SPI approaches .
5 Agile and CMMI: What we do not Know? 5.1 CMMI Support for adopting Agile According to many reports [28, 29, 30, 31, 32], companies using CMMI provide support to agile making it an easier and smoother methodology to adopt than in non CMMI companies. In the meantime, ,  and  stated ideas similar to the ones raised by Paulk . In [31,32] raise theoretical questions about merging CMMI and agile. Nevertheless,  and  stated that even presenting similar ideas, merging Agile and CMMI could be a difficult experience, since there are other aspects seen in a different perspective. Such approaches may become an issue to make this perspective a convergent one. This topic looks contradictory because the support provided by CMMI to any methodology come from its generic practices (see Fig 1). And there is only one published paper  about this subject. Sutherland et. al.  stated how generics practices at capability 2 and 3 could support and improve Agile. They recommend to the CMMI community that agile methods can fit into your CMMI framework and will provide exciting improvements to your organization. This paper shows how generic practices support Agile in a general way. CMMI requires that the generic practices must be applied to all process areas. It is not possible to identify with precision which process area the generic practices are referred to. To the Agile community this is the first, and unique, report about this subject and it is necessary to raise awareness that more research is required about this topic. The companies under study (company 1 and 2) ignored how Generics Practices support agile. 5.2 Merging CMMI Specifics Practices and Agile, is still agile at all. Mnkandla & Dowlatzky  define the value of agility in “allowing the concepts defined above to mutate within the parameters set by the agile values and principles” . Boehm and Turner  defines that “all agile methodologies follow the four values and 12 principles as outlined in the agile manifesto.” None of the researches mentioned in section 4.1, neither companies 1 and 2 considered the values and principles stated in Agile Manifesto.
As a matter of fact, there is no mapping that provides a complete adjustment between Agile and CMMI. Thus, it is necessary to complement Agile with other practices, in which there is no guarantee to be following values and principles from the Agile Manifesto. Quantitatively, there are two reports,  and , which presented evolution of indicators of quality and productivity with the adoption of Agile merged in a CMMI environment. Sutherland et. al.  stated: Scrum reduced every category of work (defects, rework, total work required, and process overhead) by almost 50% compared to our previous CMMI Level 5 implementation. When comparing the projects using Scrum to the current productivity baseline it is seen that productivity for large projects shows a 201% increase in productivity. That large projects in Systematic using Scrum will double productivity going forward. The project finished early, and reduced the number of coding defects in final test by 38% and remaining coding defects in final test were reduced by 42% compared to previous processes. Price was reduced by 50%.when reevaluate the requirement specification Gravis & Jarvistock  Stated: Defects reduced by 66%, Critical defects reduced by 79%; Duration (in days) reduced by 44%; Effort (in hours) reduced by 47%; Disconnection Between Business and Technology reduced by 38%; Quality of Life improved by 81% business and 77% technology. These two reports show a positive result of merging Agile and CMMI, but, two companies represent a very small part of the software industry which uses Agile and CMMI. So these results are statistically insignificant in this universe but they are a nice start point to research in which cases Agile and CMMI could work fine together.
5.3 Aspects almost totally unknown.
Adoption of statements and values of Agile Manifesto.
There is no data in the literature addressing the adoption of values and statements of Agile Manifesto. Neither companies 1 or 2 considered the Manifesto on their processes. In this way, there is no data about how Agile Manifesto could be adopted in CMMI context.
Higher maturity levels of CMMI are close or deviate of Agile
The mappings seen in section 4.1 show to the deviation between agile and process areas and specific practices of CMMI when the maturity level is
raised to level 3. However, Sutherland et. al.  stated that generics practices of level 3 could improve agile. The lack of precise data on how much closer/far are Agile, specifics practices and generics practices makes it impossible to reach a conclusion about this subject.
Influence of the SCAMPI in merging CMMI and Agile
There is a belief that SCAMPI appraisal requires evidences which could be no necessary in CMMI process. But there is no data to suggest which process areas, goals or practices are influenced by SCAMPI. This does not means that SCAMPI affects or not an Agile-CMMI process, but there is no knowledge about when and where this influence is stronger.
6 Conclusion Merging Agile e CMMI is neither strange nor innovative, many reports from both academia and industry show how Agile and CMMI could live harmoniously in the same environment with few quantitative results indicating positive results when these approaches work together. Although, the union between low levels of Agile Manifesto (agile methods) and CMMI (specific practices), does not provide many options for companies which want to use Agile and CMMI. Ignoring the higher levels of Agile Manifesto and the others components of CMMI, the companies could not see other opportunities to improve their process, strengthen the union of the approaches and solve commons problems. There are few quantitative reports to give neither a definitive opinion about this “incomplete” way of merging Agile and CMMI nor to define in which cases that merging could bring better results. Finally there is a few diversity of researches dealing with merging Agile and CMMI. Most of them are mappings where specific practices of the following process areas of CMMI level: Project Planning (PP), Project Monitoring and Control (PMC) and Requirements Management (REQM) are merged with Scrum or XP. This factor leads to a false impression of full understanding about the subject. 6.1 Future Works.
Make further research on how CMMI Generic Practices could support and improve Agile, and establish a “distance” between Agile and Generic Practices; Compare the “distance” between Agile and Generic Practices and compare it to the distance between Agile and Specific Practices provided by the mappings; Observe the behavior of these distance when higher maturity levels of CMMI are achieved ;
Collect indicators such as quality, productivity, schedule and costs after the merging and verify their evolution comparing with themselves before the merging; Investigate the interference of the SCAMPI in Agile-CMMI process. Investigate how the Agile Manifesto could be considered in this union.
References 1. Georgiadou, E.: Software Process and Product Improvement: A Historical Perspective. Cybernetics and Systems Analysis. 11(4), 125--142 (2003) 2. ISO 9001:2008. Retrieved January 11, 2009, http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=46486 (2009) 3. Dorling, A.: Spice: Software process improvement and capability determination. Software Quality Journal, 2(93), 209--224 (1993) 4. Paulk, M. C., Curtis, B., Chrissis, M. B.: Capability maturity model (Version 1.1). IEEE Software, 10(4), 19--27 (1993) 5. Agile Manifesto.: Agile Manifesto for Software Development, http://www.agilemanifesto.org/ (2001) 6. Mnkandla, E., Dwolatzky, B.: Balancing the human and the engineering factors in software development. In: IEEE AFRICON 2004 Conference, pp. 1207--1210 (2004). 7. Schuh, P.: Integrating agile development in the real world. MA: Charles River Media, pp. 1-6. (2004) 8. Boehm, B., Turner, R.: Balancing agility and discipline: A guide for the perplexed (1st ed). Addison-Wesley. pp. 165--194 (2004) 9. Beck, K.: Extreme Programming Explained: Embrace Change. Addison Wesley. (2000) 10. Paulk, M.: “Xp from a CMM perspective. IEEE Software 18(6), pp 19--26. (2001) 11. Koch. A. S.: Agile Software Development - Evaluating the Methods for Your Organization. Artech House, Boston (2005) 12. Cockburn, A.: Agile Software Development. Addison-Wesley, Reading (2002) 13. Schwaber, K., Beedle, M.: Agile Software Development with Scrum. Prentice Hall (200) 14. Cockburn, A.: Crystal Clear: a Human Powered Methodology for Small Teams . AddisonWesley, Reading (2002) 15. Poppendeick, M., Poppendeick, T.: Lean Software Development, An Agile toolkit for software development managers. Addison-Wesley, Reading (2003) 16. Chrissis, M. B., Konrad, M., Shrum, S.: CMMI®: Guidelines for Process Integration and Product Improvement. Addison Wesley. (2003) 17. Ahern, M., Armstrong, J., Clouse, A., Ferguson, J., Hayes, W. Nidiffer, K.: CMMI SCAMPI Distilled: Appraisals for Process Improvement. Addison Wesley. (2005) 18. Nawrocky, J., Walter, B., Wojciechoeski, A.: Comparison of CMM Level 2 and eXtreme Programming In Lecture Notes in Computer Science of Software Quality. (2002) 19. Martinsson, J.: Maturing XP through the CMM. In proceedings of the Extreme Programming and Agile Process in Software Engineering. (2003) 20. Reifer, D.: XP and CMM. In Innovative Technology for Computer Professional Magazine. pp. 14-15. (2003) 21. Vriens, C.: Certifying for CMM Level 2 and ISO9001 with [email protected]
In proceedings of the Agile Development Conference. pp. 120--124. (2003) 22. Bos, E., Vriens, C.: An agile CMM. In proceedings of the Extreme Programming and Agile Process in Software Engineering. (2004)
23. Cardoso, C.: Aplicando práticas de eXtreme Programming (XP) em equipes SW-CMM nível dois”, VI Simpósio Internacional de Melhoria de Processo de Software – SIMPROS (2004). 24. Käkhönen, T., Abrahamssom, P.: Achieving CMMI Level 2 with Enhanced Extreme Programming Approach. In Lecture Notes in Computer Science of Product Focused Software Process Improvement (PROFES), p. 378--392. (2004) 25. Hurtado, A., Bastarrica, M.: Implementing CMMI using a Combination of Agile Methods. XXXII Conferência Latino-Americana de Informática – CLEI. (2006) 26. Marçal, A., Freitas B., Furtado, F., Belchior, A.: Mapping CMMI Project Management Process Areas to SCRUM Practices. In proceedings of the 31st IEEE Software Engineering Workshop. pp. 13--22. (2007) 27. Salo, O., Abrahamssom, P.: Integrating agile software development and software process improvement: a longitudinal case study. International Symposium on Empirical Software Engineering. (2005) 28. Turner, R,. Jain, A.: Agile Meets CMMI: Culture Clash or Common Cause. In proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods XP/ Agile Universe. pp. 153--165. (2002) 29. Lyccet, M., Macredie, D., Patel, C., Paul, R.: Migrating Agile Methods to Standardized Development Practice. In Innovative Technology for Computer Professional. pp 78--95. (2003) 30. Anderson, D.: Stretching Agile to Fit CMMI Level 3. In proceedings of Agile Development Conference. (2005) 31. Glazer, H., Dalton, J., Anderson, D., Konrad, M., Shrun, S.: “CMMI or Agile: Why Not Embrace Both!. Technical Note CMU/SEI-2008-TN-003. (2008) 32. Jakobsen, C., Johnson, K.: Mature Agile with a twist of CMMI. In proceedings of the Agile Development Conference, p. 212--217. (2008) 33. Sutherland, J., Jakobsen, C., Johnson, K.: Scrum and CMMI Level 5: The Magic Potion for Code Warriors. In proceedings of the Agile Development Conference. pp. 466--471. (2007) 34. Jarvis, B., Gristock, S.: Extreme Programming, Six Sigma & CMMI – How they can work together, a JP Morgan Chase Study. http://www.sei.cmu.edu/cmmi/presentations/sepg05.presentations/jarvis-gristock.pdf