Our company is a 33,000-employee regional financial corporation in the United States with an IT staff of around 1,300 people. It has a long history of predictive or Waterfall 
software development practices. In the past three years, a small group of us has been developing the capability to do adaptive or Agile software development, and applying the approach to real projects. We have completed several projects successfully, beginning with small scale efforts and building up to enterprise projects. The following are unsolicited comments of the business managers who were customers of our first three Agile software development projects:
"I've been working with IT departments on software development projects for over 32 years, and this is the first methodology I've seen that actually works."
"I never want to do a project any other way again."
"I love you! I love you! I could cry!"
Their reaction is understandable, in view of the track record of Waterfall methods for software development. The Standish Group found that, as of 3rd quarter 2004, some 71% of software development projects either failed to achieve their objectives or were cancelled outright prior to completion. Such is the legacy of the Waterfall approach.
Yet, not every IT project lends itself to adaptive development. The next step for us is to institutionalize Agile development in the enterprise. To do so, we must understand when adaptive development is appropriate and when predictive development is appropriate. The same question is facing many IT professionals around the world, as adaptive development gains credibility as a mainstream software development approach.
Answers are easy to find, but hard to interpret. In Balancing Agility and Discipline: A Guide for the Perplexed , authors Barry Boehm and Richard Turner suggest the following guidelines:
| Agile home ground | Plan-driven home ground |
|---|---|
|
|
The Boehm and Turner book is generally very informative and useful; I still turn to it as one of a handful of books that help me with Agile project management on a practical level. But here they seem to reflect the fear of adaptive development that was common in the early days. Organizations were reluctant to risk an important project on an approach that was, in their experience, unproven. Circa 2002 or 2003, the authors' guidelines made sense. In the 2006-2007 time frame, as the industry moves toward institutionalizing adaptive development at the enterprise level, we need decision criteria that are more in keeping with the realities of large organizations, and with the (now) proven ability of adaptive development to deliver high value and high quality results.
Criticality and Changeable Requirements
Because the criteria presented by Boehm and Turner are reiterated on Wikipedia , and will therefore be read by many, let's begin by re-examining them. First, I disagree with the authors' understanding of the criticality criterion. In our experience, it is precisely on projects of high criticality that adaptive methods have shone. Predictive methods tend to bog down in paperwork and procedure. By the time project teams have completed the heavyweight up-front analysis required with the predictive approach, the criticality of the proposed project may well have escalated to crisis proportions. An adaptive approach enables us to deliver results incrementally, and thus to begin to chip away at the critical problem almost immediately. Customers are usually more satisfied with that outcome than with the typical outcome of predictive projects. Increasing the chances of success is surely all the more important when the project is critical.
Another criterion the authors suggest is the likelihood that requirements will change. Predictive methods, as the name suggests, are based on the assumption that it is possible to predict requirements with a high degree of accuracy before starting development. Adaptive methods, in contrast, are designed to accommodate changes that cannot be predicted in advance. This remains a valid criterion for choosing the appropriate approach for a project.
The first two criteria — criticality and requirements variability — are the tip of an iceberg. A more substantial decision criterion lurks below the surface. We need to understand what it is in order to make good choices about which approach to apply to a given project.
It turns out that the question is very similar to problems in public policy analysis, primarily in making policy decisions related to environmental issues. These problems call for reaching rapid decisions involving a variety of constituencies in cases when "facts are uncertain, values are in dispute, stakes are high and decisions are urgent." The basic idea is to substitute this approach for lengthy scientific study when such study would take too long to yield results. (This approach is sometimes called post-normal science to distinguish it from more conventional methods as defined by T.S. Kuhn in The Structure of Scientific Revolutions .)
Many software development projects have similar characteristics. Solutions must be delivered in time to provide business benefit in a competitive environment; there is no time to analyze every detail in depth before reaching decisions. Multiple stakeholders with different backgrounds and needs have an interest in the solution. Stakes are high, since the enterprise may invest a considerable sum of money and substantial time in the creation of a software solution that must deliver business value on the first attempt.
Stephen Schneider of Stanford University, concerned with public policy related to climate change issues, shows how post-normal science fits into the larger scheme of decision-making. He finds that normal science applies very well to problems that have low uncertainty and low risk (or "decision stakes," in his terminology). A problem characterized either by high decision stakes (risk) high uncertainty calls for post-normal science.
It's easy for a software professional to see how the same idea can relate to development projects: For "Applied Science," substitute "Waterfall approach" or "predictive methodology;" for "Post-normal Science," substitute "Agile approach" or "adaptive methodology." When uncertainty and stakes are both low, a predictive methodology works well; when uncertainty or stakes are high, an adaptive methodology works well. Figure 1 illustrates this idea:
Figure 1: Choosing an approach based on urgency and uncertainty
To apply this criterion usefully, we need to have a firm idea of what we mean by urgency and uncertainty. Everyone will say their project is "urgent." Everyone will say there are costs involved in a delayed implementation. And everyone will be right, to an extent. But urgency, in this context, does not only mean that there is a deadline. It means there is some exceptional cost associated with any delay in delivery. This is an important factor to understand correctly, because practically every predictive/Waterfall project will miss its original delivery date, by definition. We can't afford to "guess wrong" about this factor.
As an example, consider one of our first Agile projects. Its goal was to provide field representatives with handheld clients to access a back-end application for submitting and tracking loan applications for high-end recreational purchases such as boats, airplanes, and exotic cars. Without such a tool, the field representatives could not tell customers whether their loan applications had been approved for several days. In that time, customers often came to their senses and canceled the purchase. To finalize more sales, we wanted to be able to approve loans and notify customers while they were still in the showroom and still excited about the purchase. Furthermore, the business unit manager had calculated an opportunity cost for delayed delivery of the solution: If we missed the spring sales rush, we could lose up to $2 million in revenue per year, as well as losing market share and conceding the market leadership position to a competitor. That is an example of an urgent project. (The traditional IT group put in a bid of 10 months at $850,000; we did it with 4 people in 6 weeks at a cost of $73,000.)
The concept of uncertainty ought to be easier to grasp, although believers in predictive methods have a tendency to underestimate its effect on software development projects. The fact uncertainty exists in proposed software development projects is the Achilles' heel of predictive methods. The predictive approach assumes uncertainty can be eliminated before development begins. Depending on the nature of the problem at hand, the assumption may fall anywhere along a spectrum from slightly wrong to totally wrong. The adaptive approach deals with this reality by embracing constant change as normal, and by establishing robust methods to handle it in a manageable way.
There are two general kinds of uncertainty: End uncertainty and means uncertainty. End uncertainty is uncertainty about what is to be built, and means uncertainty is uncertainty about how to build it. The predictive approach to software development usually assumes that end uncertainty can be entirely eliminated by the end of the requirements specification phase, and that means uncertainty can be largely eliminated by the end of the design phase. The documented track record of predictive software development methods casts doubt on these assumptions. The reality is that both end uncertainty and means uncertainty may remain high almost to the very end of a software development project, depending on the type of project it is.
A customer-facing custom business solution is likely to be characterized by very high uncertainty right up to the final delivery. A technically routine project, even if complex (such as building a service-oriented architecture infrastructure), is far less susceptible to end uncertainty and somewhat less susceptible to means uncertainty than a customer-facing business solution development project.
Senior and Junior Developers
Boehm and Turner's assertion that Agile methods can only be applied successfully by senior developers is due for an update. When Agile methods were new, it surely required people who had enough experience to understand the potential benefits of adaptive development and to recognize and cope with the risks and obstacles. Once an organization has completed one or two initial projects using an elite team, it is appropriate to expand the capability for adaptive development by bringing in junior developers to pair with their experienced colleagues. The result is knowledge transfer in the context of real project delivery. In fact, this is just how we have built our internal Agile development group from its initial size of 6 people to its present size of 70. It's worth noting that a number of these have relatively little overall professional experience, yet every one of them is an effective Agile practitioner.
The key personnel factor is not senior-level professional experience. We have found that by staffing the adaptive development group with individuals who have certain personal characteristics, we have been able to maximize the chances of success on our Agile development projects. Figure 2 illustrates the characteristics we look for in priority order. The figure is not chosen arbitrarily; it visually reflects that fact that the prioritization is not linear, but that higher-priority attributes are geometrically more important than lower-priority attributes. Thus, character is very significantly more important than critical thinking, and so on through the list. Note that technical knowledge is the lowest priority characteristic, in direct contradiction to Boehm and Turner's suggestion that senior developers are necessary for success with Agile development.
Figure 2: Personal characteristics desirable in an Agile development group
Order vs Chaos
The statement that Agile methods work well in a culture that "thrives on chaos" as opposed to one that "demands order" carries the weight of unstated assumptions. First, there is the assumption that predictive methods lead to "order." While they do lead to predictable outcomes, they do not necessarily lead to "order." More commonly, predictive methods lead to cost overruns, extended project schedules, missed requirements, long bug lists, excessive overtime, death marches, staff burnout, dissatisfied customers, and poor business value. If that is "order," then give me chaos!
Of course, when Boehm and Turner mention "chaos," they may not be referring to disorder, but to the concept of the edge of chaos , a region where creativity and innovation can thrive, assuming that people aren't too bothered by uncertainty and change. That pretty well describes the typical software development project. In other words, "chaos" is not an abnormal state, but the normal state in our industry. To say Agile methods work well in a culture of chaos is tantamount to saying Agile methods are good for software development; not exactly a revolutionary observation.
Number of Developers
In the first few years of the Agile movement, adaptive methods were applied mainly to small-scale proof of concept projects. In that crucible, the quotidian details of adaptive project management were hashed out. In those days, it was probably correct to say that Agile methods required a small development team. It remains true that direct, face to face communication is the first and foremost principle of the Agile approach. But it is no longer true that a small team is necessary. We have had success with teams in excess of 30 people, working on projects that lasted up to 18 months and worked through 15 iterations and 5 product releases. Agile methods have matured to the point that they can be applied to projects of enterprise scope.
There used to be a perception that Agile methods were not scalable. I believe this was due to the fact that the best practices were still being worked out by the first wave of Agile practitioners. At our company, we have encountered some of the stumbling blocks to using Agile on a large scale, and we have learned that the obstacles can be overcome. Iteration length, release length, total project length, number of stakeholders, and level and consistency of customer involvement have been the most significant factors for scaling Agile methods. Agile development does scale, after all!
In theory, it always seemed as if Agile development ought to scale. After all, it is predictive methods that attempt to eat the whole elephant in a single bite, by trying to predict and design everything in advance. The larger the elephant, the harder it is to swallow whole. Agile methods always ate the elephant one bite at a time. As long as you do that, it really doesn't matter how big the elephant is. So, it seems it is the predictive approach that doesn't scale, as its track record suggests.
Non-repeating Process
We have addressed the decision criteria suggested by Boehm and Turner explicitly because they are widely referenced, but they are not the only factors involved in choosing an adaptive or predictive approach. Among the most critical decision points is whether the development project is a repeating or non-repeating process. When a software development process is a repeating business process, it makes sense to apply the predictive approach; when the process is a non-repeating process, it makes sense to apply the adaptive approach.
Software development may be a repeating process when the company's core business is software development. In that case, it's typical for a software package to be developed through a scheduled series of releases. Business application packages usually support regulatory requirements, industry standards, general best practices, and technical integration requirements that are fairly predictable. The relative predictability of requirements and the repeating release cycle lend themselves to a predictive development approach. This model of software development is analogous to a manufacturing process.
Software development for one-off, custom business applications is not a repeating process. The same solution is not produced time after time, in project after project. Certain design patterns, engineering principles, and technical best practices may be applied routinely, but each solution is unique. There is no analogy with a manufacturing process here; this sort of development is more analogous to the construction industry, in which each house, bridge, or other structure is built according to design patterns, engineering principles, and technical best practices, and yet each is a unique product. This model of software development lends itself to an adaptive approach.
New Product Development
Another decision criterion for choosing an adaptive or predictive approach is whether the purpose of the project is to develop a new product or to deliver a more routine sort of output. In his book, Agile Project Management: Creating Innovative Products , Agile proponent Jim Highsmith writes,
[The] design process is about producing economically useful information. With a product design, until the final manufacturing engineering plans are in place, no one knows for sure whether the product can be made to the specifications that the customers demand. However, as a product gets closer and closer to the "release to manufacturing" stage, the more confident the development team should be. Product development is first and foremost about generating and processing information, not predictability. If a process is predictable, if all variations are accounted for, if thte process is repeatable in a statistical quality control sense of the word, then it won't generate any new information. The ideal of statistical repeatability sought by "heavy" process proponents flies in the face of product development reality.
This is one of the main reasons for the poor results obtained from predictive methods in business software development. Since each customer-facing, one-off, custom business solution is unique, each is a "new product," and every development project is really "about producing economically useful information" and not about repeating the previous project step by step.
Customer Involvement
Even if your project meets all the criteria for Agile development we have discussed so far, unless you can get a real commitment from your customer to participate directly and continuously in the project, you should not choose the adaptive approach. Adaptive or Agile development can only work when the customer is personally involved and committed throughout the project. Customers cannot drop off a list of requirements and then return a year later to pick up their finished software. If your customers are not ready for the level of participation Agile development demands of them, then they are not ready to enjoy the benefits of Agile development. They need the framework and structure of a predictive methodology. And they will obtain the predictable result.
Conclusion
And so we come at last to the cookbook recipe you've been waiting for to help you choose an adaptive or predictive approach for a software development project.
| Adaptive (Agile) | Predictive (Waterfall) |
|---|---|
|
High uncertainty or high urgency Non-repeating process New product development Right sort of people on staff High level of direct customer participation |
Low uncertainty and low urgency Repeating process Not new product development Traditional, process-oriented staff Low level of direct customer participation |