A system too large for one person to build is usually also too large to build without an overall plan that coordinates the people working on it, the tasks that need to be done, and the artifacts that are produced. Researchers and practitioners have identified a number of software development process models for this coordination. Here are some of the main ones.
These process models are alternatives, but not exclusive ones: most describe different aspects of a process, and it is common for a development group to be following two or more simultaneously. For example,
delivered systemstage of the prototyping model.
Produce a buildstage.
Figure 1. Code-and-fix
This simple process is often said to be what unsophisticated developers follow spontaneously . It provides no guidance for dividing up the task of producing software. It doesn't distinguish the various development artifacts (they may not even be present, except for the code).
In this process, developers write code, fix the problems they notice, and repeat. There is no guidance to help developers converge to an appropriate result (Boehm1988-smsd).
|Requirements||Requirements and specification|
|Architecture||The system architecture, division into modules, and module interfaces|
|Implementation||The source code|
|Testing||The module, subsystem, system, and acceptance tests|
|Deployment||The distribution package|
|Maintenance||Bug reports and modified artifacts|
Sequential processes divide up software development by the distinguished activities of software development, each one associated with a distinct kind of artifact (Table 1), and then do one after another in some pattern. The activities and artifacts have a linear dependency relationship: each activity's artifacts depend on the artifacts produced by the activities above it in the table, and if a higher artifact changes, all lower artifacts may have to change to match it. This chain of dependence affects all software development; in processes that don't produce all these artifacts, such as XP in which requirements and specifications are not produced, the kind of knowledge that would go into the absent artifact still participates in the chain of dependence.
The sequential processes make the development activities the top-level elements of the process, and deal with the chain of dependence by doing up-chain activities before down-chain activities in some pattern.
The waterfall model was in use as early as the late 1950's. It was first described explicitly (by Royce in 1970) as a way software should not be produced.
Figure 2. The simple waterfall model (modified from Boehm1988-smsd)
The simple waterfall model (Figure 2) describes a sequence of activities and corresponding artifacts, from the most general (requirements) through successively more detailed steps to implementation, after which the software is put into operation and maintained. Each step has a corresponding verification and validation activity: requirements are validated, architectures are verified against requirements, module designs are verified against architectures and requirements, implementation is tested, and the operational system is revalidated through its use. In this idealized process, each stage is a (relatively) complete and correct description of the final system, produced using the results of the preceding stage. If all goes well, the final system will meet the initial requirements and the stakeholders will rejoice.
As Royce pointed out, though, this can't actually be expected to happen. In working out the requirements, we can't possibly take advantage of what we will learn later when doing the architecture, design, implementation, and operation of the system: and it is not humanly possible that the requirements will be completely appropriate without that later knowledge. The same thing is true for each later phase: to make an appropriate architecture, we have to know the things we won't learn until we do the design, implementation, and operation; and so forth. Each phase has to be partially re-done based on the knowledge gained later. Figure 3 shows the feedback paths that occur in practice: every later phase uncovers knowledge that is needed to do each early phase well. In order to work well, this feedback and rework has to be organized somehow.
Figure 3. The waterfall model with feedback (modified from Royce1970-mdls)
The simple waterfall model is perhaps most useful for showing the minimum dependencies that occur among the software development phases and artifacts: everything else depends on the requirements; design and implementation also depend on the architecture; and implementation also depends on design.
Figure 4. The sashimi model
The sashimi software process (Takeuchi+Nonaka1986-nnpd) is quite similar to the waterfall, except that the phases overlap to show that requirements can't be completed until architecture is at least partially explored, and architecture can't be completed until module design is at least partially explored, and so on.
The sashimi process is most appropriate for medium-sized projects for which the communication between phases can be handled in an improvised manner. For larger projects, high-risk projects, or projects in which few of the developers are experienced, a spiral approach may be better.
Figure 5. The prototyping model
In this approach, an initial prototype is produced (by whatever development process is desired) and used by stakeholders in order to validate the requirements and identify problems and promising solutions. The prototype must exhibit at least enough of the eventual system's intended characteristics for the stakeholders to evaluate it, but typically a prototype will run slower, lack the desired ilities, and afford only incomplete functionality. The final system is then developed from scratch (again by whatever process is desired) benefiting from the lessons learned.
The use of prototypes has been common in engineering and in less formal approaches for building just about anything. For building physical objects, a prototype is often a model at reduced scale. In software development the most memorable discussion of prototypes is by Brooks, which he summarizes famously as
plan to throw one [developed system version] away; you will anyhow.
But note that twenty years later, Brooks goes on to say
This I now perceive to be wrong, not because it is too radical, but because it is too simplistic p.265 … An incremental build model is better. p.267
In contrast to sequential processes,
in which a list of distinguished activities
are done one after another,
cyclical processes do
the same thing over and over.
The goal is that each cycle brings the development
closer to its successful completion.
The various cyclical processes
things to do over and over,
and may have specific relations between successive cycles.
Figure 6. The essence of the spiral model
The spiral software process is a cyclical model whose steps are not the activities of development (requirements, architecture, etc.) but rather four phases for addressing whatever problem has the greatest risk of causing the development to fail. For each cycle, developers follow these phases:
Notice that at the end of each cycle,
the developers have a
Initially this product may be an overall concept;
as the spiral process goes through successive cycles,
the product becomes an implementation.
Figure 7. Boehm's example of a spiral model development (Boehm1988-smsd)
In the first (innermost) cycle of the example, beginning in the upper left quadrant,
concept of operation(probably we would call this an overview of the system); and finally
In the second cycle of the example, developers determine the cycle's objectives, etc., and perform a more substantial risk analysis, then produce a second prototype. As in every cycle, they then do simulations, etc. In this cycle, they next develop and validate requirements for the system. In the planning phase they produce a plan for the development.
In the third cycle, the highest current risk is determined to be the software's architecture and design. The developers perform a risk analysis and produce yet another prototype (the third one). After further simulations, etc., the developers produce an architecture and design and validate and verify them. Finally, they plan the integration and testing.
In the fourth cycle, developers focus on detailed design and implementation. As always, they do a risk analysis and produce a prototype; this prototype is an operational one that can be evaluated in terms of the system's eventual operation. After simulations, etc., developers produce a detailed design; implement the modules and unit-test them; integrate the modules (probably in several steps at several levels) and test the results; and put the system through its acceptance test. At this point development may have reached a successful conclusion; if not, another cycle will be needed.
The Rational Unified Process or RUP is perhaps the only one discussed here whose use was and is promoted and supported by a specific company whose business is based on it (Rational Software, now owned by IBM). RUP can be characterized as a spiral process, with each iteration driven by risk mitigation, within which the activities follow a waterfall or sashimi pattern.
Iterations are grouped into four successive phases, one or more iterations per phase, in which first the early activities, then progressively later activities, are dominant:
Figure 8. The RUP model
(original of figure)
and the activities are categorized into nine disciplines, comprising six engineering disciplines:
and three supporting disciplines whose activities cross-cut the engineering disciplines:
The distinctive features of RUP are in the details of its prescriptions for requirements, analysis, and design (specifically in how development knowledge from one artifact type directs the next kind), beneath the level of abstraction of software processes, and are not discussed here.
Rational Unified Process
arose from the history of the company promoting it.
The company was originally named Rational Machines
and produced Ada development tools in the 1980s.
In the 1990s Rational hired or bought the company of
each of the three prominent object-oriented methodology proponents
(Booch, Rumbaugh, and Jacobson),
whose competing design notations and processes
and were analogous but distinct,
after which they developed a single notation
and a single process unifying
the concepts behind all three,
namely UML and RUP.
Figure 9. The incremental model
An incremental process is one in which
the functionality of the desired system
is divided into small increments
that are implemented and delivered one after another
in quick succession.
Each increment is chosen
so that it expands on the previous one,
and is small enough to produce quickly.
The most important functionality is implemented first,
in the earlier increments
The initial increment
produces a running system
that does next to nothing,
but [does] it correctly
Builds are frequent,
An incremental process has several advantages:
Virtually all modern development processes are incremental.
Figure 10. The test-driven model
The test-driven software process is the one followed for agile development, extreme programming, and similar approaches. It is an incremental approach in which each increment is defined by a new test. Each iteration of the cycle produces a running system that passes all its tests (Williams+Maximilien+Vouk2003-tddd).
If the new test doesn't fail, the developers have to back up, figure out why, and fix the test so it fails.
good enoughfor the stakeholders, the development is complete. If not, the stakeholders request a new behavior and the cycle begins again.
Agile describes a group of related development processes that are usually presented in opposition to traditional development processes such as the waterfall and spiral models. Agile development is characterized by an emphasis on small teams of skilled individual developers, changing requirements, frequent version deliveries, daily contact with stakeholders, possibly with one or more captive stakeholders that sit with the team constantly, and face-to-face interactions. See http://agilemanifesto.org/ for the principles behind agile development.
A related approach from the 1980's, Rapid Application Development or RAD, was similar to agile development in many ways but (as far as I can determine) was not test-driven.
Perhaps the most prominent agile methodology is the test-driven Extreme Programming, or XP (Beck2000-epee). In XP, a group of up to about a dozen highly-skilled developers and a stakeholder sit in the same room. The developers work in pairs. Requirements are expressed as tests, and the tests are the requirements. There is no documentation and no artifacts other than the tests and the code. Everything else is handled by face-to-face discussions (thus the need for everyone to be in one room).
jUnit was developed for use in XP.
Figure 11. Scrum model
The Scrum process organizes development into a sequence of sprints, each of which results in a potentially usable product with an added increment of function. The tasks for each sprint are set, in consultation with a stakeholder representative, during a sprint planning meeting and cannot be added to during the sprint. Each task is typically expressed as a user story. Each sprint is timeboxed: the end date of the sprint does not change. Tasks that can't be accomplished in time are returned by the team to the backlog for future consideration.
During a sprint, the team has a brief Daily Scrum meeting, facilitated by the designated Scrum master, in which team members say what they did yesterday, what they are going to do today, and what obstacles are in their way. No brainstorming or discussion is allowed; anything other than answering the three questions is deferred to meetings among the specific people involved.
Takeuchi and Nonaka (1986)
use the metaphor of rugby,
and talk of
moving the scrum downfield [p.138]
to describe all members of a team moving en masse
towards their goal,
but don't specifically describe a scrum process.
The Scrum software process appears to have been first described by
DeGrace and Stahl (1990),
who say it originated in camera and automobile development,
and introduce it with
If Scrum were applied to software development,
it would go something like this.
Beedle et al. (1999)
Rising and Janoff (2000)
are important early reports on Scrum for software development.
The waterfall process is best as a means of explaining software development phases, activities, and artifacts. It gives a general overview and is a good starting point for discussion. However, the processes by which people usually produce software are a good bit more complex.
The spiral model is so general that it does not provide much guidance for novice software developers. For more sophisticated developers, it can help guide them in focusing on risks; that is what the spiral model is best for.
The context in which a waterfall process is perhaps most useful is that of a large system developed by hundreds of people. In such a context, the development process is necessarily unwieldy, because so many people are involved and the cost of changing requirements is so high. Here, the payoff from a big up-front investment in getting the requirements and architecture right is highest, and a waterfall approach is most plausible. However, because the risks of getting requirements or architecture wrong are high and the costs that result from doing so are so great, a spiral modification of the waterfall is just about essential.
It should not be surprising that the waterfall and spiral processes were a result of big software projects; the waterfall model is said to have arisen from the SAGE project, a large air-defense system developed in the late 1950's, and the spiral model was a reaction to problems with large defense projects in the 1970's and 80's.
The test-driven process and other lightweight processes were a reaction to the software practices developed for large systems. These software practices often involved committees, thick requirements documents, lots of reviews and management, complicated processes, and other not-fun things. Most projects are not large and so most projects did not need or benefit from large-system software practices.
Agile processes are best for projects that are not large, and involve a small group of developers working for a limited time period. But agile processes require that the developers involved be highly skilled.
How to decide?