Merging the developer and tester roles
Topic: Continuous improvement
Warning: This post expresses ideas that are not politically correct. It will offend different people for different reasons. Parental guidance is advised.
In the past few years, one of the most interesting and positive trends in IT has been the merging of professional roles and the growing popularity of the idea of the generalizing specialist.
Business application development doesn't usually call for narrow-and-deep specialized skills in any one particular area, except possibly for short consultations on occasion. Instead, that sort of work benefits from generalists who can keep in mind the big picture and who can carry out whatever technical tasks the project calls for at any given moment. The agile approach to software development is especially well-suited to business application development, and it has become very popular over the past several years. Agile practices such as collocated team, cross-functional team, dedicated team, stable team, and collective ownership, as well as the guideline to maximize a team's truck number or bus number, have meant that technical specialists who previously worked in separate silos were working side by side for extended periods of time, spanning multiple projects. It was quite natural for them to start learning one another's specialties, at least to the extent necessary for business application development and support.
Those who have worked on business application development projects will relate to this. The level of database-related work involved rarely rises to the level that requires a narrow-and-deep specialist in database technology. There is no reason why an application developer cannot take care of the database-related work necessary for a business application development project in nearly all cases. A DBA can be brought in on occasion when necessary. The same general rule holds for other technical specialities, as well: Network technology, security, user interface design, etc. While user experience design (UX) is a genuine specialty, most business applications don't require more UI design than to follow conventional, well-known guidelines. While technical architecture is a genuine specialty, business applications almost always follow one or another reference architecture; every CRUD webapp doesn't require a fresh architectural design. The old saw that technical people and business people don't know how to talk directly to each other has been false for many years. So it goes for other specialized roles, as well. The result has been the emergence of the technical generalist — an ideal skillset for most agile-style application development and support work. The term generalizing specialist refers to a specialist who learns other skills to level-up his/her skillset.
From a lean point of view, a team of generalists alleviates flow problems stemming from all three of the famous M words: muda, muri, and mura. Its most direct effect is to reduce mura: unevenness or irregularity in work flow. To visualize how a team of generalizing specialists can achieve smoother work flow than a cross-functional team of specialists, consider the canonical Rational Unified Process (RUP) process diagram:
RUP was a step on the journey from the dysfunctional methods of the 1980s toward improved methods of delivering value through software. To address problems of long lead times and misalignment of results with business needs, RUP defined an iterative development/delivery process. Notice that the horizontal representations in the diagram of various types of activity on a project grow thicker and thinner depending on the RUP phase. This shows how the project's need for particular types of work grow or shrink depending on exactly what the team is working on at the moment. Sometimes there is more analysis going on than coding; sometimes there is more testing going on than analysis; and so forth. When the team comprises specialists, each of whom carries out only one type of activity, the work flow is uneven. Although work is always happening on the project, individual team members experience long periods of relative inactivity punctuated by short periods of very intense activity and, probably, overtime. This is mura. This sort of mura results in some misalignment of results with business needs and some amount of needless additional lead time, although it is far better than the linear and spiral processes that predate it.
There is a sort of domino effect in this process model, as well. The mura leads to muri (overburden) in that during the periods of intense activity for any given specialist on the team, that specialist is trying to complete a great deal of work in a short time. Whenever humans do that, they inevitably overlook details due to fatigue or excessive mental context-switching. Other team members are idle while they wait for the results of the specialist's work. Waiting is a form of muda (non-value-add activity). While his/her teammates are waiting, the specialist's mura leads to defects. Defects lead to rework, which is another form of muda. In extreme cases it can lead to employee burn-out and high turnover, which reduce the organization's effectiveness in delivering value on a larger scale and for a longer time than the scope of any single project.
The problem is not inherent in the RUP model as such, but is a natural consequence of maintaining traditional role specialization. You can see the same effects using Scrum or Kanban or any other process framework, if the team members are specialists. It just so happens that the popular RUP diagram illustrates the natural variability in activities quite nicely. Variability in activities is normal; problems occur when we assume each activity requires a different specialist to perform it.
The growth of the generalizing specialist did not stop with purely technical specialties. As IT professionals have pulled testing forward in the process, increasingly emphasized behavior-oriented examples over implementation-based test cases, and improved the tools to support executable requirements, collaborative elaboration of requirements, and acceptance-test driven development (for example: FitNesse, Cucumber, Freshen, etc.), the traditionally-separate roles of Business Analyst and Tester have grown together into a single role. Even in the early years of agile adoption, Testers found it frustrating to deal with code bases that were not "stable," because the tools available to them assumed all testing would be done after-the-fact. To deal with continuous testing during development, Testers found the only logical thing to do was to sit down next to the Business Analysts. Many agile team leads, ScrumMasters, and project managers have reported that even when they do not explicitly plan to combine these roles, team members tend to fold the two roles together anyway. It's understandable — it would be difficult to separate the activities of Business Analysts and Testers in an agile context. If the functionality of a system is described in the form of executable examples, and the definition of done is that the examples run successfully, then it's only natural for analysis and testing work to be done by the same team members. Indeed, when those roles are formally separated on a project that is ostensibly "agile," the separation itself is a process smell.
A similar phenomenon has occurred at the line management level. Scrum is an iterative process framework that supports empirical process control; a natural fit for agile-style product development work. Some of the proponents of Scrum predicted that the traditional Project Manager role would evolve into something quite different; something along the lines of the Scrum role called ScrumMaster. This is a process facilitator, team coach, mentor, enabler, obstacle-remover role; but explicitly not a commander or boss role. The ScrumMaster role is formally defined to have responsibilities but no authority. As agile methods, and Scrum in particular, became part of mainstream IT, what actually happened is not that PMs transformed into SMs, but rather they became generalizing specialists who combine the functions of a PM and the functions of an SM. The result has been a shift in the IT industry from a Theory X management approach, common in the 1980s and 1990s and fundamental to the dysfunction of IT in that era, to a Theory Y approach that is consistent with agile and lean thinking.
At the present time (early 2010), we seem to have reached a cultural barrier when we attempt to fold together the roles of Developer and Tester. People who self-identify as testing specialists are especially resistant to the idea of merging the roles. They go to great lengths to explain why this isn't feasible, or why developers are genetically incapable of testing software. Most of the examples they present to support this view are very weak. I'll describe one example shortly.
I think the reason for this is we have reached an old status quo stage of transformation, industry-wide, with agile adoption (in terms of the Satir Change Model). Professional testers have carved out a niche for themselves as specialists in the agile world. They know exactly how to function and how to add value on agile projects provided role definitions do not change again. As Satir observed (paraphrased), people prefer the familiar to the comfortable; the comfortable to the better. The Satir Change Model is well-documented. It's usually illustrated like this:
In the context of step-by-step agile transformation, I like the way Willem van den Ende explains it in his blog. His illustration of progressive, incremental improvement represents each improvement as a foreign element introduced to an old status quo. The pattern Satir identified is repeated as each improvement is introduced and gradually becomes part of the new status quo. It's easy to visualize the growth of the generalizing specialist concept in this way, as work practices have evolved over the past decade.
The foreign element being introduced to the old status quo at the moment is the notion that development and testing activities can be carried out by the same individuals on any given software development team, in the context of business application development. (I keep repeating the bit about "business application development" because other domains have other requirements.) The goal is to improve delivery effectiveness by reducing mura (unevenness of flow, caused by hand-offs between specialists on the team), which in turn will reduce muri (overburden, caused by peaks and valleys in demand for specialists), which in turn will reduce muda (non-value-add activity, in particular defect correction and rework, but also inventory in the form of "completed" programming tasks awaiting formal testing).
When trying to explain why developers "can't" test their code effectively, testing specialists sometimes come up with rather weak examples. In describing a "new" category of software defect he calls Ellis Island bugs, the respected testing specialist Michael Bolton describes a small program that illustrates the Ellis Island problem:
The program takes three integers as input, with each number representing the length of a side of a triangle. Then the program reports on whether the triangle is scalene, isoceles, or equilateral. [...] no matter what numeric value you submit, the Delphi libraries will return that number as a signed integer between -128 and 127. This leads to all kinds of amusing results: a side of length greater than 127 will invisibly be converted to a negative number, causing the program to report "not a triangle" until the number is 256 or greater; and entries like 300, 300, 44 will be interpreted as an equilateral triangle.
He goes on to repeat the example in C and Ruby, and reaches the conclusion that this represents a category of software defect that is very difficult to predict, and that developers are likely to miss. Each language and each set of underlying library functions makes different default assumptions about how to interpret the input strings, and each platform handles integer values differently. Therefore, the only way to catch this type of defect is by exploring the behavior of the code after the fact. Typical boundary-condition testing will miss some Ellis Island situations because developers will not understand what the boundaries are supposed to be.
The italicized phrase in the preceding sentence illustrates what I mean about "weak examples" to support status quo role segregation. These programs, in whatever language, accept any input without validating it. That's just plain bad programming. In the context of agile-style development work, we would not begin work on a feature that was so ill-defined that boundary conditions were not understood. When we don't clearly understand what "done" will look like, we continue our discussions with stakeholders until we do understand it. That's common to all agile processes.
Using the typical behavior-driven approach that is popular today, one of the very first things I would think to write (thinking as a developer, not as a tester) is an example that expresses the desired behavior of the code when the input values are illogical. Protection against Ellis Island bugs is baked in to contemporary software development technique, and is redundantly supported by other agile practices the team would normally follow: Pair programming, automated unit testing, customer collaboration. When you add to that the growing trend of software craftsmanship, the idea that developers will tend to overlook Ellis Island conditions doesn't hold watir. (Just trying to lighten it up, since this is one of the offensive bits.)
I can definitely agree that if a development team did not follow contemporary software engineering practices, then they would need someone to tag along after them to look for bugs in their code, like the shovel-bearing clowns who follow horses in a parade. The real problem in that case would not be that developers categorically write unbelievably bad code like the triangle program, but that the team in question was not following well-known effective development practices. The real solution is to fix the real problem, and not just to grab another shovel.
Eventually, I foresee a different set of professional career paths than those which evolved from traditional software development processes. At one level, there will be narrow-and-deep specialists who build technical infrastructures and deal with edge-case non-functional requirements for high throughput, high availability, high performance, high volume data storage/access, or extreme security needs. These people will usually work in the central IT department of a large corporation, or as innovators in start-up firms that have creative new ideas for using information technology. These will be the software craftspeople. At another level, there will be generalists who can assemble the building blocks of business applications cleanly and with appropriate attention to quality, following standard guidelines for issues such as UI design, solution architectures, and data storage. In contrast with present-day organizational structures, they will work in the business units that need the software, and not in a separate department. They will have deep expertise in the business domain and reasonably good general technical skills.
I think the resistance we see at the moment to merging certain traditional roles is a point-in-time snapshot of a longer-term process of change. The choice before us as individual practitioners is whether to prepare for and progress with the changes, or find a niche job where we can play out the remaining years of our working lives within our current comfort zone. The cheese is being moved, with our without our approval.
Posted by Dave Nicolette
at 10:06 AM EST
