Submissions for some of the key agile-related industry conferences are presently under review. I've read quite of few of these, and I've noticed that this year there are a number of submissions on the subject of how to manage non-functional requirements on agile software development projects. All offer different recommendations, but they have one thing in common: Each proposal is described as if it were The Answer to The Problem with non-functional requirements on agile projects. Is there really a single problem — The Problem? And if there is really a single answer — The Answer — then why so many different recommendations?
Let's take a closer look at the issues in managing non-functional requirements in the context of agile development methods.
I will state a working hypothesis: There are two reasons why agile teams seem to have difficulty managing non-functional requirements.
1. When people first adopt agile development, they tend to approach it from a process-centric perspective. They expect an agile methodology — DSDM, XP, AgileUP, or what-have-you — to present yet another process cookbook for them to follow. This is not a failing on their part; all previous approaches to software development have, indeed, been cookbook processes for us to follow. It takes a bit of experience with agile methods to begin to grok the implications of the people over process emphasis stated in the Agile Manifesto.
Many development teams don't grasp the fact that accountability for getting the requirements (all the requirements) right falls to them, and there is no other role and no formal process step to handle it for them. Agile values and principles do not spell out any particular technique for fulfilling that responsibility, other than to collaborate with others to obtain whatever information is necessary. The team is expected to come up with a way that works in the context of their project and their organization.
I've heard more than one team say that their customer never explicitly told them about any non-functional requirements, and therefore they need not worry about it. That attitude doesn't mesh well with agile methods, because if the team fails to take care of the non-functional requirements, no one else will do it for them. It isn't anyone else's job. Welcome to the world of self-organization!
In team coaching situations, it's not uncommon for team members to ask me whether they are "allowed" to capture non-functional requirements and other common information in some central place, such as the project wiki, since the "standard" is to base all work on User Stories or similar lightweight artifacts. The frequency and innocence of this question underscores the fact many people assume any given methodology only "allows" them to do the things the methodology spells out explicitly. They actually feel as if they are being naughty if they do anything beyond that.
"Agile" development differs from this pattern in that it presents only a set of values and principles, and leaves the specific activities up to developers to determine based on their own working context. If developers only do the things spelled out by a methodology such as, for instance, Extreme Programming or Crystal Clear, they will almost certainly omit many important details. Agile processes do not provide detailed instructions about matters such as software engineering principles, technical architectures, quality-of-service issues, usability, accessibility, or general design principles. This by no means implies teams can afford to ignore those things.
2. The requirements artifacts typically used on agile projects are focused on the behavior of the product that is visible to end users. There is no "standard," formal requirements artifact that provides information about non-functional requirements. This point is really an extension of the first, because it derives from the people over process philosophy. Developers are expected to understand and apply generally-accepted good practices in software engineering. That includes (among other things) understanding the architectural and quality-of-service implications of a general description of software functionality, at least well enough to ask the right questions and ensure non-functional requirements are covered.
Requirements artifacts on agile projects are most often one of the following:
Each of these artifacts is explicitly limited to describing system behavior that is visible to the end user and that provides value as defined by the end user. All the ancillary considerations that go into building a functional and robust software product are still necessary, of course. "Agile" as such just doesn't define any sort of standard, formal artifact to capture them. It's up to the team.
The requirements artifacts typically found on agile projects deal only with functional requirements:
There are many ways to think about and model the elaboration of software requirements. Personally, I like the FURPS+ model, described in an article on IBM developerWorks. The letters stand for:
The lightweight requirements artifacts used on agile projects deal with functional requirements, so that is not the issue here. Everything else is a non-functional requirement.
Usability is a broad category that may include (a) enterprise (or product) standards for UI layout, look and feel, and branding; (b) UI navigation design (users are not "surprised" by UI behavior); (c) User Experience (UX) considerations; (d) accessibility; and (e) localization.
Reliability concerns factors such as (a) system availability; (b) recovery time after a failure; and (c) data integrity.
Performance generally involves two broad categories: (a) perceived response times of individual transactions from the perspective of an end user, and (b) overall system throughput under load.
Supportability is a somewhat subjective category that concerns the difficulty of understanding the code base, tracking down the causes of reported production problems, and extending or enhancing the application. Agile methods deal with this area through the use of design and development techniques that control the accumulation of technical debt, such as test-driven development, refactoring, automated test suites, and continuous integration. This is not a category for which specific "requirements" are spelled out, in most cases.
Additional considerations may include anything and everything relevant to the application and its execution context, including (but not limited to) security, auditability, licensing, integration with other systems, regulatory compliance, instrumentation for business activity monitoring or other purposes, and any other special considerations, especially those with implications for the architectural design of the solution.
It should be apparent that some categories of non-functional requirements may be elicited from the customers who provide the functional requirements. It is just a matter of asking them the right questions and guiding them through usage scenarios that bring out non-functional requirements.
Some quality of service requirements can probably be learned from the customers. The term "quality of service" spans multiple categories of non-functional requirements, the most obvious of which are Usability, Reliability, and Performance. However, it is not safe to assume all requirements in these categories can be learned from the customers. If the project constitutes internal development for a company, then the enterprise may have standards or guidelines pertaining to Usability, Reliability and Performance that must be supported in addition to the customers' particular needs.
In addition, many enterprise IT shops offer internal customers a selection of two or three service levels that have different charge-back costs. The cost of each level of service may inform the customer's choices in areas like Reliability and Performance. If there were no costs, customers would always choose the maximum possible levels of service. Therefore, you can help your customers achieve the business value they need at the right price point while controlling the scope of development so that you can deliver rapidly, if you can collaborate with them to select quality of service requirements.
When you are developing a conventional business application in a corporate environment, it is often the case that the customers do not know in advance exactly what they need the application to do. They have a vision of how the software can support a business opportunity, and they refine their understanding of their own needs based on their use of increments of the solution the team delivers periodically. In this sort of development context, the team has to be ready to absorb new requirements at any time, including non-functional ones. There are techniques for collaborating with customers who do not already know precisely what their needs are, such as the "nine squares" or "nine boxes" technique described in the book, Solution Selling, by Keith Eades. The technique is also briefly described in a blog post, Nine Boxes, which shows an example of how some non-functional requirements may be determined through customer collaboration. It is also possible to approach solution architecture incrementally, so that you can remain flexible enough to accommodate unanticipated design constraints that come up in the course of development.
For commercial software development, quality of service requirements are based on customer demand. It is not feasible to collaborate directly with bona fide end customers. Therefore, development teams need access to an internal person or team who understand customer needs well enough to serve as a customer proxy for purposes of defining acceptance test criteria, functional requirements, and non-functional requirements for the product. It remains the team's responsibility to engage this person or team to ensure they understand the requirements for their product.
In some development contexts, the non-functional requirements are derived from the operating environment of the product. Embedded environments do not offer much architectural flexibility, for example. When developing a product targeted to a well-known environment, it is not necessary to go through an elaborate procedure to determine the non-functional requirements. You can usually make many valid assumptions about architectural constraints and other non-functional requirements based on your knowledge of the operating environment.
There are development contexts in which software development is only a small part of a larger program that may involve multiple subcontractors whose work must be coordinated and whose results must be seamlessly integratable. In these cases, even if some or all the software development work is performed using an adaptive approach, it is likely that the architectural standards and other non-functional requirements will have been specified in advance, since interfaces between subsystems (both hardware and software) must be defined before the various subcontractors can understand how to design their respective pieces of the solution. Although this type of project is only partially agile at best, at least the question of non-functional requirements is simplified.
This section really has two points: First, the sources of non-functional requirements may vary depending on the development context. Second, regardless of context, responsibility for learning and supporting the non-functional requirements resides with the development team. The team must take the initiative to engage others who have information relevant to non-functional requirements, and they must exercise self-discipline to ensure their test suite covers the non-functional aspects of the solution as well as the functional ones.
Coaches, mentors, consultants, writers, speakers, and pundits (did I overlook anyone?) are all full of advice about how agile teams should deal with non-functional requirements. Each believes him/herself to have The Answer, and each Answer is different. There are, however, a handful of common patterns:
That's what other people recommend. Now it's time for me to ride in on a white horse brandishing The Answer. Cue the orchestra!
Or...maybe not. Sorry. I must have left the horse in my other pants.
The truth is there isn't a one-size-fits-all answer. Those who recommend these various solutions are not wrong, and they are not trying to trick you. Each has achieved good results using whatever approach he/she is promoting. If they are wrong about anything, it's the assumption that their way is the only way. All the approaches have merit, and all have potential risks for a project. In keeping with the principle of people over process, it's up to you to understand the implications and make a reasoned choice. All the various conference talks, workshops, training courses, vendor products, and consulting advice are fine. Each will work, provided it is applied in an appropriate context and provided the team takes the potential downside risk into account.
| When the current situation is... | and the recommended action is... | the potential benefits are... | and the potential problems are... |
|---|---|---|---|
| Team is not having any difficulties with non-functional requirements. | Any | None | Sometimes people recommend a process or tool because they have attained good results with it and they assume every team should use it. They are not actually addressing any specific problem in your environment. When you try to fix something that isn't broken, you only create new problems. You may cause many types of problems by introducing a solution to a non-problem, including delayed delivery, compromised product quality, reduced team morale, increased project costs, reduced customer satisfaction, reduced return on investment, and loss of customer trust. |
| Team is having difficulty learning what the non-functional requirements are. | Any | Indeterminate | If you throw a solution at the problem without first understanding the root cause, you may only succeed in making it more difficult to determine the root cause. Begin with a root cause analysis so you can learn what the problem really is. It may be a lack of communication from the customer, a lack of available information about enterprise standards and guidelines, or a lack of accountability on the part of the technical team members for discovering the non-functional requirements. Most of these causes do not call for a change in process or for the introduction of a new tool. |
| General non-functional requirements pertinent to the organization (regulatory requirements, enterprise standards, etc.) are not readily available to development teams. As a result, teams deliver software that does not comply with all non-functional requirements. | Any | Indeterminate | If the information is not available in the first place, then introducing a new process or tool to manage non-functional requirements will have no effect on the problem. A first step is to collect and centralize the common non-functional requirements for the enterprise and train teams to look for the information in the prescribed location. If difficulties in supporting non-functional requirements persist, then there is another problem and you will have to determine its cause in order to choose the appropriate corrective action. |
| Team knows (or can easily learn) the non-functional requirements but they are not delivering code that supports them because they overlook non-functional acceptance test criteria. | Training and/or mentoring to include non-functional criteria in the executable acceptance tests. | Product begins to support non-functional requirements. | (1) Learning curve time (especially to change long-standing habits). (2) Training and/or mentoring may not "stick" after trainers/mentors leave. (3) Organizational constraints may interfere with good practices. |
| Add non-functional requirements to each User Story. | Developers will remember to write acceptance tests and unit tests to cover non-functional requirements. | (1) The repetition of non-functional requirements in multiple User Stories is a form of muda. (2) Statements of acceptance criteria for non-functional requirements may vary across User Stories, leading to inconsistent application behavior. (3) Some non-functional requirements may still be overlooked when the User Stories are prepared. (4) The extended User Stories require more time to prepare, resulting in increased lead time. |
|
| Create a special category of Story to deal specifically with non-functional requirements. Include these stories in each iteration. | The special Stories call the team's attention to the non-functional requirements. | (1) If the non-User Stories have Value > 0, then metrics will show inflated numbers for overall value delivered due to repetition of non-functional requirements across multiple iterations or MMFs. (2) If the non-User Stories have Value = 0, then the customer may tend to prioritize them low, causing the product not to support the necessary non-functional requirements. In addition, the overall value delivered by the team will appear to be lower than it really is. (3) If the team is using timeboxed iterations and the non-User Stories have Points > 0, then metrics will show inflated numbers for Velocity, making it appear as if the team is producing a large quantity of work but relatively low business value. (4) If the team is using timeboxed iterations and the non-User Stories have Points = 0, then metrics will show unrealistically low numbers for Velocity, making projections of future delivery dates inaccurate. This makes the Velocity metric useless for its intended purpose. |
|
| Create a long-duration type of Story to deal specifically with non-functional requirements. Never move this Story to the "Done" column, since it will apply to all or most User Stories throughout the project. | The special Story calls the team's attention to the non-functional requirements. | (1) Because the long-duration Story is never formally closed, the team receives no credit for any work done in support of non-functional requirements. Any metrics used to track work capacity (Velocity), lead time (Cumulative Flow), value delivered (Earned Business Value), or delivered functionality (Running Tested Features) will be unreliable. Furthermore, the team's morale may be negatively affected since they will always have incomplete work left over at the end of each iteration or release. | |
| Record non-functional requirements in a central location such as a wiki, a SharePoint site, or an artifact such as a Supplemental Specifications document, and train the team to find them there. | There is a non-zero probability that some team members will remember to look for non-functional requirements some of the time. However, this solution does not really change the current situation with respect to availability of non-functional requirements. | The team may or may not remember to look in the central location to check for relevant non-functional requirements every time they should, since they must use a different mechanism to look up the non-functional requirements than they use to find the functional requirements. | |
| Record non-functional requirements on cards and put them on the wall in a designated area. These cards are not moved across the wall as work items are completed; they are present on the wall for convenient reference. | There is a high probability that most team members will notice the cards most of the time. | No particular downside, since the team is using the same mechanism to track both functional and non-functional requirements. | |
| If the team is using an electronic tool rather than a task board to manage its progress, then include non-functional requirements in the same tool as they use for functional requirements. | Benefit should be similar to that of creating non-functional cards to put on the task board. | No particular downside, since the team is using the same mechanism to track both functional and non-functional requirements. | |
| "Use my favorite tool!" If the team is already using the recommended tool but they are not taking advantage of its support for non-functional requirements, then they can start to use those features. | If the difficulty in supporting non-functional requirements is merely the lack of a suitable tool, then this recommendation may solve the problem. | It is likely that there are additional barriers to managing non-functional requirements than merely the lack of a tool. When this is true, then adding a tool to the team's existing overhead activities will have no noticeable effect on the requirements problem. | |
| "Use my favorite tool!" If the team is not already using the recommended tool, but they are familiar with the technologies involved with the tool and their existing tool set is consistent with it, then it may be feasible for them to start using the tool. | If the new tool is adopted early enough in the project to make a difference, the cost-benefit balance is acceptable, and the team is willing to take on another tool, then this may alleviate the problem with non-functional requirements at least partially. | (1) If it is already late in the project, it may not be worth the disruption to the team's pace to introduce a new tool. (2) If the cost of implementing the tool is greater than its value, then the existing problems may be less harmful to the project than the effort to introduce the new tool. (3) There may be additional barriers to requirements management besides merely the lack of a tool; in this case, even if the tool works perfectly and is happily adopted by the team, it may have no noticeable effect on the problem. |
|
| "Use my favorite tool!" The recommended tool comes from a radically different world than the team's working technology stack. The person recommending it is enthusiastic about the tool itself and has not considered the team's environment or skill set. | More harm than good. | Adding a tool that requires an unfamiliar technology stack that is inconsistent with what the team knows how to use will only add waste as team members will have to spend part of every day dealing with unfamiliar commands, UI conventions, and configuraiton details. They may spend so much time trying to make the tool run that they are never able to use it properly. They would get more value from a less-functional tool that works in their environment. |