effective software development

The beatings will continue until morale improves

Tue, 30 Jun 2009 14:35:59 -0500

I've been involved with agile software development for seven years now — doing it, reading about it, hearing about it, speaking about it, writing about it, teaching it, coaching it. No two organizations or teams put agile thinking into practice in exactly the same way. For that reason, there's always something new to learn; something I haven't seen before; something unexpected. It shouldn't surprise me to be surprised. Yet, I must say the last thing I ever expected to see was an agile sweat-shop.

I've worked in software development sweat-shops before. They used to be pretty common. Software professionals (in the US, anyway) used to take it for granted that they were expected to work 80+ hours a week all the time. Career burnout, divorce, and nervous breakdowns were the price one paid to work in the software field. It was considered normal. This was one of the reasons (along with role silos, lack of feedback, lack of collaboration, lack of accountability, lack of career advancement options, and soul-crushing bureaucracy) that prompted me to exit the IT profession in 2002, when a colleague caught me by the leg (not in the Russian sense) on the way out the door and introduced me to a different way of working. For me, "agile" rekindled the enthusiasm and passion that I had felt earlier in my career, before the bureaucrats took over the IT field. I'd sooner open a donut shop than go back to the Old Ways.

It may be because of that background that I was caught completely off-guard when I learned about an agile sweat-shop just recently. It wouldn't have occurred to me that a sweat-shop could be set up on the basis of agile values and principles. The two seem fundamentally incompatible. Nevertheless, it can happen. It's happening right now in a certain company, and that fact makes me wonder whether it's happening elsewhere, as well. Some people are rather antagonistic about the idea of agile software development. I've never really understood why, since in my experience it's been overwhelmingly superior to the Old Ways in every sense. If someone's experience of "agile" has been in a sweat-shop environment, then I can certainly understand why they would have a negative attitude about it.

The other day I heard about an XP team at a certain company that had decided to work four 10-hour days per week. It sounded interesting to me, since I've worked on XP teams that made the same choice. I was curious to know the reasons why this particular team had decided to try a four-day week. I remember the reasons why we did it on those past teams, and I still think they were good reasons:

Less ramp-up time at the beginning of the day and after lunch; eight ramp-ups as opposed to ten per week;
More focused time for development and collaboration; and (last but not least)
A three-day weekend.

From that experience we learned that there is a cost to working four 10-hour days. On an XP team, you spend most of your working time pairing. Pairing is an intense activity that demands one's full attention at all times. Working solo, one usually does not remain intensely focused continuously; focus ebbs and flows such that the mind stays fresh. With pairing this doesn't happen naturally. You have to break up the time intentionally, using an explicit time-management technique such as the Pomodoro Technique (or similar). Pairing is more like sprinting than jogging. You can keep going for a long distance when you're jogging, but when sprinting you have to stop and rest between runs. With the standard eight-hour work day, an XP team member is likely to pair about 5 to 5 1/2 hours per day, and not all in one burst. It's quite sustainable. With a ten-hour day, we found ourselves pairing about 6 1/2 to 8 hours per day. By the end of the fourth day, we were physically tired. We found that that "free" Friday had to be spent in recovery, as often as not. Ultimately it comes back around to the agile value, sustainable pace.

You can see why I was curious about the team that wanted to try the four-day schedule. I wondered whether they had considered the potential downside, or if they were just thinking about the fun they would have with three-day weekends. So I asked what had prompted their decision. Now I almost wish I hadn't. The more I learned about their situation, the more hairs stood on end on the back of my neck. They weren't looking forward to a long weekend. They weren't looking forward to anything positive at all. They were looking for an escape valve for needless pressure.

The first neck-hairs stood up when the team explained that they had not "chosen" to try a four-day work schedule in the sense that a self-organizing team is empowered to make such a choice. They had been begging their management and their HR department for "permission" to work four-days weeks for several months. So, this ostensibly "agile" team is managed in a command and control style.

Be that as it may, I have a hairy neck and that wasn't enough to raise a significant alarm. There may have been perfectly reasonable organizational reasons why one single XP team out of many shouldn't work on a different schedule than the rest of the shop. Command and control may have extended only as far as necessary. Perhaps foolishly, I asked for more information.

I learned that the team uses a time-management technique similar to Pomodoro to schedule pair-programming sessions and breaks throughout the day. This seemed quite reasonable to me, since I know how it feels to get into "flow" and lose track of time, and end up wearing yourself out. You can't work at peak effectiveness when you're exhausted. Hence the wisdom of controlling the amount of pairing time, not to mention the plain fact that certain activities don't benefit from pairing.

But there is a sinister side to the time-management technique. When team members have to take care of routine personal business during the day, such as visiting the doctor or picking up children from school, they are required to use up some of their accumulated vacation time. This draconian rule not only runs counter to agile values, but it also defeats the very purpose of vacations (to ensure employees have a chance to refresh themselves and reset their minds periodically) and may even violate Fair Labor Standards law by treating some (and not all) exempt employees as if they were hourly (but only at the employer's convenience). More hairs were now standing upright on the old neckeroo.

I had to wonder what might have led management to impose such a rule. The answer came from a separate discussion with some team members about pairing. They asked what the "extra" person was supposed to do. At first I didn't understand the question. How can you have an "extra" team member? What does that mean? They explained that they were required to follow the rule that production code could only be touched by a pair. They were not permitted to work on production code alone. Thus, when there happened to be an odd number of team members present on a given day, there was an "extra" team member who was not permitted to touch production code.

I suggested that they may have misunderstood the mechanics of pairing, or that they were being a bit extreme (no pun intended) about practicing pairing. When there is an odd number of developers, then someone works without a partner. The next time people switch around (and that won't be very long; just the length of one Pomodoro or however else the team manages its time), someone else will be left to work solo for a while. It's not a big deal, especially when the team is following most or all the other XP practices, since that would result in a high bus number, and a high bus number makes it feasible for individual team members to be out-of-pocket once in a while without crashing the project. I really didn't understand what their problem was. I started to probe them about their understanding of the rationale for pairing.

It quickly became clear that this team follows XP practices for one reason only: Their management requires it. The "reason" they pair is that "management doesn't trust us to work on code unless someone is looking over our shoulder." (I know from separate discussions with management that this is not actually true, but sometimes perception trumps reality.) They are unable to describe the value proposition of XP as a whole or of any particular XP practice. "It's a rule." "Management makes us do it." "We'll get in trouble if we don't do it." They are not making informed professional judgments based on principles. They are merely following rules. That means nothing less than this: They value processes and tools over individuals and interactions. At this point, the back of my neck was at DEFCON 1. What I saw before my eyes was the heretofore unimaginable: An agile sweat-shop.

This team is so stressed-out by the fact they can't pick up their kids or get a tooth filled without burning some of their vacation time that they're willing to take on an even more stressful work schedule just to free up a weekday for routine errands. Horrors!

Of course, practitioners will recognize that this isn't "really" an agile team because they're "doing it wrong," but I hesitate to call that particular spade by name because of the vocal minority who like to bash the supposed "agile religion." Yeah, you agilists (popularly mis-spelled agilest, for reasons unknown) always say "they did it wrong." How convenient! Okay. Whatever. Call it what you will. (They did do it wrong, though. A rose by any other name, etc.)

So, what could have led to this sorry state of affairs? It turns out to be a question of interpretation; specifically, interpretation of one of the seminal books about XP, Extreme Programming Installed, by Ron Jeffries, Ann Anderson, and Chet Hendrickson. Published four months before the Snowbird meeting, the book quickly became a guidebook for many budding agile teams. It became so widely known that for several years it was identified in casual conversation as, simply, The Pink Book.

The agile community has learned much and made many advances in practice since then, but The Pink Book remains basic reading for those who are interested in applying agile software development methods. There's one statement in particular, at the head of Chapter 12, on page 87 in the paperback edition, that speaks to pair programming. When I look at page 87, these are the words I see:

On an Extreme Programming team, two programmers sitting together at the same machine write all production code.

Here's what it means: Ideally, most production code should be developed by a pair who use other XP practices while they are working together, because by applying this set of practices they can produce clean code with few defects without wasting much time. If it's feasible and reasonable, all production code ought to be built this way, to take full advantage of the benefits of pairing. Short of that, pair as much as makes sense while keeping the single most important factor in mind: effective delivery value to the customer. It's a very good guideline. And that's just what it is: A guideline.

Problems occur when people interpret the statement not as a guideline, but as an inviolable rule that supercedes everything else, including the effective delivery of value to the customer. Those people would rather sit idle, waiting for a pairing partner, than to deliver value to their customer. When they look at page 87, they see different words than I see:

Brother Chet reads from the Book of Jeffries, Chapter 12, Verse 1: Firstly shalt thou code in pairs. In threes shalt thou not code; neither shalt thou code as one, excepting that thou then proceed to two. Four is right out! Amen.

This very slight difference in interpretation leads to very different implementations of XP, some of which are darkly familiar to a hairy-necked survivor of the Old Ways. Don't get me wrong — it's a great book.

But it's only a book, you know.

Follow-up on the recent PMI panel discussion

Thu, 25 Jun 2009 09:28:52 -0500

I received an email from Jesse Fewell this morning thanking the participants in the recent agile panel discussion hosted by PMI, where I had the honor to serve as a panelist. He pointed to the write-up of the event he posted on his blog. His post includes a couple of photos and a summary of some of the panelists' comments.

It certainly was a pleasure to work with several of the big brains in the agile community - Richard Cheng, Linda Cook, and Rodney Bodamer. I look forward to working with them again soon.

Mentoring the value of pairing

Tue, 23 Jun 2009 06:07:37 -0500

I'm coaching a team that is relatively new to techniques like TDD and pairing. A team member asked me to pair with her the other day to have a look at her unit test cases. Like many people who are still getting used to building up a suite of automated tests, she had been writing Happy Path tests and only an occasional abnormal case. She felt that she wasn't considering enough test cases, but she wasn't sure what to add. In the course of the pairing session, we started building a new feature in the application and wrote test cases for boundary conditions, null arguments, exceptions, resource unavailable situations, and so forth.

The TDD coaching as such was pretty typical, but the interesting thing was that she asked why people should pair unless they're transferring knowledge or mentoring one another. I took the opportunity to demonstrate one aspect of the value proposition of pairing. As we worked, whenever one of us helped the other by noticing a minor mistake — mis-keyed keyword, missing closing tag, forgotten semicolon, etc. — I made a tick-mark on a piece of paper. Furthermore, whenever we had a brief design discussion as we were working — pull this method up? break up this if/else block? etc. — I tracked it with a second set of tick-marks.

After our last check-in for the session, we took a few minutes for a mini-retrospective. I asked her to make a reasonable guess about the amount of time we had saved each time one of us noticed a minor coding error in the moment. She figured each one was worth about 30 seconds, based on the notion that finding the error after the fact while debugging would have taken about that long on average. We also talked about the brief design discussions that had taken place. We figured that on average each of the resulting design improvements prevented two hours of future code maintenance effort by avoiding a little bit of technical debt.

In one of his early books, Alistair Cockburn computed that the cost of an IT professional amounts to about $2.10 per minute. That's a little higher than my own scratch calculation, but he's a Doctor so I'll go with his numbers. In our pairing session we had two mini design discussions that led to small refactorings. By our reckoning, that saved four hours of future code maintenance effort. That's worth about 2.1 x 120 = $252.00. There were 12 moments when one of us noticed a minor mistake. If those occurrences saved an average of 30 seconds debugging effort, then it was worth .5 x 2.1 x 12 = $12.60. In total, we saved the company $274.60 in 90 minutes' time, or an hourly saving rate of about $180.00.

Bear in mind these numbers are just rough-and-ready estimates based on our observations of a single pairing session. They may be high or low. The calculation also ignores the opportunity cost we avoided by keeping the code base clean. What I mean by "opportunity cost" in this context is the cost of the time developers could not have spent on value-add work in future modifications, if they had to spend the time working around the technical debt that we avoided. (That's a clumsy way to say it; I hope it's clear enough.)

The company has a small IT department with a total of about 40 developers working in several XP teams. If we assume the developers pair about 5 hours a day, then that's 20 pairs x 5 hours x 5 days per week = 500 pairing hours per week. Assuming a savings of $180 per hour per pair, that makes the average hard savings attributable to pairing $90,000 per week. If that rate of work is more-or-less constant throughout the year, and the teams work 50 weeks a year (it's in the United States, so vacation time is short), the company enjoys a savings of about $4.5 million per year specifically because its software development teams pair-program as a standard practice.

Of course, by projecting the results of one pairing session in this way we are subject to the effect of cumulative error. If you think the numbers are inflated, then go ahead and chop them in half and say that the company is saving a mere $2.25 million per year. The precise number isn't really the point; the point is that pairing really does deliver hard value. Given that a professional's time is the most expensive component of IT cost, techniques that save professionals' time are well worthwhile.

The team member found the numbers quite surprising. I think her reaction is normal. We humans tend to look for dramatic, memorable events as evidence that something yields value. Pairing rarely generates dramatic, memorable events. The value of pairing comes in the form of very small course corrections that prevent errors from occurring in the first place. The course corrections are of such small scope and they occur so seamlessly within the pair's work flow that they are usually not noticed at all.

This is as it should be; clean code should be a natural outcome of our work, and not something that calls attention to itself by virtue of being abnormal. The value is easy to overlook because the effect of pairing is to prevent situations that would have caused extra work at some future time. Defects do not enter the code base. Technical debt does not accumulate. It's not easy to observe or quantify these effects after the fact, because the bad stuff never happens.

PMI PMTools meeting at Tyson's Corner, Virginia

Tue, 9 Jun 2009 11:23:43 -0500

I'm honored to have been invited to participate as a panelist at the upcoming PMI PMTools meeting in Tyson's Corner, Virginia, on Tuesday, June 16. The other panelists are Linda Cook, Richard Cheng, and Rodney Bodamer. All four are consultants in the lean/agile space. We will be taking any and all questions from meeting attendees. A few starter questions have been prepared by the meeting organizers in case no one has anything to ask (yeah, right). These have to do with the challenges in implementation, whether agile methods live up to the hype, and how the PM's job changes when an organization moves toward agility. I think it's going to be informative and a lot of fun. You can register here: http://www.eventbrite.com/event/309542851. I hope to see you there!

Success factors for self-organizing teams

Mon, 8 Jun 2009 09:25:44 -0500

The notion of the self-organizing team is fundamental to agile thinking; but not everyone agrees that self-organizing teams are practical. As for me, I think the viability of the self-organizing model depends on the team members, their management, their organizational context, and (possibly) on the nature of the work they do. If we limit the discussion to software development teams, then I would say there are three critical success factors for growing effective self-organizing teams.

1. Clearly define the boundaries. The team has well-defined boundaries within which they may feel free to self-organize and self-manage. Without this, teams aren't sure how far they can go or how much they should attempt to do to manage themselves. When people are uncertain about this, they tend to err on the side of caution and do little or nothing without asking management's permission. That pretty much nullifies the benefits of self-organization.

2. Tolerate mistakes and allow time for learning. The organization is tolerant of mistakes and of learning-curve time. All too often, management declares teams to be "self-organizing," and the first time the team makes a mistake they step in and take "control" of the situation. Once management does that, the self-organization experiment is finished. It will take a very long time and considerable effort before teams will believe management is serious about allowing them to try and self-manage again.

3. Keep the team challenged, yet not frustrated. The team's administrative or functional manager is sensitive to the team's limitations, and adjusts boundaries and expectations carefully so as to provide the team with sufficient challenges to keep them in a state of learning and growing, while not expecting more of them than they can handle. I think it is unlikely that every team in any given organization will be at the same level of readiness and maturity for self-management. Each team will need its own boundaries and challenges so that the organization as a whole can improve its effectiveness.

Issues in remote collaboration

Mon, 8 Jun 2009 09:21:12 -0500

An interesting thing happened the other day in a meeting of a working group charged with creating a roadmap for organizational transformation. This post isn't about organizational transformation as such, though; it's about an aspect of remote collaboration that became obvious to me during the meeting.

The working group comprises several management-level people in the client company plus three agile/lean coaches, two of whom are consultants. The meeting took place in two locations using an audio/video link and network-based remote collaboration tools. Thus, the barrier to effective remote collaboration that people most frequently talk about was handled: Technologies were in place to bridge the distance, and they worked well. Also, this form of collaboration is routine in the company and the situation was a familiar one for the participants.

The details of the meeting agenda are not important for my purpose here. The way the meeting flowed was that by the time we had used about half the allotted time, we were bogged down and making no progress. Key people were disengaged, including the senior-most manager in each location and one of the coaches in each location. In other words, the key decision-makers and two of the three people who ostensibly know how the organization ought to transform had stopped participating.

I noticed this had happened, but didn't see where to begin fixing it. Then, one of the managers on the local side of the link (from my perspective) spoke up and said that the suggestions the others had just made indicated they did not trust the team members in our location. The ensuing uncomfortable silence was broken when a participant pointed out that trust is a fundamental agile value. More silence. I took the opportunity to mention transparency is another fundamental agile value, and suggested that to get things moving again perhaps the people on the remote side of the link could clarify their feelings and intentions. The two managers in that location did so, and the result was not only to clarify that they did indeed trust the rest of the team, but also to expose an action item for us to address after the meeting. The feeling of not being trusted had been a misunderstanding.

Yet, the coach on the remote side did not clarify his feelings or intentions. He remained silent. I took it to mean he was not ready to talk just then, and accepted the small victory of having clarified some intentions and having identified a practical action item. I remained alert for an opportunity to draw out the coach's concerns.

The meeting concluded with a summary and mini-retrospective. When a natural pause occurred, I took the opportunity to recap the earlier incident about trust. I thanked the two managers on the remote side of the link for their candor, and mentioned that I didn't recall what the coach had said at the time. At that point he finally expressed his concerns, with a certain amount of anger or frustration in his voice. He felt as if the members of the working group at our location had gone forward with decisions and actions without consulting him. He felt we were working "in secret," as he phrased it. I think it's important not to invalidate or deny someone else's feelings. If that is the way he felt, then something must have happened that caused him to feel that way. I asked the whole group what had happened that might have led to that sort of impression. We want to understand how we gave him that impression so that we can change our behavior to avoid repeating the situation. That seemed to result in a constructive outcome; people agreed to be more proactive about including remote members of the team in any decisions or significant discussions.

Obviously, just getting the concerns out into the open doesn't magically solve the problems. Equally obviously, we can't solve a problem that no one will talk about. So it was a valuable step, I think.

We have these working sessions daily. In the two days following the incident, I noticed that the general tone of the meetings was far more positive. There was no undercurrent of tension and no indications of mistrust or suspicion. At a minimum, people are making an effort to keep things on a constructive level and to look forward instead of backward. It's possible that the change runs deeper than that. In any case, we're making progress toward the goal now.

The lessons for this form of remote collaboration that I took away from the experience are these:

Help the people on the other end of the link hear you. Speak slowly and clearly. Don't slur your words or allow your voice to trail off at the end of sentences. Don't run sentences together. Face the microphone; don't face the whiteboard or other materials that may be stuck on walls or windows when talking. Learn to pause at natural points in the narrative and not just whenever you happen to run out of air.
Don't expect people on the other end of the link to be able to read material that is posted on the walls or windows of the room you're in.
Ask clarifying questions. If you made a statement, ask people on the other side of the link to repeat it back to you in their own words. Don't worry about repeating yourself; it's important that people have the same understanding of what was said, and you're not wasting time if repetition helps ensure that.
When someone on the other end of the link makes a statement and they don't think to ask you to repeat it, take the initiative to repeat their statement back to them in your own words and ask whether that is what they meant. This will not make you appear stupid; you're just trying to make sure everyone has the same understanding of what was said.
If you need to clarify something with a person who is in the room with you, avoid engaging in a "private" side conversation. You are visible on the video feed, and it comes across as conspiratorial. People at the remote location will think you're keeping secrets from them. In addition, if you are speaking above a soft whisper your voice may interfere with the audio feed. If you need to ask a question, however brief or minor, wait for an appropriate time to interrupt the flow of discussion and ask the question to the whole group, or at least "in front of" the whole group.
If interruptions or multiple simultaneous speakers becomes a problem, pause and agree on some sort of visual signal to indicate that a person wants the floor.
Become aware of your own nervous tics, such as tapping on the table where the microphone is located with a pen or with your fingernails. It may sound pretty loud on the other end of the link.
Sit or stand in a place where the camera can see you. Otherwise, it gives the impression you're hiding from the people on the other end of the link.
People appreciate eye contact in conversations. When you look at the image of a person on the monitor, it appears to them as if you are looking off to one side. Try looking directly at the camera, to give the impression of eye contact across the video link.
Keep in mind the Retrospective Prime Directive (even if the meeting isn't a retrospective). Resist the temptation to assume people have a hidden agenda. Occasionally they might have, but most of the time misunderstandings are simply a result of distance and not of bad intentions. It's all too easy to initiate a vicious circle of mistrust, and easy to keep the mistrust going by failing to question the assumption that someone has a hidden agenda.

Focus on how we can, not on why we can't

Mon, 1 Jun 2009 21:22:31 -0500

I mentioned that Pierluigi Pugliese and I had discovered many of the approaches and techniques the two of us use in coaching and in problem-solving are quite different and complementary, through our interactions at XP2009. On my first day back on the job, I found an opportunity to use one of the techniques he demonstrated in the Open Space session he facilitated last week.

Key client managers and the two agile/lean coaches (Cindy Bloomer and I) held a meeting today to chart a roadmap for organizational improvement over the next three quarters. Meeting last week while I was away at the conference, the others had already developed a list of observations of current problems and options to address them. In today's meeting, we hoped to get a start on the roadmap by defining the desired end state. After the first half hour or so, it seemed to me that we were thrashing. We were tangled up in the detailed observations of problems and a discussion of possible tactical solutions to those problems. This was nothing at all like a roadmap. People were beginning to get frustrated.

The group began to slide into "plan-the-next-meeting-without-accomplishing-anything-substantive-in-the-present-meeting" mode. I remembered the technique Pierluigi had demonstrated last week, and I proposed that we change gears. I pulled a whiteboard into the middle of the room where everyone could see it clearly (some meeting participants were on a video link). I wrote the title, Happy Company at the top (only I wrote the real company name) and asked the group what it would feel like to work there. After a long pause, they began tentatively to suggest ideas.

The approach did not come naturally to everyone. One person suggested, "Resources are properly aligned with the business." I asked her if she meant that the chairs were lined up straight. Everyone laughed. She said, "No, I meant that people were working on the right things." I said, "Ah, you meant people then, and not resources." Everyone nodded in acknowledgement. I wrote, People are working on the right things. The pace of idea-generation picked up, and soon we had a whiteboard filled with brief statements of characteristics describing the Happy Company, mostly devoid of bizspeak like "resources" and "aligned." I knew they were getting the hang of it when someone suggested, "People are having fun." By that point, we were having fun. We were thinking about what was possible rather than what was wrong. Another suggestion was, "People focus on how we can, not on why we can't." I love that one!

I couldn't help noticing the difference in the "feeling" in the room before and after we stopped talking about problems and started visualizing the Happy Company. At one point, as I mentioned, we were somewhat stuck. People were frustrated, spoke in low tones, and there were no smiles. At the end, having generated a list of characteristics of Happy Company, everyone was smiling, voices were lighter, and we left the meeting relaxed and cheerful. It's amazing how much power there is in making just a minor change in wording or perspective.

Tomorrow we'll start with this description and work backward to discover a path that leads from the present state to the Happy Company. The others are thinking this process might take a couple of weeks. I have a feeling it won't be as hard as that, provided we can resist the temptation to line up the chairs.

<gesture type="raise-glass" content="Chianti">Grazie, Pierluigi!</gesture>

XP2009: My report

Mon, 1 Jun 2009 21:19:35 -0500

The Venue

There is a saying that any publicity is good publicity. Vaudeville and Broadway legend George Cohan is credited with saying, "I don't care what you say about me, as long as you say something about me, and as long as you spell my name right." It is in that spirit that I do not mention the name of the hotel where the conference was held. It doesn't deserve even bad publicity.

With clean white beaches, a beautiful emerald-and-blue sea, sweet cool breezes, and a night sky resplendent with stars, the place would be an ideal spot for a romantic getaway — assuming, that is, that they fixed the plumbing and the air conditioning, improved the quality of the food, installed window screens to bar the mosquitoes, and provided reliable cell phone and internet services. It is not, however, a suitable location for a technical conference. 'Nuff said.

Process Smells and Root Cause Analysis

Monday afternoon I facilitated a workshop entitled Process Smells and Root Cause Analysis. I haven't yet uploaded the workshop materials to my wiki, so I can only refer you to the description in the conference program. This is a new workshop that I was running for the first time. The workshop seemed to be well received and those participants who spoke with me afterwards told me they got value from it. It really functioned as two different workshops, since people left at the break and other people joined after the break.

The basic idea is that a process smell, much like a code smell in a code base, is an observable characteristic of a process that is sufficiently questionable to warrant investigation to determine whether there is a deeper problem. Starting with simple problems and progressing to more challenging ones, we practiced using three root cause analysis tools: Five Whys, Ishikawa Diagram, and Diagram of Effects. There were enough participants (both before and after the break) to have two separate break-out groups working on each scenario. They came up with different analyses, which led to some very interesting discussions. I suggested a "smell" to be investigated using each tool, and participants suggested smells of their own for each tool. We were able to complete six analyses, doing two with each of the three tools I introduced. Of course, there are many more root cause analysis tools than these three, but they were not included in the workshop.

I was honored to have Diana Larsen, noted consultant and author and chairman of the board of directors of the Agile Alliance, participate in the second half. All the participants contributed actively to the session. I thank them all, and make special mention of Werner Wild and Fabio Armani, who contributed additional smells and facilitated break-out groups during the workshop.

One participant, Pierluigi Pugliese, contributed a great deal to the workshop. He made an intriguing comment at one point, that his own approach to these issues is diametrically opposite the approach I was covering in the workshop. He was inspired to propose an Open Space later in the week in which he presented quite different ways to deal with issues such as "process smells." Pierluigi and I had several long conversations during the week about coaching styles, techniques, and tools. Ultimately I suggested that he and I enter into a peer mentoring relationship to help each other grow as coaches, and he agreed.

I learned about the peer mentoring idea from Pascal van Cauwenbergh and Portia Tung, although I don't know whether they invented it. The idea is that two people who don't work in the same company, the same industry, or even in the same country help one another with ideas, problems, and general professional growth on an ongoing basis. It's a powerful idea that is starting to catch on, if only in a small way so far. For me, finding a peer mentor was an unexpected outcome of the workshop and one of the high points of the conference.

Test-Driven Development Performing Art

Emily Bache facilitated a session structured along the lines of a coding dojo whose purpose was to demonstrate TDD technique for people who were new to the practice. She invited several people to prepare demonstrations in which they would pair program the kata of their choice with a focus on demonstrating and teaching TDD to those attending the session. Lasse Koskela and I were among the invited "performers."

For the first demonstration, Thomas Nilsson and Andreas Larsson of Responsive Development Technologies in Linköping, Sweden, demonstrated the classic Bowling kata, created by "Uncle" Bob Martin. At a coding kata session at XP2006, they learned the Bowling kata for the first time. They liked the idea so much that they founded Coders Dojo Sweden when they returned home. For this demonstration they used Java and JUnit with Eclipse. Since the goal was teaching rather than direct practice, Thomas did all the coding while Andreas narrated the sequence of steps for the audience. It was an effective demonstration.

Next to take the stage were Danilo Sato and Patrick Kua, who work for ThoughtWorks UK. Danilo is originally from Brazil, where he founded the Saõ Paulo dojo after participating in Emily's dojo session at XP2007. They did a demonstration using cucumber and ruby with rspec. It was a nice demonstration of storytest-driven development (with cucumber) and behavior-driven development (with rspec), and also showed how rapidly a working application can be produced with ruby. Their development environment was TextMate on Mac OSX. The only criticism from the audience was that after writing the initial story with cucumber, they never closed the loop by going back to cucumber to show a green acceptance test.

Thomas Sundberg and Tobias Anderberg of Agical in Stockholm, Sweden, did the third demonstration. (I really like one of Agical's slogans: "Software is developed by people, not resources.") They used a new kata they created themselves in which the goal is to parse text files that represent blackjack hands as part of a program to train a neural network to play the game. They worked pretty fast, and at one point the audience had to ask them to pause and explain what was going on. Apart from that, it was an interesting idea and a bit different from the usual. Their performance generated some interesting discussions about test isolation and unit test scope.

Lasse and I were last on the program. Because the goal of the session was to demonstrate TDD technique rather than just to practice katas, we decided to do something a little different. We started with a "legacy" scenario and worked on a User Story to make an enhancement to the legacy system. We practiced our "project" using Java with JUnit on eclipse. Later we thought it would be cool to show JUnitMax, an extension to JUnit that runs tests automatically on every save. After the session started, but before we were scheduled to go to the front, Lasse told me JUnitMax had maxed out his machine's memory. Maybe that's why they call it JUnitMax. ;-) Anyway, at the very last moment we decided to run the demonstration with JDave, a BDD tool for Java. It's an Open Source project hosted on Laughing Panda, the Open Source incubator supported by Lasse's employer, Reaktor Innovations in Helsinki, Finland.

Since I was unfamiliar with JDave, we decided to follow Thomas' and Andreas' lead and have the same person at the keyboard throughout our demonstration. As we were getting started, I mentioned to the audience that I didn't know JDave, but I thought the product had a great name. It's a good tool. It uses intuitive BDD syntax and it's easy to create fixtures. JDave also incorporates JMock 2, so when you need a mock you've got a mocking framework under your fingers already. Lasse did the driving and I did more than half the talking.

Our "legacy" scenario was designed to keep things light and fun. We assumed a product called Emily's Server was in production. Emily's Server handled arbitrary client requests in one of the world's oldest industries. Emily was concerned that transaction durations were highly variable. In our scenario, Emily had given us a Story Card that read as follows: "As Emily, I want transaction durations to be long enough that I will be fully satisfied." That got a nice laugh from the group, including Emily, who did not know in advance that we were going to do this. Using a BDD approach and making borderline-appropriate comments along the way, we added code to filter transactions and calculate their duration. In addition to demonstrating BDD and JDave, we showed a technique to isolate unit tests from the system clock.

At that point I held up the Story Card (which was actually blank, of course) and said to Lasse that Emily had neglected to specify acceptance criteria. I asked her how long a transaction had to be in order to satisfy her fully. Group laugh. She said, 30 minutes. I protested that we were "only mortal men," and asked her to think of something in the range of seconds. Another group laugh. She said, 30 seconds. Lasse asked her if it could be 20. I told him to take his vitamins, and we continued the demonstration, adding code to support a "broadcast notification of satisfaction" whenever a transaction met or exceeded a duration of 30 seconds. Not exactly a kata, but then again, it wasn't exactly a dojo.

Emily was a really good sport about it. She went along with the theme as if it had been scripted. Afterwards, Charlie Poole stopped by to say, "I'm glad to see metaphor isn't dead." I replied, "Not dead; just badly bruised!"

Business Value Game

Pascal van Cauwenbergh and Vera Peeters are quite creative about developing games that teach agile and lean principles experientially. An interesting game that they've been testing out in between sessions at conferences over the past several months is the Business Value Game. I was never able to play along due to schedule conflicts, but I've been interested in the game. The game is provided under a Creative Commons license, so you can download and print the materials and run the game yourself. The only additional supplies you will need are a few dice. I suppose if they can figure out a way to download solid objects, the dice will become available online, as well.

Artem Marchenko and Vasco Duarte were scheduled to run the game at this conference, but Vasco was unable to go. I was very pleased when Artem asked me to fill in. Sorry to miss Vasco, but I had been wanting to learn to facilitate the game, and this was a fine opportunity. I think the game will be helpful at my present client, so the timing was very good.

Although we had only seven players, it was enough for two teams and the game went fairly well — despite a couple of n00b mistakes I made in advising the players about the rules. It's a very well thought-out game that brings out various aspects of work prioritization and scheduling, release planning, balancing return against cost, practicing continuous improvement, and managing the happiness of multiple customers while working with limited delivery capacity. To get all that from a relatively simple and enjoyable table game required a certain amount of creative genius on the part of Pascal and Vera. All teams have the same work to deliver under the same conditions, yet no two teams achieve the same result because they take different strategies and make different trade-offs as they go along. Artem did a fantastic job setting everything up, and the players all said they got a lot of value from the experience. I'm looking forward to running the game in future.

The New New NEW! Product Development Game

This is a conference that is mainly for experienced practitioners and researchers. As such, most sessions are filled with peers, and they tend to take the form of an interactive group effort rather than an expert teaching a group of novices. Many of us take the opportunity to run new material so that we can get knowledgeable feedback from participants. One example was this game, created by Marc Evers and Willem van den Ende, agile consultants in the Netherlands.

The idea is to have different teams work through the same project using different agile techniques to explore how the techniques differ. Marc and Willem prepared a fairly comprehensive set of story cards for a hypothetical development project, as well as cards representing events that might occur during a project, variations in velocity, team member absences, and other factors. We divided into two teams and stepped through the project using user story mapping, dimensional planning, and kanban.

The results were promising, although the game needs further development. It turns out that user story mapping and dimensional planning are pretty similar. There isn't much to discover about differences between them. In general, we seemed to move through the cards faster with the kanban system, but there was no sense of the big picture. The other models gave us a clear view of the relative business value of each story. I think what was missing is not a failing of kanban as such, but that because of the way the game was designed there was no planning with the kanban variant. The development team just worked through the cards in the order presented. User story mapping and dimensional planning are techniques for determining which features to build to deliver the most business value in the shortest time. This was missing from the kanban technique as presented in the game. In fact, I see no reason why either of those planning techniques couldn't be used in conjunction with a kanban development process. The setup also didn't bring out certain key elements of the kanban approach, including customer pull, controlling work in process, and stop-the-line events.

In my opinion, it would be interesting to set up the game for new product development and for ongoing incremental enhancement of an existing product. If the kanban setup is modified so that there are more states between "started" and "done," and playing cards are added depicting situations that drive decisions about changing WIP limits or stopping the line, we may be able to get a truer sense of the differences among the various agile approaches represented in the game.

Telling Your Stories: Why Stories Are Important For Your Team

This was another new workshop that was run, in part, to get feedback from participants and to work out the kinks. Diana Larsen and Johanna Hunt created this workshop to explore techniques for organizational storytelling as a way to bring out issues and assumptions that may not be obvious or that may be difficult for people to discuss openly, and to help people understand what is really going on in their organizations. They used tools such as the Archetype Storytelling Cards and Once Upon A Time Cards to give participants a starting point and some guidance in how to craft stories.

The process was left very open-ended, and teams were free to take the exercise in any direction that made sense to them. They wrote their ideas on sticky notes which were then posted under specific categories and discussed by the entire group. I'm not quite sure what was achieved, but my impression is that the workshop needs a longer time slot. I felt as if we were only getting started when the clock ran out.

Johanna will be running this workshop again at Agile2009 with Rachel Davies. I'm definitely planning to participate again and see if we can take the idea further. It's a compelling concept.

Keynote: The Cultural Assumptions Behind Agile Software Development (Mary Poppendieck)

I've yet to see a talk by Mary that I didn't like. This one was a really interesting survey of general cultural differences around the world, and how the values and principles of the Agile Manifesto might affect organizations and teams in regions that have quite different cultural characteristics than the United States around the turn of the century. The basis of the cross-cultural comparisons was the well-known work of Geert Hofstede.

Mary showed how the assumptions underlying the Manifesto came from American culture in response to problems typical in American companies; problems that stemmed from American cultural characteristics. She proposed that the first value statement in the Manifesto, which emphasizes individuals and their interactions over processes and tools, reflects the American cultural emphasis on individualism. According to Hofstede's Cultural Dimensions, the United States, United Kingdom, and Australia score very high in "individualism." This may be less relevant in a culture that emphasizes collective values more strongly, such as that of the Scandinavian countries. She presented a number of other very interesting examples, such as the differences between Japanese and American decision-making and project execution patterns, that give agile values and principles different meanings in different cultural contexts. At an international conference like this one, the subject was especially relevant.

Keynote: What they don't teach you about software at school: Be Smart! (Ivar Jacobson)

I was very interested to see and hear Jacobson in person, since he is one of the legendary figures in the history of software development. I was pleased when he had some problems getting his presentation to show up on the screens, since that usually means the speaker has to work without slides. That's when you can really tell whether the speaker knows the subject well. Unfortunately, Jacobson told a couple of jokes while waiting for the technician to get the slide show working. Perhaps he was keen to show the clever little animations he had built into many of the slides. It was good that the slides had some entertainment built in, because the content was surprisingly light. The whole thing seemed like a canned corporate sales presentation for his training and consulting services. This really wasn't the right audience for that sort of thing. Disappointing.

Keynote: A Journey Beyond budgeting - "Because the future ain't what it used to be" (Bjarte Bogsnes)

Bjarte Bogsnes is Head of Performance Management Development at StatoilHydro, a Norwegian petroleum company with operations in some 40 countries. He has developed an approach to management that dispenses with the annual budget cycle in favor of pushing decisions out through the organization and enabling business decisions to be made at the last responsible moment. He calls the approach Beyond Budgeting, and it has become something of a "movement" in industry. His book, Implementing Beyond Budgeting: Unlocking the Performance Potential, describes the approach more fully. In his talk he explained with clarity and panache just why traditional management assumptions and practices don't work, and why the approach now in place at StatoilHydro and several other companies does work.

Bjarte's keynote talk was on Friday morning, and in the afternoon he ran a tutorial about the Beyond Budgeting approach. Later, about 35 of us went to dinner together away from the hotel, and I happened to sit across the table from him. We continued the discussion of Beyond Budgeting and other issues, along with several other people seated nearby. He is very passionate and enthusiastic about these ideas, to spend an entire day talking about little else!

I must say that I think these are game-changing ideas. Bjarte is not a technologist and his concept is not based on agile software development; it's purely pragmatic business thinking that supplies key pieces of the puzzle beyond the scope of agile methods and outside the professional experience of agile practitioners. Moreover, the ideas have been implemented and proven successful in several large companies. The ideas are important for creating organizations that can embrace and facilitate lean and agile thinking seamlessly.

...and the rest of it

As is often the case at conferences with such an elite level of participants, the conversations in between scheduled activities and in the evenings proved to be among the most informative and enriching experiences of the week. Surrounded by thoughtful, intelligent people who care deeply about their work, in an environment that all but forced me to socialize more actively than I normally do (since there was no practical internet connection and no working air conditioning), I couldn't avoid learning something useful every day.

The funny thing is that my wife strongly and repeatedly advised me to take advantage of the opportunity to eat mass quantities of delicious Italian food, since Italy is famous for its food. In the remote location where the conference was held, there just weren't many choices for dining. The Swedish barbecue the Agical guys hosted on Thursday evening turned out to be the best meal of the week, by far. The conference was held in the middle of nowhere. I asked Charlie Poole about that on the last day, and he explained it was a tradition of this particular conference to choose venues that are off the beaten path. Personally, I think there's a lot to be said for the beaten path. A selection of restaurants, for example. And reliable internet service. And cell towers. And steady water pressure and temperature. Things like that. Nothing too important, I guess.

XP2009 is next week!

Thu, 21 May 2009 10:52:39 -0500

I'm excited about next week's XP 2009 conference in Sardinia. I really like the practical, hands-on sort of events much more than the talking-head events. XP 2009 will feature 20 workshops and 15 tutorials and plenty of Open Space opportunities. There's also an academic side to the conference for which 22 papers have been accepted.

I'll be facilitating a workshop on process smells and root cause analysis. A process smell is an observable symptom that may (or may not) indicate an underlying problem. Root cause analysis can help us determine whether there is a problem and, if so, what sort of corrective action is appropriate. We'll be looking at simple, complicated, and complex problems and applying different tools to each type, including Five Whys, Ishikawa Diagram, and Diagram of Effects. Some smells are pretty obvious: "Defect count is rising over time." Others might not smell very bad at all, at first sniff: "The team never experiences any difficulties at all in meeting its commitments."

I'll also be joining Lasse Koskela to participate in Emily Bache's Kata Workshop. Four pairs will each demonstrate TDD for 30 minutes, followed by questions and discussion after each round. This sort of thing is always a lot of fun. I don't get a chance to pair with Lasse very often since one of us lives on the wrong continent. (We're still working out which of us that is.)

There are quite a few compelling sessions on the program. What may not be clear from the name of the conference is the fact that the content spans the range of leading-edge ideas and emerging practices in the software industry, and covers all aspects of the work from ground-level software engineering practices to process frameworks to human factors. The program includes familiar topics like Scrum and XP and newer, lean-inspired topics like Kanban covered in practical, hands-on tutorials and workshops facilitated by industry thought leaders like David Anderson, Mary Poppendieck, Joshua Kerievsky, and many more. It promises to be an enriching experience for everyone. I hope to see you there!

Measuring the soft stuff to encourage desired behavior

Fri, 15 May 2009 08:24:12 -0500

I met several very interesting people at last night's APLN meeting. I don't mean to slight anyone by failing to mention all the names, but I do want to mention one: David Schmaltz, whose online writing I have enjoyed for some time. He made a number of insightful comments during the presentation, and I picked up a couple of ideas I'd like to put into practice as soon as possible on my current engagement. We chatted after the session and he offered some practical pointers for using the ideas effectively.

One of the hardest things to measure is customer satisfaction. David mentioned the book, The Ultimate Question, by Fred Reichheld. This is a well-known book already, but it was news to me as of yesterday. I guess that shows you how wisely I spend my spare time. Anyway, the question itself is this: How likely is it that you would recommend this company to a friend or colleague? The answer is given as an integer in the range 1-10, where a low value indicates a low likelihood of giving a recommendation, and a high value indicates a high likelihood. It struck me that a variation on this question might be a good way to get feedback from a team's customer: How likely is it that you would recommend this team to deliver software for a colleague who runs a business unit like yours? If we asked for this feedback every few weeks, we could track a trend that shows a team's success in satisfying their customer.

I'm thinking of the Ultimate Question as a motivational metric, in context. The client organization is such that there are three layers of people involved in delivering business software internally: The business unit itself, the development team, and a middle tier of "connecting tissue" between them. Each of the three reports up a different administrative management hierarchy. The business unit views the middle tier and the development team monolithically. The development team views the business unit and the middle tier monolithically. The people in the middle feel as if they are stuck between a rock and a hard place. So, anything that can encourage these people to behave in the spirit of the "whole team" concept will be useful. If the customer knows she can provide feedback by answering the Ultimate Question, she may actually show up for iteration retrospectives, and by virtue of being present, she might become more directly involved with the project. If the development team and the middle tier know they are viewed monolithically by the customer, they may feel they are in the same boat and collaborate more closely. And so forth across the model, from each point of view. We'll see how it goes.

The second idea I took from David is the notion of measuring the reliability of promises. This may be used as a leading indicator in the category of Team metrics. It may also be useful in helping the team embrace the idea of making commitments. There is a tendency for people to say "I'll try to get that done by Friday," which as every Star Wars fan knows, is a recipe for sitting down and giving up. The word "commitment" makes the team members nervous. They fear they will be judged or punished or shamed if they fail to deliver on a commitment. They prefer to say "I'll try" because the cost of failure appears lower, as far as they can see. David's definition of a reliable promise offers a clear framework for making and keeping commitments that seems non-threatening. He explained that a reliable promise is either kept or is reniged for good reason as soon as the promiser knows it can't be fulfilled. So, if on Monday I promise to deliver X to you by Friday, and by Wednesday I have learned more about the problem and I realize it won't be possible, it's my responsibility to inform you immediately and explain why X will not be delivered by Friday. There are two ways to keep a promise: Either deliver as promised, or renig on the promise as soon as you know it can't be done, without delay. By counting the number of times people behave this way, we can encourage desired behavior on the part of the team. This one may be perceived as a little bit squishy by the team, so I don't know how they will take to it. We'll see.