Topic: Agile management
"We have come to value...working software over comprehensive documentation..." "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."
Working, value-add software is the product of a software development project. (That's why it's called "software development" and not, say, a "barn-raising" or "cookie-baking.") It may be necessary to produce other things besides software in order to support the software development process, but those other things must not take priority over the primary goal of a software development project. Things like architectural diagrams, domain models, and requirements documents are not deliverables in their own right; they are interim artifacts whose purpose is to convey information to the people who are building the actual product.
One of the effects of an organizational structure that defines a multitude of narrowly-specialized job descriptions is that the individuals working in roles that produce interim artifacts tend to focus single-mindedly on the artifacts for which they are personally responsible. Since they do not contribute directly to the software under development, it is all too easy for them to lose sight of the larger objective.
Example 1
This example comes from a project at a company that has a long history of applying the Unified Process in its software development projects, and has recently been trying to become more agile and lean in its processes. There was a person on this project who had considerable seniority and many years of professional experience in software design, architecture, and modeling. He was fairly expert at RUP and especially domain modeling using UML. He managed to have a position defined for him on the project entitled Requirements Architect. The title is largely meaningless, since requirements don't have "architecture," and what the person actually did was domain modeling, and nothing to do with either requirements or architecture. Still, it's an impressive title, isn't it?
He managed to arrange things on the project such that no development work could be done unless and until a model of the relevant part of the domain had been elaborated using a certain UML modeling tool. Even routine, cookie-cutter domain entities like a "street address" had to be modeled down to a fairly low level of detail before real work could begin. Of course, he was the only person on the project who could do this work, so he was a bottleneck. Here is an example of the sort of diagrams he produced:
Why did such simple things have to be modeled to a low level of granularity and the models elaborated in a highly polished form before they could be shown to the developers? I believe it is because the interim artifact itself — the diagram — had taken on the same level of importance as the true end product — working, value-add software. This tends to happen when individuals on a team focus solely or mainly on producing an interim artifact of some kind, and they rarely, if ever, work on other aspects of the solution. They lose sight of the forest because they are so narrowly focused on a particular tree.
The impact of this sort of thing on the effectiveness of software delivery may be illustrated by the following value stream map. This is not a comprehensive map of that team's process. It focuses on the impact of the heavyweight, up front domain design on delivery effectiveness.
This is a stylized representation that doesn't take into account other factors on the project, so it is not absolutely accurate. In particular, the inventory times were variable. The last inventory, which is meant to represent the time between releases, was actually much longer than 5 days; but that isn't the fault of the up front modeling activity. We can see, though, that with this process it takes ((1 + 5 + 0.5 + 5 + 0.5 + 5) x 3) + 5 + 0.5 = 56.5 days to deliver value $$$.
What would be different if the developers collaborated in person with the Product Owner, Customer, or customer proxy to work out the street address model at the last responsible moment (LRM)? Maybe they would come up with a quick-and-dirty model on a whiteboard that would look something like this:
Feedback is immediate, so there is no backflow to refine the model. The same developer (better yet, the same pair) who work out the model on the whiteboard in collaboaration with the customer (or with someone who is authorized to make decisions on behalf of the customer) also write the code. They do it immediately, since they didn't begin to work out the details of the model until the last responsible moment (LRM), and the LRM comes when they are ready to play the user story. Thus, there is no delay (inventory) between the creation of the model and the writing of the code. With an agile approach like that, maybe the team could deliver the feature (and the value associated with it) in a bit less time. Maybe the value stream would look more like this:
So we deliver the same value in 6.2 days.
Do you see a flaw in my reasoning? I do. But that's not fair to you, because there's something I haven't told you about the project: There were distributed development teams. If whiteboarding took place in one team's workspace, the other team could not see the whiteboard.
As the "requirements architect" created diagrams representing various parts of the solution domain, he extracted them from his UML tool as PDFs and posted the PDFs on the project wiki. The wiki served an important role on the project because there were two teams at two locations working in concert. The issues that always arise when agile methods are applied with distributed teams were present on this project. Although it is a very simple bit of the design, the street address is common to both teams and they need to deal with it in the same way throughout the solution. By posting UML diagrams on the project wiki, both teams were able to see the latest and greatest model at any time, and their work did not get out of synch.
But there is still a considerable amount of delay built into the process. Would it be feasible to achieve the time savings of the simpler approach while still ensuring both teams were looking at the same version of a given model? I think it would be feasible. To create the example for this piece, I took a photo of a whiteboard with my cell phone and posted the photo on this blog in about one minute. It would be no more burdensome than that to post the same picture on a project wiki. That way, both teams would be looking at the latest and greatest model at any given time, and they would not have to incur very much process overhead to achieve that.
Entity changed? Write on the sticky note. Entity removed? Throw away the sticky note. New entity? Write another sticky note. Relationships changed? Erase and draw lines on the whiteboard. Constraints changed? Write them on the whiteboard. Then snap, zap, it's on the wiki.
But...what about the beautiful, polished model from the UML tool? Well, if the customer wants it then they have defined it as a deliverable and assigned it some amount of business value. In that case, once the model has been finalized, enter it in the tool and generate the pretty diagrams. If the customer doesn't want such an artifact as a deliverable, then there's no work to do.
Example 2
This example is based on a number of projects at a certain company in which the same general pattern was followed. Development teams were trying to apply agile methods, but the structure of the teams and the formal process imposed on development were very waterfallish. The development team was divided into sub-teams around traditional, specialized roles. The sub-teams operated in siloes and communicated with each other solely through formal, written documents and formal, scheduled meetings. The process was an iterative waterfall rather than a truly agile process.
At the time, I wanted to experience life in every role on an agile project so that I could learn exactly how agile development works and exactly what the issues are, from the perspectives of all participants and stakeholders in a project. On one of these projects, I managed to have myself assigned as a Business Analyst, and joined the sub-team of BAs. I had interacted with BAs from other perspectives — as a developer, as a project manager, as a Scrum Master — and now I had the opportunity to learn just what a BA really does from minute to minute throughout the day.
I learned that what a BA does from minute to minute throughout the day is to focus so single-mindedly on the documentation artifacts he is assigned to produce that he completely loses track of the context in which those documents are used. It is no exaggeration to say that the BAs spent 90% of their time debating which font to use for requirements documents, or how wide the margins should be, or which style of bullets and numbering they should use for lists. They spent so much time trying to perfect the documents before handing them off to the developers that the developers often had no stories to work on. I guess I needn't draw a value stream map of this. I'm sure you can visualize it. Depressing, isn't it?
When it came to analyzing business processes, the BAs didn't. They sat down with customers and users and asked them what they wanted. They made notes of the responses, and transcribed it all into the pretty documents. They did not perceive opportunities for general process improvement or for technology to streamline a process. "Hmm. I notice Betty walks over to that printer 50 times a day, and hardly anyone else uses it. She spends about 2 hours a day just walking back and forth. What if we moved the printer next to Betty's desk?" Nope. They didn't notice things like that. "Hmm. I notice that for every document you process, you open a terminal emulator window, run a CICS transaction, and then manually type the result into the document management system. What if we changed the document management system so it looked up the information for you behind the scenes and inserted it automatically?" Nope. They didn't notice things like that.
Why? Because their job was to produce documents. Their performance reviews were all about how many documents they had produced and how pretty they looked when they were printed. Their job was not to deliver customer-defined business value or to support the software developers who were delivering cusotmer-defined business value. I don't blame them for focusing on the documents instead of on the big picture. They were compelled to do so, to keep their jobs. As the saying goes, all problems are management problems.
