Topic: Mindset and culture
In these past 6 years or so of learning and applying agile thinking and methods in enterprise-scale software development projects, I've experienced and observed a certain pattern again and again, in organization after organization. In a nutshell, the pattern is this: A group of people in a company get sufficiently fed up with the status quo that they decide to try and bring agile development methods into their organization. They manage to get a small project approved, they put together a team, they get some training and/or mentoring, and they give it a shot. They don't do everything perfectly, but they have the right attitude and they don't give up.
By the end of the project, they have astonished their customer with incremental delivery and demonstrations of working, value-add software, starting with the most valuable features as defined by the customer. From the earliest iterations — sometimes even from Iteration 1 — the team demonstrates working software to their customer. Maybe it isn't a significant amount of functionality, especially in the early iterations, but it's real and it runs; it isn't just a Gantt chart and a wet sack full of promises. Customers — the "business side of the house" in a large corporation — have never received results that good from traditional IT. I myself heard a customer say, in front of his peers: "I've been working with IT departments for 32 years, and this is the first methodology I've seen that works."
That sort of thing is a wonderful start on the agile/lean road. But I mentioned that I've observed a pattern, and a pattern has more than one element. The next part of the pattern is equally interesting. In many cases, once the agile group gains the attention of senior management, they begin to receive significant political and financial support from the organization. But at this stage, senior management doesn't view the agile effort as a proof of concept or an experiment. They aren't interested in continuous improvement, because continuous improvement is not part of their professional mindset. They're interested in institutionalizing whatever the team came up with for Project Number One and replicating it hundreds of times for the rest of the IT portfolio. Did I mention the team didn't do everything perfectly when they delivered Project Number One? I think I did. Now they're going to replicate the imperfections hundreds of times over, and institutionalize the whole thing.
So, here's a dramatized conversation synthesized from many real ones.
I was talking with some folks who had introduced agile at their company about a year and a half earlier. They experienced great success with their first efforts, and they went through the pattern I just described. They were very pleased with what they had accomplished, and they were eager to describe it to me. They said many things you would expect any agile group to say. Among the things they said was one that sharply caught my attention: We haven't missed an iteration commitment in a year.
Interesting. So, you've delivered everything you set out to deliver to every customer in every iteration on every project with every agile team for the past year.
Yep. That's it, they said, beaming with pride.
Have you had any close calls? I wondered. Occasional necessity for temporary overtime? Features deferred to a future release due to unforeseen development issues? A simpler version of a feature delivered on time, with the full version delayed until later? Escaped defects that caused a delay in deployment? Anything like that?
Nope. Everything has run like clockwork for the past year. Everything.
Wow.
They beamed with pride some more.
So, I asked, what are you doing about the problem?
The beaming went on for several seconds, until one by one they realized what I had asked. Their faces assumed expressions of surprise, bemusement, and even a bit of offense.
What problem? Everything is working great! We've out-performed the traditional development teams at our company at every turn!
I don't doubt it, I assured them. Then again, all one has to do to out-perform a traditional development team is Not Suck. It doesn't even require agile development methods. Obviously, to Not Suck is a great step forward. But surely it isn't your ultimate professional goal!
You're a real smart-ass, aren't you? they queried.
No, but I play one on the Internet, I replied. So, what you've told me so far is that you don't suck, and that you're not putting into practice one of the most fundamental values of agile and lean thinking.
What do you mean? We're doing all the XP practices, and we're using Scrum. We have Certified ScrumMasters, and everything! Which value are you talking about?
Continuous improvement. You can out-perform the traditional development teams in your company, which is sort of like saying you can beat your cat at chess two out of every three games, but you haven't pushed the envelope in a year or more. You're staying in your comfort zone. You don't know what you're capable of.
Okay, wise guy: What would you suggest we change, if everything is working well for us?
You mean, if it ain't broke, what do you fix? I scratched my chin for a moment as if thinking about the question, although in reality my chin was itchy. What's your iteration length? I asked.
Three weeks, they replied.
In that case, go to two-week iterations.
Are you kidding? Then we wouldn't have time to do all the things we do in an iteration!
Why wouldn't you?
Because we've got everything down to a science. We know exactly how to do each task, and exactly how long it takes.
In other words, you've settled into a comfortable routine and you're not re-examining your process because you aren't feeling any pain.
Are you sure you're not a real smart-ass?
Are you sure you really need to do all the things you routinely do during an iteration? And if you do need to do a thing, are you sure there's no waste in the process that you could squeeze out?
Well, we don't think there's waste in our process.
But you don't know, because you haven't discovered your limits. If you push yourselves to the limit, any waste in the process will show up in the form of "pain." Then you'll be forced to re-examine what you're doing. That's when improvement happens.
Look, man, you haven't seen our teams at work. You haven't seen our organizational context. We're just having a conversation, here. Who the hell are you to say we should push the envelope to discover the waste in our process?
I guess I don't know much, I admitted with a shrug. But I know that as agile teams go, three weeks is a pretty long iteration. I know that any group that hasn't changed the way they work, or even introspected about the way they work in over twelve months is quite likely in a rut. I know that people generally don't bother looking for opportunities for improvement unless they're feeling some pain. And I know that the simplest way to avoid pain is to stay well within one's comfort zone.
How do you know three weeks isn't right for us, in our situation?
I don't know that. More importantly, you don't know it. If you try two week iterations for a while and you can't make it work, then you'll know three weeks is the right length. But I'll bet that's not what'll happen. I'll bet you'll discover a few things you could streamline.
Like what?
Well, for starters, what roles do you have on your teams?
We have a PM, a ScrumMaster, BAs, QA Testers, Developers, a DBA, and an Architect.
And what's your process look like?
The Architect lays out the solution architecture and the domain model. The DBA defines the database schema. We do that during a "design iteration." BAs write stories and hand them off to Developers. Developers write unit tests and code, and hand them off to QA Testers. QA Testers write test scripts for larger scale testing. Then they feed defects back through the process for correction.
Would it save time if the solution architecture emerged naturally from the evolving design, throughout the course of the project?
It would be quicker, but how would we know the solution fits into the enterprise technical infrastructure?
That's enterprise architecture, not application architecture. It's covered in your non-functional specs and company standards. You said you had an Architect role on each team. I assume that's for application architecture work?
Well, yeah, I guess that's right.
And application architecture emerges alongside technical design, as features are added and the codebase is incrementally refactored?
Well...yeah, that sounds right.
And the database schema can't really be defined fully up front, either, can it? Doesn't that evolve, too?
Yeah, that always happens, now that you mention it. In fact, it changes a lot.
So, you can eliminate the "design iteration," the Architect, and the DBA.
Nervous laughter all round. I don't think they would stand for that!
They don't have to stand for it. They're not the customer. They're not paying for the project. They can pick up a keyboard and play the Developer role. Their architectural and database skills will still be useful on the project, but they're going to have to do some real work, too.
You really are a smart-ass, you know.
I know. Now, what if the same person who writes down the stories and requirements just writes them in the form of an executable acceptance test instead of as a document? That eliminates interim artifacts, work-in-process inventories, and hand-offs, and reduces the chance of mis-communication between people in different specialized disciplines. It also reduces role specialization, which in turn raises the bus number.
Bus number?
The number of team members who can be run over by a bus and killed without taking the whole project to the grave with them.
Charming imagery.
Yeah. It also means you have more flexibility to deal with different types of work that come up at different stages of the development life-cycle. If you recall the standard RUP process architecture diagram, it shows how the relative proportion of work in different disciplines varies according to where a project is in the lifecycle. Generalists can pick up whatever type of task needs to be done at the moment. Specialists can't, so there are times when the high-end specialists aren't working, but you're still paying their high-end salaries. If one team member has multiple skill-sets, you're in better shape both from a schedule standpoint and a financial standpoint.I know our management would appreciate that argument. But how realistic is it? Can the same person do both those tasks? Even if they can do both tasks, would they want to?
Only if they want a job in the agile future. Narrowly-specialized roles belong in the past. Even that is only a step along the way. It's possible to go further with role consolidation.
What do you mean?
I mean, there's nothing a DBA, a BA or a Tester does that a Developer couldn't learn to do just as well.
Ha! Well, that's where your theory breaks down. Most of our Devs wouldn't stoop to talking to customers directly to elicit requirements! Most of them don't consider DBA work or writing test scripts to be "real" development. They just want to sling code.
Agile projects aren't about slinging code. They're about delivering business value to the customer. That's primarily a matter of slinging code, but it's not solely about slinging code. An agilist does what the project needs done. The code-slinger who doesn't care about delivering business value can hop on the time machine along with the BA who doesn't want to learn how to write executable acceptance tests. They'll work well together back in 1985 where they belong.
But if the Developers are also doing the testing, then they'll get bogged down with defects and they won't be able to build value-add features.
It's interesting that there's such a significant backflow for defect correction. You say you're using all the XP practices. Some of those practices are designed — and field-proven — to minimize or even eliminate defects before they get past the developers. Why do you have enough "escaped" defects to keep a team of QA Testers busy full-time?
Well...we use the XP practices when we need to.
Ah, yes, of course, I said, smiling and nodding. I wish I had a dollar for every time I've heard that one.
So, what is it you do for a living, anyway? Go around making people feel crappy about their work?
That's not the basic job. It's just a perk. I usually do agile team coaching and mentoring, and sometimes serve as ScrumMaster.
Shortening our iteration length will create obstacles for the team. Isn't the job of a ScrumMaster to remove obstacles from the team's path?
That's the tactical role, yes.
Tactical?
Right. On a strategic level, organizations need their teams to keep getting better. Sometimes, that means throwing obstacles into the team's path, deliberately. Shake things up. Turn up the heat.
What for?
To make them stronger. Like push-ups.
Doesn't that mess up a project?
Short term, small scope, yes. Long term, large scope, no. It improves the organization. If you're not improving, you're deteriorating. Stability doesn't exist. Stability and repeatability are myths of a bygone era. Inspection and adaptation on short cycles — that's the key.
So, you're saying we're supposed to be feeling pain all the time, so we'll be forced to improve. Frankly, that doesn't sound very practical. Doesn't sound very fun, either.
If you're in pain all the time, all you can think about is the pain. You're paralyzed. In that case, you're right; that wouldn't be very practical or very fun. But if you're just on the edge of pain, and you bump into the pain from time to time, then the pain won't paralyze you. It will just indicate where you need to look for opportunities to improve. It will keep you awake and alert...tell you when you're approaching your limits, whatever they may be. Use your own pain as a tool to help you improve.
There's no single "right" way to be agile. There's no cookbook recipe. There are values and principles that can help you stay focused on the right things. There are proven practices that help you achieve specific goals. Use them until someone comes up with something better. But if you feel yourself becoming too comfortable, it's a "smell." Don't ignore it. If you do...
