Wednesday, February 13, 2013

Agile in the waterfall



"Agile in the waterfall" is a hot topic! The question that comes up in every agile class I teach is: 'how do I do agile in the waterfall?'

I'm tempted to say: Read my book! It's about agile in the enterprise. (Did I write to the wrong theme? Well, I may have missed the meme of the moment. But no, I think enterprise is a bigger idea; and it encompasses what my students are really asking about)

Agile refrigerators?
Think about this: the Samsung T9000 refrigerator, due to be in stores this year, runs the Android operating system, comes with apps preinstalled; integrates with Evernote to recall receipes, videos, etc; has a Wi-Fi interface; and of course has a tablet-like flat panel display.

That's a lot of user interface via software in a product that is traditionally a hardware product. One can only wonder about the methodology for all the code that the T9000 runs.

Back to the water thing.
Now, 'waterfall' is not my word, but a lot of people use it. Winston Royce wrote the seminal paper about it 40 years ago. "Traditional" or "plan-centric" are my words.

What is traditional?
Since words are important, I prefer 'traditional', by which I mean:
  • Can be modeled as a finish-to-start sequential architecture with tools like Monte Carlo simulation
  • Is largely planned before it's executed; and then it's largely tested after it's developed. See Royce (above) for many important variants on this simple sequence
    • Reasonably complete requirements to start
    • Change will be manageable
  • Presumes -- on the part of the sponsor -- that there is a commitment to the plan (by the project team):
    • Deliver what's planned for the resource commitment (time and money),
    • Accept some allowance for re-direction at gates; level-of-effort and cost reimbursable variants; and allowances for various incentives for value engineering and mission imperatives.
What is agile?
It's the last point about sponsor presumptions that is the sticky wicket re moving to agile in any part of the project, though there are other issues with agile, by which I mean:

  • Is not susceptible to finish-to-start sequential modeling, except at a very high level (release plans)
  • Is tactically planned on the fly but adheres to a largely centrally planned vision-set-to-narrative and top level architecture
  • Presumes a best-value commitment -- as directed largely by the customer -- to the vision-set-to-narrative and top level architecture
  • Schedule trumps scope
Other people's money (OPM)?
OPM comes up in every conversation.
  • Traditional: Do what's in the plan, but don't take more resources to do it. In the popular vernacular: don't be a taker!
  • Agile: Apply the resources given and deliver the best value possible; then STOP!
    • Best value: the most valuable stuff -- importance, urgency, usefulness -- as determined and directed by the customer/user (not the sponsor) and affordable within the resource constraints.
  • All: write a business plan (before), and write a roll-out and integration plan(s).
    • Business plan for what? (I'm tempted to say 'read my book' but I'll say instead) that the business plan supports the economic and business utility of the vision-narrative
    • Integrate to what? The business, the enterprise, the market environment. Nobody builds stand alone stuff that lives in a green field.
Agile in a traditional wrapper
Now the question is not about one or the other but about the peaceful co-existence in the same project envelope. Invariably, agile is wrapped in a traditional cover; not the other way around.

Why? It's the nature of intangibles v tangibles. Traditional methods are optimum for tangibles, and it usually takes tangibles to usefully connect to anything useful (reuse of word is intentional). Thus, intangibles are folded in amongst their tangible brethren.

There are some obvious conflicts/tensions between a traditional wrapper and agile:
  • Traditional may be fixed scope (sometimes not); agile is never fixed scope
  • Traditional projects may be large scale (usually are); agile is optimum for small teams
  • Traditional projects may be globally distributed (often); agile is optimum for collated teams
The way this will work is for both methodologies to give a little at the office. Neither can be as purely defined.

Some stitching required
So, we've now arrived at the hard part: stitching a somewhat non-sequential tactically planned project unit to a traditional carrier (for want of a better word) that is sequential and strategically planned.

To handle the scope tension he word "de-coupled" jumps to mind -- or loose coupling. This is a concept from system engineering used to isolate independent units such that -- within limits -- there is flexibility of maneuver. The practical way to get at this is through very structured interfaces that themselves can not be very tactical -- they should be viewed as strategic elements (like the pass through the mountains ... this way an no other way). Agile must respect these interfaces... there can not be lisence to disrespect their specifications.

The diagram provides some imagery about this concept. We still need a charter of authority and we still need architecture -- those things are not repealed just because of a methodology choice.

To handle the scale issues, we recognize there are ways to scale some practices and others are best left as small bore. For instance, stories can be scaled to use cases and committed to a database. But small persistent teams need to be left as small persistent teams.

To handle the global distribution issue, there are three things you can do:
  • Don't do it (may be impractical)
  • Reorganize the boundaries of the backlog so that work is segmented and localized by object set, even if teams are remote; refrain from remote participation below the team level
  • Decouple major elements by interface so that integration is simplified and remote workers are isolated by defined interfaces

Steady state or transient?
And, last we come to transition, typically from traditional to agile or agile in the waterfall. Mike Cohn has the receipe on this one: begin with a pilot project. Learn how to do it, and then move to a steady state.

What can go wrong: Here's Mike's top 20 tips on how to fail at agile and avoid success.

Did I say "read my book"?