Saturday, May 15, 2010

Kanban, TDD, and Iterative Development

Where I work we make a lot of use of Kanban type systems. It's a key tool in our process and has helped us manage bottlenecks and make activity more visible.

We also favour Test Driven Development which encourages a very fine scale of iteration, and we get benefits from that.

The red green refactor cycle of TDD is pretty good at tuning interfaces and driving looser coupling between modules. Our Kanban process is pretty good at finding bottlenecks in our delivery process and helping us reallocate resources or use our time differently.

Recently we've had some serious production bugs, relatively new code (six to nine months old) being extended with new functionality is causing us grief. Whatever we did in our initial implementation we were missing the secret sauce, we implemented a design that is now hard to fathom, and work in the area is blowing out well beyond initial estimates.

Our designs don't make ongoing use of the code easy.

We were treating Kanban + TDD as our iterative process and we hoped we would get design improvements from our iterative processes, but we weren't really. At the scale of design and iteration of design, neither Kanban nor TDD have given us what we want.

The design bit in TDD is a bit of a lie, it's the hope that if you pay attention to the little stuff then the big stuff will look after itself. Evolution can be a cruel and terribly inefficient process, and there's no guarantee about the future fitness of it's outcome. A program emerges from TDD and satisfies the tests but nothing more is guaranteed.

Moreover, it seems to me that Kanban works against constructive iteration. Of course a larger feature may be broken down as a series of tasks on a Kanban board, and that will be an iteration of sorts, but the motive for task breakdown in Kanban is managing and coordinating resources; creating opportunity to revise earlier decisions is not really Kanban.

Fundamentally, Kanban is monolithic within it's model of a single task. If any iteration is visible within the task cycle you see tokens being pushed back, and anyway you look at that it's negative: it's regression, it's a bug, it's something being rejected. If I take iteration to mean go back and repeat, then from a Kanban perspective iteration within the cycle is a sign of problems.

I feel we're missing something between TDD and Kanban, or parallel to them, that promotes giving attention to higher level code quality issues, quality design and integration, and implementations that align with domain models. Particularly we're missing anything that promotes iteration over higher level design decisions. I'm reluctant to just propose adding more baroque details to TDD or Kanban because any success had with those version is likely to be confused with the basic practice, and when people are weary or don't understand the elaborations they'll slip back to the default practice. I look for methods that support me in my weakest moments, rather than methods that require me to be on top of my game in every moment.

Around the basic development activity (which for us is TDD) I'd like to explicitly promote fine grained analysis and design, review of progress with all sorts of relevant parties, and refactoring to explicitly improve design. Something that encourages multiple iterations for any given task, reinforcing that for any given task a single design step is probably not enough, something that highlights the positive cycle of reconsidering design. Something that revisits the integration of units and their interactions and not focusing just on requirements satisfying interfaces. Something that rewards the developer for going back and having another look.

These concerns are sometimes at a higher level than a naive TDD. I'm concerned with how the collection of objects and services fits together, the coherence of the system as a whole, thinking about future ease of work, clean mapping between implementation and our understanding of the business domain. But at times it's low level but invisible to TDD, it's behind the interface. With TDD alone a thing can be tested and correct and yet have an utterly impenetrable implementation from the perspective of future development.

What might the solution look like? I don't know just yet, perhaps a check list with multiple columns, each column is a fine design iteration, start the next column if you have to put a cross in the box you've already ticked, and then treat getting into more columns as a good thing. I'll be trying out something like that and see where it takes me.

P.S. I know I've presented a very naive view of TDD, but the name itself suggest a simple interpretation. The name TDD gives no clue that you should look for more subtle and nuanced practices even though that's what Beck described in his books.

No comments:

Post a Comment