Having been through the Agile mill a few times, trying what works and what doesn’t; some related unchallenged assumptions often appear. “How big is it?”, “how long will it take?”, “Have you estimated it?”
The assumption is that estimates are required, and by having better estimates then the work will go more smoothly and we will be more ‘Agile’. Why?
Most Agile requirements for estimates stem from planning. In a traditional project management perspective on the world, you need to know how long something is going to take, what the dependencies are, what resources are required etc in order to know when you’ll likely to be finished. Then if progress isn’t meeting the plan then the project manager can start doing something about it.
However software development isn’t like building a house. It is always fraught with unknowable factors and the only way to know how long something will take is to do it and look back on how long it took. Psychologically speaking, we are generally predisposed to be optimistic about the future and how achievable certain things are. Estimating will always miss out hidden nasties that won’t be uncovered until you start work for real.
I was fixing our shower the other weekend and had a rough idea how long it would take. I had bought a new one of the correct type and had previously looked under the cover of the old one. I managed to remove the old one in the kind of time I expected, but found that I needed to make screw holes in the wall in different places and that I needed some extra pipe connectors to attach the water supply. I didn’t have the parts or the time so I used some PTFE tape to fix the leak in the old one (i.e. a temporary patch) and proceeded to put it back. Again a new problem arose – the nut on the water pipe had receded into the wall and it took an hour to get it back again.
So even with a simple real world job there are often unanticipated obstacles and extra tasks to complete which push back the original scope. In this case I finished on time but with a different solution! Hofstadter’s Law states that “It always takes longer than you expect, even taking into consideration Hofstadter’s Law.”
Another purpose for estimating is to attempt to understand some of these problems. Often teams use estimation or planning poker sessions to discuss user stories. I feel these are proxies for the actual job of understanding the requirements. Debating the differences in relative point scores of user stories has value in provoking a technical discussion; but story points in themselves are valueless and arbitrary. Just have the discussion instead of trading points or days.
Commercial types are often wedded to estimates as they understand the relationship between ‘man-days’ spent on a task and a chargeable daily rate. Hence the request to development teams to provide an estimate of how long it would take to write a particular feature so they can quote a figure to the customer. Unless your software organisation is a body-shop or churns out highly repeatable instances of code, these features are likely to be unique and really require stringent analysis. If you are supporting an ongoing product, then adding features adds to the complexity of the product set for the future, so any new work has to take into account these custom features. Therefore the ‘cost’ of the feature request is not in the time taken to develop and implement it but in the consequences for the product in future. This is rarely taken into consideration when quoting a customer for work, and the estimate gives false reassurance to commercial folk that the costs will be covered.
I find that a rough “T-Shirt” sized approach to pricing is more effective, although this takes some explaining (particularly with customers used to daily rate quotes). The result is quicker quotes and a more realistic price taking into consideration future support. It also stops the “knock a couple of days off that quote/do we need to pay for testing?” type negotiations which diminish the return on the work.
Having estimates on user stories is also useful for measuring performance and progress. Knowing how many points remains in this sprint/release, what our velocity is etc provides development managers with metrics they are comfortable with. Again this is an illusion, since as shown above the estimates will in all likelihood be wrong and reality was much different. If your estimating process is very detailed and they end up being correct; at what cost? How much time have your testers, developers, analysts and whoever else spent producing estimates rather than in working software – where the real value is?
What is the alternative then? Recently I had the opportunity to revitalise a software product and build a backlog, roadmap, features etc from scratch for it go flourish in future. To do this I kept overhead and process to a minimum and focused on what was important to deliver working software that customers will use and benefit from.
I started with a general ‘to-do’ list of bugs, features and improvements that had been languishing for some time. A quick triage of these things led to writing more detailed user stories and importantly prioritising the stories into some kind of order – based on urgency, value, speed to implement etc.
The act of writing the stories and thinking about them in some depth is where the ‘how complex’ questions should be answered. Think about the perspective of the user, but also from that of developers and testers who need to turn this need into real software. A story becoming more complex needs turning into more user stories.
The need for measurement should be on more objective criteria than points – so I use number of stories. Through the analysis work I try to ensure that one story is of a similar size to another. This will vary but on the average size shouldn’t have too much variation.
For planning, I just ensure that the most important user stories are at the top of the queue. When the team have finished one story, they pull the next one off the top of the queue. If it turns out this story has some problems (maybe too ill defined) it goes back into the backlog to be analysed further and the team pick the next in line to work on.
At the end of the time marked for production work (typically two weeks) all Done stories make the release and can then go into the pre-production test and release process. Planning becomes redundant as it’s not a matter of working out what can fit in a release but to work on the most important things in the time available, however long that may be.
In software development we are often asked for things other than working software. The value of these things should be challenged. Find better ways of doing what you do, strip away as much of the unnecessary overhead as you can and you will find those doing the work will enjoy it more and be more productive.