Releasing to production: every iteration?
At Huddle, we run two week agile development iterations, and each iteration is a potential production release candidate. That means that the code developed is of a quality that could be released to production, but it may or may not make sense to do so from a product perspective. For example, when we developed the new meetings functionality within Huddle, the development of the initial release spanned two or three iterations, and to release after the first iteration would have meant releasing an unusable product: you could create a meeting but not delete it. Production ready refers to the quality of the code (meeting creation was implemented, tested, code reviewed, refactored etc.), not the completeness of the product.
So the two week iteration cycle means that we have the option to release every iteration, but we do not always do so. As well as product inconsistency, the other reason not to release every iteration is the overhead of a release. Almost always, live application problems are introduced by change. It does happen that issues can magically manifest themselves without a software release - e.g. load hitting a certain threshold, running in to resource limits like disk space, or encountering time-based conditions that trigger a bug, like not handling daylight saving time correctly or crossing year boundaries - but almost always problems are caused by introducing a new version of the software.
This is what makes regression testing and post-deployment testing really important - in order to reduce the risk that you put something live that breaks stuff. At the end of an iteration we know that all new stories are fully tested, but regression testing is there to ensure that we didn’t break something else while we were about it. So for a production release we run a one week regression test before releasing, that runs in parallel with the development of the next iteration. As our regression testing is currently a manual process (everyone in the office helps out) this is quite an overhead, and not something we want to go through unless there is a business reason to get the last iteration of stories in to production. We are actively looking at automating our regression testing (more of this in a future post) but we will never completely eliminate the need for some manual testing (to check UI layout for example) both before release and immediately after it.
What this means is that when we have a larger project on (for example launching a major new feature) we would probably release after about 3 iterations (6 weeks), whereas when we are working on a number of smaller changes (especially when they need to coincide with dates for launching new partnerships) we will release every iteration. The average is probably every 2 iterations.