Agile development – the best choice?

I seem to remember our Professor, Pete Sawyer, telling us that they were going to puncture some of the hype around agile development methods. I got the impression that this was one of those innovations that Silicon Valley types like to rave about as if it’s the solution to every problem. There is even a “manifesto” filled with grandiose principles and ideals, with an image that looks like a designer revealing some kind of gift from God, which quite honestly seems like something out of a satire.

 

It was admittedly hard to argue with a lot of the arguments in the lecture (and especially the accompanying reading) but honestly this all felt a little bit off to me. It may have been down to the overzealousness of its proponents, which made me feel like there must be some downsides that were being overlooked. But also, it seemed to simply go against common sense for me. Surely there are some instances where agile development is a mistake? So I decided to research the method a bit more deeply to see what the issues might be.

Objections to agile

Part of the way agile development works is that requirements are allowed to change dramatically over the course of the project.

Lajos Mozcar writes:

In theory, developers code while collaborating with stakeholders to define, refine and change requirements as the projects goes along. The methodology, however, does not distinguish between big and small changes.

Every change has a cost, but agile does not account for this. The result? People often change really big things late in the game using the rationale that since it’s an agile project, it can handle it.

(Mozcar, 2013)

This can potentially cause problems on two fronts. In the first place, if poorly managed, multiple project changes can add up and lead to “function creep”, where developers continue to add new features until the project barely resembles what was originally desired. Secondly, major changes late in the game can lead to a lot of development time being wasted on tangents that end up being discarded.

Other objections to agile development concern the way it puts unnecessary pressure on developers. On the one hand, agile is meant to allow developers some measure of freedom by accepting changes late in the development process, but on the other hand the methodology requires consistent and repeated progress reports, which can lead to a feeling that the developers are constantly being watched. The manifesto writes that “at regular intervals”, the team meets and “reflects on how to become more effective”. Michael O. Church points out that “When you force people to play a side game of appearing productive, in addition to their job duties, they become less productive” (Church, 2015).

Michael O. Church also suggests that one weakness of agile is that it leads developers to focus on certain kinds of tasks over others. If, for example, your progress is measured every few weeks, career-minded employees are likely to “causes programmers to favor the low-yield, easy stuff that the business doesn’t actually need (even if bumbling middle managers think otherwise) but that is “safe”” (Church, 2015).

Does it depend on the project?

My gut feeling is that agile development would work better in smaller, better-integrated teams working on (relatively) projects. In this way, changes to the project could happen on an organic and democratic basis with input from all members. The larger the team, the greater chance that someone will lose out when the project changes direction. However, this could be alleviated by intelligently dividing large teams into smaller ones and giving them some measure of independence.

I am also curious about the distinction between IT companies and non-IT companies and organisations which have an IT component (for example, a large retailer that wants to create a website). A company like Google is primarily based on software, so agile development makes sense – the goal is to create the most effective software in the shortest amount of time, even if that means changing the nature of the software. On the other hand a large online retailer is using software to achieve very specific goals which are basically unchangeable – sell more products, provide better information to customers. Although there are opportunities for agile development to be used “under the hood” in areas such as developing the algorithms that make recommendations to customers, the overall requirements of the website will not change; if they do change, it will be determined by stakeholders and not the developers. So it is possible that agile works best when software is the core business, rather than something which supports the business.

My experience and conclusion

I recall a project from one of my previous jobs in which our organisation (the National Bee Unit) was creating a mobile app to record diseases in bee colonies. This app would be used by around 50 bee inspectors, working independently all over the UK, many of whom were reluctant to use this kind of technology in the first place. One can only imagine the level of resistance we would have suffered if we had been changing the app at a moment’s notice.

Ultimately, it is hard to argue with the results of agile development. The reading from this week (Iterative and incremental development: A brief History) lists some frankly remarkable results where agile has been successfully, such as the X-15 jet which was not only extremely complex but also involved many disciplines outside of software. So I cannot reasonably argue that agile only works for small projects and not large ones, or that it should only applied in the software sector. I can only raise some potential issues with the methodology and advise caution to anyone who tries to apply it to every project. The bottom line is that every project is different, and there is no one-size-fits-all solution.

References

Church, M. O. (2015, June 6). Why “Agile” and especially Scrum are terrible. Retrieved from Michael O. Church: https://michaelochurch.wordpress.com/2015/06/06/why-agile-and-especially-scrum-are-terrible %5BAccessed 27 January 2017]

Larman, C. and Basili, V. (2003, June). Iterative and Incremental Development: A Brief History. IEEE Computer Society. Retrieved from: https://pdfs.semanticscholar.org/058f/712a7dd173dd0eb6ece7388bd9cdd6f77d67.pdf [Accessed 27 January 2017]

Mozcar, L. . (2013, June 4). Why Agile Isn’t Working: Bringing Common Sense to Agile Principles. Retrieved from CIO: http://www.cio.com/article/2385322/agile-development/why-agile-isn-t-working–bringing-common-sense-to-agile-principles.html %5BAccessed 27 January 2017]