The Agile Way I
This is something I wrote back in 2019. The agile software development was a "favourite" topic of mine back then. I've put a lot of thought in it, because it was a major cause for my professional frustrations. I've been thinking lately that despite the seemingly major technical disruptors - the LLMs - not much has changed in the way we organize software projects. So, this experience might be useful in avoiding the pitfalls of the very loosely defined agile process.
There are trends and fashions. Trends and fashions are statistical categories. When you say to me that something is trending, I understand that a lot of people are using it/doing it. The existence of a trend doesn’t say anything about the meaningfulness of its subject to me. Some of the people following the trend have a solid understanding of the subject. Some are just cheerleading. Some are genuinely excited. Some are comfortable with the status quo and are just tagging along, generally not opposing the fashion, thus passively enabling its further growth. Some are abusing the subject of the trend, either unwillingly or knowingly, for their own purposes.
Eventually the latter are giving the subject a very bad name. I would like to write a word or two about the latter. My idea is to show with examples from my own experience what the abusing or misusing of the agile principles can lead to in a company, that follows the agile way without using a strict set of rules for its projects. Or in the immportal (I would leave the typo as it is for apparent reasons) words of Captain Hector Barbossa, these are „more what you’d call guidelines than actual rules“.
For the last couple of weeks I’ve been looking for a new job and I’ve hardly seen an ad that doesn’t mention agile. “We are an agile company“, “we live the agile process“, “we are looking for agile developers“, “you will be working in an agile team“, “as part of our dynamic agile developer’s team“, are some of the most common clichés used in the descriptions. It seems that in the HR world, the term agile stands for freedom or some coolness. But freedom from what? From the oppression of the non-developers? From the deadlines and the time constraints? What makes the term agile so cool?
My first encounter with the agile development methods was at the university in a lecture about software engineering. I can still remember what the professor said back then: the agile developing methods are best suited for innovative projects with a lot of unknown factors; basically projects where one is not able to make a reliable time estimation for a requirement or a set of requirements. As a side note to his main statement, he mentioned, with a bit of a smirk, that one would better apply the agile principles every time one notices that the end user/client can’t say exactly what they need. Which kind of makes sense. And that is exactly what one of the principles of agile development states:
"Business people and developers must work
together daily throughout the project."
Agile manifesto
The idea is basically that you try to incorporate the people with the requirements in hand in your developing process, thus avoiding the danger of implementing something they didn’t ask for. We all remember the old comic strip - what the developer sees, what the user wants, what the marketing people sell, etc. - you know how it goes. I guess this principle was a reaction to the traditional way of developing applications, where business people gather and write a specification document that's a few hundred pages long and pass it to the development along with a deadline, which is, as the saying goes, defined as “yesterday” because it’s almost never possible to meet. It seems that the principle solves a couple of problems very neatly by simply bringing all people having stakes in the project together and letting them exchange in a friendly manner. On one side, you restore balance by allowing the developers to get out from the basement and say to the business people what they really think about their unreal deadlines and requirements specifications. On the other hand, you allow the business people to tell the developers that their idea of a product can’t be sold to anybody who isn’t a developer/nerd, a hardcore fan of Star Wars / Star Trek, or a person who isn’t happy about a product that only works on some developer’s machine.
As the CEO of the company, you should be extremely happy to have straightened these sorts of things out as early in the project as possible. But here comes the tricky part: as the CEO of the company or any executive who is responsible for the project, you should know what kind of people you have in your product team. And what’s more important – what kind of roles are they having and what responsibilities each one of them is having. In a waterfall world, the results and deadlines are extremely well-defined in terms of time and scope. The product team works together with the end user for two months and delivers a requirement specification document. Then, the developer team works for one year and delivers a running application. After them come the testers. In an agile world, you don’t have a principle about time, scopes and roles. The only principle that mentions time-to-market is:
"Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software."
Agile manifesto
So basically you state that you want to be as fast as possible. But how? Who takes care about time in an agile team? If it lies only in the hands of the developers, they would enjoy their freedom for as long as they are able to. No deadlines. Finally, they are free from the oppression of the business people. It sounds right. After all, one of the values of the agile methods is:
"Individuals and interactions over processes and tools"
Agile manifesto
Real-life example – as a CEO, you have an idea for a product. You don’t know how good. But you want to see it live. Your company is modern and “lives” the agile process. You have a small problem – you don’t have a product team. You don’t have people who refine and validate your idea. Basically, you are the only business people. But you don’t have time for lengthy requirements specification sessions. And also, you are costly. But you are also agile. So what you do is gather your developers and give your best at explaining what your idea is and either let them roam freely or you give them a deadline. You don’t have project management, because the agile principles don’t mention time management. Also:
"The best architectures, requirements, and designs
emerge from self-organizing teams."
Agile manifesto
A fact of life – no developer likes time estimations, because who wants to put pressure on himself? Since your self-organizing team consists only of developers, they all agree that they won’t do time estimations. They will instead focus on the quality of the product as they understand it. Let’s assume that they are all clean code hardcore fans, and they will clean that code until they drop. According to most definitions, yours are some fine developers. But after a month (or whenever you have the time to take a look at the progress) of cleaning that code, you realize that that’s not what you’ve had in mind, despite following the agile principles.
What happened? The CEO has left the whole process in the hands of the developers, and they’ve delivered (at best) a running prototype of his idea, which he is not satisfied with. He has burned a month of developer’s time and didn’t let product people refine his idea and check if it’s useful/feasible. He didn’t appoint anyone to control time in his team and let them decide only on code quality. The developers are grumpy and unsatisfied as well, because the thing they’ve implemented will never go live, and most probably they will have to start from scratch. According to most of the principles, the company’s process was agile. Freedom is not so good after all.
In a wonderful post about the role of executives in a company, Avery Pennarun wrote the following:
"To paraphrase the book, the job of an executive is: to define and enforce culture and values for their whole organization, and to ratify good decisions."
That’s all.
Not to decide. Not to break ties. Not to set strategy. Not to be the expert on every, or any topic. Just to sit in the room while the right people make good decisions in alignment with their values. And if they do, to endorse it. And if they don’t, to send them back to try again. Avery Pennarun
-- the original quote comes from the book High Output Management by Andy Grove
Further down in his post he explains why that should be this way:
"If the executive makes their own decisions and forces them downstream: the executive doesn’t have enough information to make good decisions in detail, so the decision won’t be optimal."
Avery Pennarun
In my real-life example, the CEO was the sole businessperson in the agile team. To be fair, the scenario would’ve probably worked out OK in the end if this was his only project, his only team, and he didn’t have anything else to do but look after this one project and this one team. The only thing he should have done is to slip into multiple roles – project manager/scrum master, product developer and perhaps also a lead developer. Such a scenario is probably very typical for the early stages of a startup company. This would also explain, to a certain extent, the popularity of the agile process among start-up companies. And it does make sense. If I were an executive of a start-up, the last thing I would’ve wanted was to spend maybe 6 months refining and specifying my product on paper before I start with the actual development. It appears that the agile process becomes a trap for companies after they pass the initial stage and start to grow. This is when the executives should let go.
continues in part two ...