Agile was a promise. A two way commitment to communication, clarity and a shared understanding of the risks and expectations of a project. It was formed in response to the failures and pain engineering teams suffered through in XP and waterfall projects.
Agile as a paradigm is solid.
So why is it so frequently fragile in implementation?
Why it is FrAgile
The Agility of an organization is highly dependant on it’s flexibility. If there is a rigid process through which all work comes in, or in the delivery of the completed work then they cannot be agile. So often I see development as the only phase of a project that behaves in an Agile fashion. In isolation, the team goes through the motions, they have stand-ups and retrospectives, and sprints. But the actual work they do relies on a heavy intake and a tangled release process. This is not Agile. When the priorities are made and releases into production are not also agile, the whole system is fragile.
Ideally we all have a very clear picture of the end state of our project. It is clearly defined with robust acceptance criteria and considerable customer input. If this road map includes a coherent strategy for future features and improvements, more the better. With years of work laid out in a nice neat line going on into the foreseeable future. Sadly, in software, this is rarely the truth. Far more common in a 3 month window of understanding that is in a constant state of churn. Customer needs, technology and the world we live, evolves so rapidly that the possibility of a clear vision of tomorrow is difficult, 6 months an impossibility.
And here is where the first problem with Your process lies. Your prioritization and intake process relies on a paradigm developed for manufacturing and construction.
Both of those fields have the luxury of static requirements. Due to the immense cost of tooling a factory or building a structure there is a very high level of importance put into getting the design right. No opportunity to fail, it has to be right the first time. Consequently, there is time given to design, and refine that design. Iteration of the concept comes very early in the process, long before the actual development starts. There are places in software development where this long design potential exists. Operating System and Video Game development both have such a high cost of entry that they have time and necessity to design and refine that design early. Time is a concern, but long development is common. None of them attempt or profess Agile.
Whether your development team is aligned in a product or capability model, the problem lies with how much lead time is available to plan the technology. Heavy intake, is the way I describe organizations that spend months or longer defining the project at the expense of developing it. It is so frequent as to be cliche, a year spent planning and 3 months given to development, only to have requirements change mid development.
The image below was the proposed simplification of the intake, development and deployment process for a company I once worked for. Notice that there are several interesecting feedback loops. Notice also, how small the actual development portion of this process is. Understand that the business said that they were practitioners of Agile, while the only portion of that process that even went through the motions was development. Sadly, this is what I have come to believe is standard. [process image]