Why Agile doesn’t work


OK, so the heading was kinda link-bait. Ish. A bit. Sorry.

Agile practices absolutely do bring benefits, and really are the only practical way of getting things done properly. This article is a bit longer than my usual rant, which surprised me. It started off as a quick bullet-point list, and quickly grew into a monster. I make no apologies. This all needs saying. It is an exploration into some of the reasons why the transformation process sometimes fails.

But before we dive into the negatives, let’s review the key values of The Agile Manifesto:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

And the Twelve Principles are worth a re-read, too, if you’ve not had a look recently.

While most of my experience is Scrum-flavoured, and that’s the terminology I’ll use, the basic principles should apply across the board.

So why do things often go so badly wrong? Let’s have a look at some of the usual suspects.

Going overboard

One of the common down-falls is placing all of the focus on the left-hand statements in the Manifesto, and completely ignoring those on the right. This is a mistake!

Perhaps you’ve just had the development team trained and Scrum certified? Beware the over-enthusiasm this can generate, as they gleefully throw all processes, documentation and plans out of the window. Truly agile development is in fact a very disciplined endeavor. But don’t tell your engineers that!

Mitigation: Sprint -1

Consider having a short Sprint called “Implementing Scrum”, where your Scrum Master/Coach will lay out some Epic items to start constructing your prize-winning framework.

Maybe “Select Scrum Management Tool” could be the first one? Don’t dictate things like this. The tools must be selected based on the needs of the team, not the other way around. If they are truly happiest and most effective with sticky-notes on a wall, so be it. Find a way to make that work for them. This is one of the best opportunities to help the team understand that individuals and interactions are most important. An aide to that importance is in selecting the correct tools & processes which add value to daily activities – for example, without a Scrum/Kanban (or whatever) board, it’s impossible to quickly determine how far we’ve progressed, and what’s still outstanding. That information will help us focus the next bit of effort.

Another good Epic for the very first Sprint ever might be: “Determine ideal Sprint length”. My experience is that you’ll have to experiment for a few Sprints to find the ideal balance between keeping the end in sight (the single most important aspect of agile development), getting enough done, and maximising the value of the Sprint-change time. For example, Project A may find that one-week Sprints work best, delivering nice compact features one after the other, carrying out their ceremonies, harvesting any feedback to roll in, and jumping onto the next shiny new thing. Project B, however, may have some huge architectural parts to construct, where a four-week Sprint would be more appropriate. Don’t be afraid to experiment. Try them all. More than once if you like. Then settle on a cadence that works for your team. After that, periodically challenge the team: “does this Sprint length still work for us? On this project, with this team, at this time?”. Always be prepared to inspect, adapt, adjust. In particular, if you’re in the beta-cycle where you’re rapidly pushing out updates and consuming the feedback for the next one, a shorter Sprint may be more appropriate.

And of course, this is an ideal time to lay out the ground-rules. While “Working software is the primary measure of progress”, it’s not the only one. The team needs to establish mechanisms that enable it to see how they’re doing, how much work is left, and so on. Agile development is not just blindly whacking out code. While it’s also not about mindless metrics madness, your team does need to provide some kind of visibility of progress.

Mitigation: The team owns it

It’s critical that the team feel the ownership of the process, tools, metrics and mechanisms, so they must be encouraged to actively work through and establish the foundations. Ensure your Coach is on hand throughout this period, to keep a hand on the tiller and point out when things are going seriously off the rails.

Approaching from this people-first angle instils ownership, while quietly laying out the boundaries so that the necessary and valuable processes are retained.

Doing too much at once

Really a continuation of the “going overboard” issue. Agile development is an iterative process. Apply that to your Agile transformation as well. You don’t need to get from A to Z on day one. What’s wrong with reaching C (or even B) in the first instance? Implementing things gradually allows you the time to build trust within the team and around the business, especially if you can find a quick win to put in place early on. Let people get a little comfortable with what you’re doing, and looking forward to the next improvement. It also give you the opportunity to Inspect, Adapt and Adjust as you go.

Mitigation: Iterate, dummy!

You won’t get it right first time

Your first transformation attempt at any organisation is quite likely to be only partially (or even just a little bit) successful. Nothing went as planned. The team is demoralised. Management thinks it’ll never work. You’re wondering why you started down this path.

Mitigation: Celebrate the victories. And get over yourself.

Don’t lose heart when your first transformation attempt flops.

Put a big bright neon-green tick next to the bits that worked well, and consider everything else an opportunity to improve. Inspect, adapt, adjust. More on this later, but the key take-away here is:

Celebrate the victories.

So your first ever Sprint didn’t go as planned? Get over it. They never do. Use your Retrospective (regardless of methodology, it’s critical you have these on a regular basis) to collectively figure out there where, why and how (but never the who) of what went wrong, or what could have been done better. It’s important that the team members themselves identify these things – it’s not about management pointing out mistakes! – and own the remedial activities. By all means have your coach guide and moderate the discussions, make sure the findings and remediations are written down, and stand ready to clear roadblocks for them. Just don’t interfere in the process. Once they’ve arrived at their conclusions, they’ll probably be at least halfway to a solution or at least some ways to improve. That’s a win. Congratulate the team (and yourself) on a positive outcome, secure in the knowledge that you’ve got bigger and better victories ahead of you.

Organisations don’t get it

Generally (but not universally) speaking, the development teams themselves are seldom the primary source of resistance in the transformation process. It’s the rest of the organisation that lacks the education, alignment and buy-in. The project delivery, finance, sales and marketing teams want cast-in-stone dates. These folks work to plan (until the plan changes), and don’t know what to do without the dates those plans are built around. They don’t understand why they can’t have them, and refuse to believe that they never really did. Actually, “refuse” is too strong a word. This is not malice. It’s just failure to appreciate how things really work. We’ll explore the date-fixation issue in a little more detail shortly.

Mitigation: Change the conversation. You’re already winning.

It can be very challenging for an organisation to overcome the inertia of several decades.

“That won’t work here”

There will be countless we-do-it-this-way barriers. You will hear again and again “it won’t work here because…”, at which point you need to change the conversation. It can work. It will work. It does work. It’s already working! The rest is just detail. Start the transformation from the point of view that it’s already succeeded. Now fill in the blanks.

Mitigation: There is no but

You may find it useful to ban the use of the word “but” in your transformation discussions, along the lines of “I can see what you’re trying to do, but…”, or “That might work at XYZ, but…”.

“Yes, but…”

Ban the but! Words are immensely powerful, so leverage that power. Ensure the language stays positive. It’s corny, I know: there are no problems, only challenges. Blah Blah. It works at XYZ? Great! Let’s see what we can learn from them and apply to our own situation.

Failure to adapt

One of your worst enemies is the guy who’s read a book on agile practices and now constantly points out all the things you’re doing “wrong”. Blind obedience/compliance is always bad news. This guy has missed the point completely. The Agile Manifesto doesn’t set out hard and fast rules, it shows emphasis of importance. In fact, it specifically calls out Individuals and interactions over processes and tools.

Mitigation: make friends and influence people

The trick here is to get the book-reader on your side. Never tell him that he’s wrong. Engage him. Appreciate the research he’s done. Work with him to figure out how his input can benefit the team. Never “yes, but…”. Rather lead with something along the lines of “I like the concept. How do you see that improving our experience?”. I’m not talking about patronising him! People with strong opinions are an immensely powerful resource to tap into. Bring him on side and work with him to everyone’s benefit. There’s a side-bonus here too: passionate people are often good influencers. So if you can get him pulling in the same direction you are, you’ve gained a powerful ally.

Mitigation: adapt or die

Your Scrum Master and/or Agile Coach (and for that matter, the entire team themselves) should be constantly challenging the development team (and the rest of the organisation) to Inspect, Adapt, Adjust. If something isn’t quite working right, rather than forcing blind compliance, ask if there’s a better way of achieving the same goals. The Engineering Question can be your friend here. Apply it not just to what you’re trying to get done, but also to the principles you’re following in doing it. If something is working well, ask the same question! Better never stops.

“The system won’t let me…”

Really a sub-topic of failure to adapt, and something to deal with right at the beginning of your transformation process. Your organisation uses the Software Help Inside Track system, but your team can’t figure out how to make it function properly when working in a more agile manner. It just doesn’t quite fit, and can’t readily be modified. It’s causing needlessly complex, overly process-driven work-flows which are anti-agile.

Mitigation: Don’t be driven by your tools.

I can’t over-emphasise this enough. You are far better off using sticky-notes on a wall – if that works for you – than you are with a fancy automation system that gets in the wayMake the tools work for you. And if you can’t, then swap out the tool for something better. Don’t compromise on this.

Over-optimism: “Agile is going to fix everything, instantly!”

This is somewhat akin to the developers going overboard, but this time it pushes down from on high:

“Agile? Great! Now you can do twice as much in half the time with fewer people.”

Experienced transformation folks will cringe in recollection at the number of times they’ve had to rebuff this. Initially, output will drop as your team finds their style and cadence. It’ll pick up again pretty smartly, and if you’re doing most things mostly right most of the time, the output run-rate will climb steadily for a while. Be sure that your organisation is expecting the momentary drop. It will happen, and it’s just a part of the process. It’s nothing to panic about. But it is something to manage.

Agile practices are not a magic bullet.

The work will take as long as it takes. It brings along more transparency, progress-visibility and resilience. Absolutely. It shifts the focus from project/program management to a sustainable cadence of quality output. Yes. But it doesn’t reduce the amount of code to be written.

Mitigation: Manage expectations

As I may have said already, Organisations Don’t Get It. The world has changed. The way we develop and deploy has changed. That’s a heavy, large educational challenge. It’s difficult. It’s scary. People have to fundamentally change the way they picture things. It’s not all going to align at once. This is probably one of the most difficult aspects to manage. It’s critical to set expectations early, and give frequent updates. Be candid and transparent. Call out those areas where you’re having difficulty. Having an executive sponsor on the project can be a great help in removing road-blocks, but only if he/she is completely on-board to start with.

Mitigation: Total transparency coupled with unbreakable positivity

Ensure the senior management team is fully aware that this is not a switch to be thrown. It’s a project in its own right, a process to go through. Keep all stakeholders updated with how things are going, including (in fact, especially) the “bad news”. Like any other project, you’ll probably have a plan, and nothing ever goes to plan. There’s usually some appetite for this, as long as everyone knows where you are and what’s going on. And in reality, it’s a process that never ends, as any decent agile team will always be looking for ways to improve.

For many teams across the organisation, the Agile transformation is going to cause disruption. They didn’t ask for it, and probably don’t care about it. You’ll have your job cut out keeping people motivated and aligned. I’ve said it before and I’ll say it again:

Mitigation: Start with the win.

No alt text provided for this image

You’ve already won. This does work. Now fill in the blanks. Your own positivity as a change agent is contagious, so always show your personal belief in what you’re doing. And “unbreakable positivity” is not the same thing as hopeless optimism. It’s an unshakeable belief, based on experience, that great things are under way, and the outcome is predetermined as a major win.

Lack of trust

It can be very difficult for management to let go of close-oversight command-and-control, and there are no short-cuts here. You (and your team) are going to have to earn the trust you need. Some of that will come from delivering what you promised, Sprint after Sprint. So it’s important that you regularly hit your Sprint goals. Most of the time, at least. Even the grumpiest long-tenured waiting-for-retirement C-suite resident will admit that not all plans stay on track all the time. When you can see you’re not going to hit every target, don’t hide the bad news! This is always the worst thing you can do. Call it out as early as possible, while frantically working in the background to minimise, mitigate and manage. At all times, of course, maintain your calm certainty. Brings to mind the image of a swan serenely gliding across the surface of a lake. It’s elegant and quiet and beautiful. But below the surface, those little feet are paddling like maniacs.

Mitigation: Total transparency

Have an open invitation to the entire organisation to attend your daily stand-ups. Make a special point of personally inviting the sceptics. Let them see first hand where the team is, how far it’s come and how much of a hill they still have to climb. Make your Scrum/Kanban Board and burn-downs/CFDs easily visible to one and all. To that grumpy old executive who’s really not sure about this new-fangled Agile thing, it’s very reassuring to see measurable progress daily. Especially if he’s hung up on the incorrectly perceived lack of predictability.

The postings on this site don’t necessarily represent the views or opinions of my employer, professional organisations, political party, family, car manufacturer or anybody at all, really. I don’t know where they come from. It scares me sometimes.