What We Learned from Taking Agile Workshops Online

One of the most effective ways to demonstrate tricky, real-world concepts is to run a game that strips away any complexity and concentrates on demonstrating the concept in an unambiguous and obvious way. As agile coaches, scrum masters and change agents, we do this a lot during agile workshops. There’s nothing better than getting out the Lego, or a set of coins or playing cards and running a game around a table with one or more teams to really hammer home a salient point.

Unfortunately, the current pandemic has made it very difficult to run such agile workshops and games face-to-face. And, of course, this “new normal” where people increasingly work from home or remotely, is not just about the pandemic. More and more companies have offshore teams, offices in multiple locations and more and more people working from home for at least some days of the working week. It is likely that remote working will only increase after the pandemic as more companies realise the benefits to themselves and their employees.

Technology such as Zoom and Miro have made it possible to move many meetings online, but having attended and facilitated many such meetings, it soon becomes clear that the engagement is nothing like it would be in a face-to-face setting. There are many reasons for this; the body language we rely on in meetings is missing; you can gauge everybody’s reaction with a quick glance around a room, but not when you have to scan 20 small windows, some of which may not even be showing video. Sharing a screen can end up like watching a PowerPoint presentation, and the temptation to drift off and disengage can be strong when other tabs or devices are needily flashing notifications. And those awkward silences — do they just seem longer on Zoom or Teams?

So, we decided to see if we could do something about it. Could we build online agile workshops that fully engage the participants just like “the real thing”?

In collaboration with Matt Phillip, we built the No Estimates game. This was already a really engaging board game, so it gave us a great start. The game is a a simple Kanban game that demonstrates the futility of estimating. Participants play in teams and simply have to estimate how long it will take them to complete a backlog of work, given — unlike in real life — the exact number of items to complete, the exact amount of effort for each item, and the exact amount of effort available. Like real life, however, there are dependencies, urgent items, blockages, deployment failures, and even an evil PO called Vince who throws the odd spanner in the works.

Still, teams have way more information that they would have in real life, and it is assumed they can give accurate estimates in real life, so this should be easy, right?

Well, no. Teams always get it wrong. This game not only demonstrates this and the reasons for it. It also shows better and more scientific methods of forecasting to arrive at the answer to the perennial question “when will my stuff be completed?” (see Learnings below for more explanation)

We’ve run this game many times in various online agile workshops, most recently at the Agile 20 Reflect festival with a number of participants who had never previously met, and the level of engagement is always amazing. As a facilitator, jumping between Zoom breakout rooms, I can see teams working as if they’d been together for years. There is banter and discussion and it really feels like they are around a table playing the game (the participants also say this…).

In the Agile 20 Reflect game, I suspect there was much more engagement than if we’d placed them physically in a room — somehow, the awkwardness of a first meeting was swept away as they concentrated on playing the game on their browser in front of them.

So how come — why was there more engagement?

1. Real-time and local. Everyone in the game is acutely aware of the state of the game as it is right there on their browser. When a player adds effort to a card, it immediately shows up on everyone else’s browser. If a card is blocked, it goes red and everyone knows it’s blocked. Seeing and playing the game locally on your own browser is so much more involving than seeing it on a shared screen.
2. Active participation. Players are always actively doing something. They have no choice but to perform tasks — adding effort to items, pulling in new cards, etc. — and discuss what to do next. It’s very difficult to drift off if you are interacting with the game at very frequent intervals.
3. Focus. It is very easy to popup up a message to make a point. Every few rounds of the game, we can pop up a retro window, for instance, so teams can reflect and adapt. At the end of every round, we pop up an Event card with information and actions to be performed along with the current cards and value delivered. All this helps to keep the players engaged and focused. It’s not possible to drift off if your team mates are clamouring for your effort to be put on a work item…
4. Using real data makes the story real. We can generate all kinds of data and show it to the participants as and when it is pertinent. It is much more powerful to say “this is your cycle time so far” than “here is a stock image of a typical cycle time”. Also, as the game is software, we can do this in real-time, as and when we need to. We can even generate complex calculations such as Monte Carlo simulations with 100,000 runs at the touch of a button.

In some respects, the online game is actually better than the face-to-face version for in-person agile workshops. Any random elements, like throwing a dice to see if something is blocked, can simply be programmed into the code, and are therefore immediate. Cards move columns automatically when complete. The rules can also be more strictly applied; if you are not allowed to add effort for some reason — you don’t have the skills, or it’s the wrong column, for instance, you simply cannot bend the rules in the online version! The game can even be paused and continued at a later date if required.

At the Agile 20 Reflect game, we had participants from the UK, India, Canada, Germany, Australia, the US and the Ukraine. Such a gathering would never normally happen — it would even be highly unlikely at a face-to-face conference — and this is a further benefit of these online agile workshops. People from all over the world can attend and collaborate; ideal for large companies to bring together offshore teams or overseas offices. What better way for the remote company outposts to feel included and engaged. Watching the participants in this game, it was apparent that geographical location was forgotten about within seconds of starting the game.

Learnings from Online Agile Workshops

As fascinating as all this is, and however engaging and fun games like these are, they are only useful if they give valuable learnings. As already mentioned, we can use the actual game data to generate illustrations of these learnings, and this helps to make the debrief at the end of the game relevant and interesting.

As an example, here are some of the learnings from the Agile 20 Reflect game.

One interesting comparison between two of the teams was how they defined and refined their estimates based on new information.

Another finding that always surprises the teams is the correlation between the amount of work required on an item, and how long it takes. Surely the bigger the card, the longer it takes? This is rarely the case, in the game, or real life. Here is the correlation between effort needed and time taken for one team.

As you can see, the correlation is by no means perfect (1), and this is very often the case. We have even seen negative correlations, where smaller cards take longer!

One common estimation mistake is to try and make an estimate more scientific by using data to generate a mean and standard deviation confidence interval. However, unless the data is normally distributed, this is statistically invalid, and prone to error. Here are the distributions of days to complete work items from two of the teams:

As you can see, after 25 cards, the distributions are far from normal; one team is more Poisson, if anything. So, we can’t use mean and standard deviation. What we can do, however, is use the data we have to run a Monte Carlo simulation. Here’s an example for one team where we’ve done 1,000 runs of the data to come up with a forecast on when we would complete 100 cards.

This was a guest blog. Please review our guest blog disclaimer.