“Can you get somebody from Ops please?”
At the beginning of each Scrum of Scrums, we always used to drag somebody from Ops into the room. It’s an optional meeting but with the whole Ops team opting out, it was losing its effectiveness. Now, they come willingly. An unenthusiastic Ops team wasn’t even the problem I was trying to solve. That was a bonus. I was actually trying to solve a problem for the Product Owners.
What was the problem?
The problem was clear from the Post-Its on the wall from my retrospective:
“I don’t know when releases are going live.” “Releases are getting delayed and nobody is telling me.” “Other teams are breaking our releases.”
For us, the Scrum of Scrums is the place where representatives from our Scrum teams discuss things that impact other teams. By and large, the things that impact other teams are releases. We had a problem with those.
On our path to Continuous Integration and Continuous Deployment, we left a vacuum where Project Managers used to be. Nobody was managing cross team dependencies. Builds were failing because multiple teams were checking code into the same code base. Teams didn’t want to do regression testing with other teams’ code. Our automated code coverage wasn’t great. Releases were no longer going out on time.
My team of ScrumMasters was told we had to communicate better. Communicate? We don’t know when it is going live. Ops doesn’t tell us. We can’t communicate what we don’t know. To be honest, I was a bit annoyed that I had been forced to stop doing portfolio management. It was hard, but it worked. People who didn’t understand the challenge were telling me it didn’t need to be done and now they imply it’s my fault that we have issues.
I was turning this problem over in my mind when I went to Scrum Australia and brought it to my coaching session with Neil Killick. He brainstormed a visual release board with me. I created the MVP the day I got back to work and introduced it at our daily Scrum of Scrums. It worked.
Here’s how the Release Forecast Board works:
Most of us are familiar with visualisation boards for our teams. These boards show the progress of PBIs, (stories, bugs, tasks, etc) through the sprint. Post-Its are moved from left to right as they are worked on. Our teams have boards like that already.
The Release Forecast board is different. The board has the next two weeks pictured. Each column is a day of the week. Each release is on a Post-It on the day we think it is going to go live. If the date changes, we move the Post-It to the new date. When I first introduced the idea there was resistance. “Hang on,” somebody said “I don’t KNOW when it’s going live”. Precisely. I’m not asking for a commitment; I am asking for a forecast. If you think it is going live in the next two weeks, put it on the day you think it is going live. If it changes, move it.
The Post-Its have the name of the platform (you might call it a component or an application – it’s whatever gets released in one deployment), the expected version number and the team that is leading the release.
That’s it.
What is the benefit of the board?
The board is there for the communication and cooperation it inspires. It’s like the indicator on your car. Somebody may need to know where you are planning to go.
On the very first day, Marcus, one of the ops guys, pointed to a Post-It and said. “That’s good to know, I need to make a change to correspond with that release on Wednesday.” “It might not go until Thursday,” the team representative said. “No worries,” Marcus said. “I’m not in a hurry, just need to get it ready.”
People immediately started saying things like “We were planning to put Andromeda 1.2 live next Thursday but we see you are planning release 1.2 on Tuesday, can we combine our efforts?” Sometimes we would see eight Post-Its on the same day. “We don’t mind moving our release,” somebody would say. “Just move ours to the next day.” Another person piped up “Don’t touch ours, it HAS to go live that day.” The Ops guy suggested that we put that Post-It on the top of the list, so he knew to do it first. Awesome. I just stood back and smiled.
Engagement
It is a lovely thing to come back after a vacation and see that the board moved on without me. It was up to date. It even improved. While I was overseas, a new arrow was introduced to help people focus on today. It’s not my board any more.
How has the board evolved?
After the board had been up for a little while, cross-team impediments started showing up on the board too. We also found that it made sense to write the actual release dates on the Post-Its when releases went live and leave them in a “Live” column on the right for a little while so we had a record. People naturally started moving Post-Its when it was their turn. They started bringing their pens so they could add information.
What has gone wrong?
After we had been using it for a while, a member of a new team joined. He added a Post-It for the next release and I wrote his team name on the Post-It and whacked it on the board. A few days later he said, “We just have a small item in this release, hoping that another team leads it.” A Scrum Master got annoyed. “Your team name is on that Post-It, that means you are leading that release.” A heated discussion ensued. The new guy was upset. He didn’t want to come back to the meeting. I realised that we had made assumptions about how the board worked without documenting them or sharing them with new people. So I wrote a page on our wiki and I summed it up with the following rules:
Are we living happily ever after now?
Did the board solve everything? Well, as they say in Australia “yeah, no.” CI/CD is not an overnight thing. There are still many challenges to tackle. We improved communication, transparency and visibility and we all know how important that is.
Thanks Neil, for your gracious coaching.
Background Stuff – if you are interested in history and detail
We used to manage the tasks in releases manually in JIRA. A Project Manager would create a set of fix versions. Andromeda 1.2 is scheduled for May 4. Andromeda 1.3 is scheduled for May 18, etc. Each fix version would have a bunch of tasks. Prior to a release, Project Managers would make sure that everything was on track across all the teams. If anything wasn’t going to be code complete and tested by the release date, either we would delay the release or we would kick things out. We would constantly communicate the status of releases so that everybody knew what to expect. Of course mistakes were made in our old system too but we had a shared understanding of what we expected to be in the release.
Now the earth revolves around the sun, we follow Scrum and we respond to a lot of change. We don’t Project Manage. We do, however, still plan. Given that most of our teams run two week sprints, two week release forecasts make sense for us. Like a weather forecast, it’s not all that accurate a week in the future. Nevertheless, you still want to know if you might need an umbrella next week.
When we used to put JIRA tasks into releases manually, we knew what to expect weeks or even months in advance. Now, when the code is committed it is flagged with the JIRA task number. When the release branch is built, a script adds the next fix version to the task automatically. This is more accurate and it is saving us a lot of double handling. However, we don’t know what is in the release weeks or months in advance. We know when it is ready. That’s why we needed the board.