What are the Key Metrics for Real-Time DevOps Visibility?

Key Metrics for Real-Time DevOps Visibility
Joshua Platt

By Joshua Platt

If you’ve been reading along, you know that SmartDraw recently commissioned a survey on the topic of visibility for DevOps.

In my first post, we discussed the overall findings. My second post discussed the specific ways DevOps managers found they benefited from enhanced DevOps visibility. In this post, we’ll discuss how you measure great visibility. What are the metrics? What do you need to get right?

Measuring Success

We asked our senior DevOps manager respondents to rate how successful they were with a variety of DevOps visibility goals. We found those respondents who were the most successful with DevOps in general were nailing several visibility metrics.

Pulling Information from Different Data Sources

If you want to know how you are doing with accounts payable, you generate a report from your AP system. For sales, you go to your CRM system. DevOps isn’t that simple. Take a look at this popular depiction of the DevOps tool ecosystem:


Source: Atlassian

If you need insights, you’ll likely have to extract information from multiple DevOps data sourses and combine it to answer your question.

Contextualizing Information to Provide Actionable Insights

One of my favorite quotes comes from Clifford Stoll, an American astronomer, author and teacher.

"Data is not information, information is not knowledge, knowledge is not understanding, understanding is not wisdom."

When people ask a question, they are looking for wisdom. All too often what they actually get is data. The problem with data is that, by itself, it means very little. For example, it's easy to say that a project is delayed a month. It's another to point out that there's one issue that is blocking multiple other issues causing the delay.

Jira issue dependency

Producing Real-Time Reports

The entire goal of DevOps is to accelerate the pace with which teams deliver software. A modern DevOps workflow is fast-paced and highly dynamic. You need the contextualized information we discussed in the last point in order to make the best decisions, but those insights are worthless if they are based on old data.

Enabling Team Members to Generate Their Own Reports

DevOps teams are lean. Every person fills a crucial need and is always just a step away from being a bottleneck. So, asking team members to stop what they’re doing in order to compile reports for upper management is counterproductive. Better is to give upper management the ability to ask – and answer – their questions without involving critical DevOps team members.

Automatically Monitoring Projects

Ironically, the surest way to slow your DevOps pace to a snail’s pace is to try too hard to eliminate failures. Better is to "fail fast," meaning keep your pace fast, but react to failures even faster. You spend less time obsessing about what might happen and get good at reacting quickly to what did happen.

That principle is at play with this final rule – to set-up systems that proactively look at DevOps data, contextualize it, recognize problems and send alerts to team members so they can fix the issues. For example, you may want a report that lets you see at a glance which tasks are either late or at risk for being late.

At risk issues in Jira data visualization

Getting Good at Providing Real-Time Visibility

It’s one thing to know what you need to do to be successful at DevOps. It is another to actually do these things. We found that there was an elite class of DevOps managers who were producing real-time visibility into their DevOps operations. But it wasn’t easy.

In my next post, we’ll look at why it was difficult, and what they ultimately want to see in a tool that can help with DevOps visibility.