Content analytics KPIs vs. dashboards | The Content Technologist

This post originally appeared in the August 13, 2020 issue with the email subject line "How to measure content performance without KPIs" and a review of link management software Pocket.

Were this a well-researched magazine piece and not a newsletter, I would begin with the history of the KPI. But this is not that article, I’m not an economist, and we’re just going to start with where KPIs are, as I understand them, in 2020.

If you’ve never had to work toward or establish a KPI, here’s an overview: Short for key performance indicator, a KPI is the core measurement of business success. A KPI can be a calculation of multiple measures, but most often a KPI is typically a single metric, a good grade, a number to aim for when you can’t directly aim for straight-up revenue.

Why does everyone use KPIs?

KPIs are phenomenal for getting everyone on the same page. For example, together we can all agree that, say, the fewest number of deaths is a great KPI for measuring COVID response. When we agree on that KPI, then when an executive says, “But look at our case death rate,” which means case fatality ratio, we can say, “Yes, our case death rate is... ok? But it’s not our KPI, which is numbers of deaths.” Or something like that.

A reporter scrunches up his face, looking at a chart for a data point that has been misread by an off-screen companion [gif] — How many of you watched this interview and had flashbacks explaining basic data points and paper charts to high-powered executives? And let me tell you, executives absolutely love it when young women tell them, “I think you’re reading the data wrong.”

KPIs are also fantastic when programming a computer… or optimizing an advertising algorithm. A computer can only optimize toward one outcome at once. The KPI is the ur-metric, the definition of success, and the algorithm will do everything it can to meet that goal.

When I help clients develop measurement strategies for content marketing programs and campaigns, I say, “We need a true north, though. We need a KPI. Do you want to measure engagement or visibility? Soft conversions or number of sales? The computer can only optimize toward one.”

A screenshot from Robocop that reads "Prime Directives: 1.) Serve the public trust. 2.) Protect the innocent 3.) Uphold the law" — Robocop is a movie about prime directives and changing KPIs. I intentionally didn’t use the phrase “prime directive” because my partner and I also got into a long discussion about the phrase “prime directive” as it’s meant in Star Trek vs. Robocop. Welcome to quarantine, where nerding out is easy.

Optimization means choosing a KPI because the computer can only optimize toward one thing. And KPIs help focus our efforts on tactics that support that KPI. If the KPI is reader engagement, I will use a different content structure and optimization tactics than if my KPI is brand mentions. Robocop is a movie about prime directives and changing KPIs. I intentionally didn’t use the phrase “prime directive” because my partner and I also got into a long discussion about the phrase “prime directive” as it’s meant in Star Trek vs. Robocop. Welcome to quarantine, where nerding out is easy.

The KPI that will live in infamy

The most famous digital KPI in all of publishing was Gawker’s scoreboard, where writers were ranked by the number of unique pageviews, compensated based on pageviews, rewarded for how much trouble they could stir up.

Following Gawker was the age of Chartbeat and “real-time analytics.” As a digital writer, I never worked in a Chartbeat newsroom, but I was fascinated with how one could measure exactly what worked to drive more pageviews or eyeballs or whatever. And I was frustrated with writers who dismissed analytics platforms like Chartbeat outright because I saw the value: with platforms like Chartbeat we could understand what our users wanted, or so we thought.

[Read my proposed alternative to pageviews here.]

For media businesses operating on a pageviews- or traffic-based model, content writing and publishing rewarded exactly the same type of stories that newspapers and cable news produce: short, neatly packaged content with maybe an outrageous or ranty element, but also a beginning and end. It didn’t reward the long-term engagement and reader loyalty that drives subscriptions.

So we’ve switched our KPIs. Now media companies reward for how many subscriptions they can bring in. We’ve gone from “vanity metrics” to “business metrics.” But they’re still single KPIs.

Shifting from ur-metrics to data stories

If you’re in content marketing, you may also have some sort of performance-based rewards or flat-out compensation based on a single KPI. Did you grow engagement by 5%? Did you increase paid subscriptions with your fierce opinions?

If you’re compensating or rewarding, or if you’re reporting up to leadership on performance based on a single KPI — not an uncommon practice — please consider broadening your view to more complex content performance dashboards.

When you reward people for optimizing toward a KPI, you’re rewarding only certain optimization behaviors and not necessarily incenting long-term content impact or even recognizing the full benefits that your content marketing is providing. By rewarding based on one, or three, metrics, you’re holding people accountable to computer-based (or finance-based?) standards, when it’s increasingly clear that content businesses cannot survive that way.

Those single KPIs don’t account for all those fun economics terms like outliers and externalities, which are having very real effects on our society right now. Most KPIs reward optimization and efficiency but not, for example, inclusiveness in sourcing, depth of research and citation, story structure, innovation and originality, or any non-emotion-driven risks. Brand standards can control for style and substance, but it’s difficult to translate those into reports, or even test changes, if they’re not recognized in the KPI.

It is crucial to measure content performance. And it’s always fantastic to receive shares, engagement, subscriptions, leads. But when deciding editorial direction or identifying success, complete data stories built with long- and short-term metrics and illustrated with dashboards.

A man shakes his head. A hand rubs a car dashboard with a mat. [gif] — What an unsatisfying dashboard!

Why content dashboards?

Good content performance dashboards answer core business and audience questions with multiple dimensions and different types of metrics. These dashboards should always be subject to regular review, discussion and interpretation, whether it’s monthly or quarterly. (Daily is unsustainable with content reporting, especially in the age of inconsistent content lifecycles and algorithmic recommendations.) What an unsatisfying dashboard!

As creators, we need to fully understand and define the metrics that tell our story and provide feedback how those metrics are changing. If you’re on the management side, start facilitating conversations with performance-obsessed leadership and those who mistrust digital measurement. Find some kind of common ground and better system with content-specific dashboards. Begin the work of telling and reading clear stories with those dashboards, interpret them with our vast and deep storytelling abilities, and provide insights not only on optimization but on new directions.

We are already paying attention to so many details and ideas in our content development processes. Our dashboards need to better reflect the process and complexity of content creation.

As an industry — both in marketing and editorial content — we need to be better at giving bad news when KPIs don’t behave the way we expect. In business, we’re always told to chase the positive, highlight the action. We celebrate spikes in performance but provide excuses for drops, when we need to be setting an expectation of consistency. We need to look at other factors that may be affecting our readers — from the number of banners or pop-ups on a page to major global changes. And we need to set up reporting systems that allow us to tell broader and longer term stories with our data.

The only positive that I can see about the Coronavirus epidemic: We’re all better at reading charts and telling complex data stories now. We know the stories need to be more nuanced and use multiple metrics. We know not only to take into account the R0, but the rate at which we are testing, the absolute number of cases of COVID, and the absolute number of deaths. To understand what’s happening in our daily world, we need to embrace all the chart-reading skills we learned in school that maybe we put behind us with our English major.

[I also wrote How are we being optimized? at the beginning of the pandemic.]

Content performance dashboard dimensions: Here’s a start

So how do you build a content performance dashboard? As with all things digital, every case is different. Developing a content performance dashboard is a collaborative process that involves editorial input and intention like any other piece of content.

Here are a few dimensions to consider when starting out with your dashboard:

Visibility - Have people seen the content?
Engagement - Do people interact with or share the content?
Sales/conversions/subscription - The money numbers
Sourcing - What is the quality of sourcing and research for the content?
Entity/pillar support - How does the content align to a full body of research?
Churn - Did you lose any support because of a content campaign? (I have never seen a performance marketing dashboard that looks at churn and that’s alarming!)
Audience loyalty - How often do users return for your content?
Volatility - How volatile are the channels where we are posting? The topics we are posting about? Are the algorithms changing frequently? Are the people changing?

For more specifics on how to build a better dashboard, watch this space. Or, if you’re in need of a dashboard and content performance measurement strategy, like, tomorrow, reply to this email.

Measurement is not going away (and that’s a good thing!), but it’s changing drastically.