When you run your own shop, you get to decide what's important for your business' growth. For me, that means choosing the types of projects I like and companies I want to work with — and as long as I can make a living, that’s really all I need. I can have the Best Year Ever according to my own internal happiness or the size of my income. Doesn’t matter to anyone but me and my clients.
But the capitalist growth-only system incentivizes even the largest businesses to prove what they're doing is valuable to others so they can make more more, scale up, etc. In the age of proprietary digital data where everything is top secret do not look at me just my product but still give me money and trust, companies still have to report on success metrics to be attractive to investors, shareholders. It’s always best if you can make those metrics go up and to the right.
I’ve never taken an econ class but I know a thing or two: Growth can’t multiply forever, there are only so many target markets, you can’t please everyone, and at some point you’re going to not grow as fast as you did before. Unless!
Unless you move the target! Which Netflix did once in 2019 and again in 2021.
The 2019 KPI: Why two minutes per unique household doesn't make sense
In December 2019, Netflix changed its content measurement KPI. Instead of watching a show or movie to 70% completion to count as a View, from 2019-2021 Netflix counted a mere two minutes as a view. Yep. If you or anyone in your household watched a show just a titch longer than your average Ramones song, Netflix would count that as a view.
Tyler Coates broke the two-minute metric down nicely in Wired and said it better than I could:
…Two minutes of The Irishman, which runs three and a half hours, amounts to 0.95 percent of the total film. You’d barely get through the opening tracking shot at the assisted living facility where an aging Frank Sheeran (Robert De Niro) is primed to reflect back on his long life of crime. I like to imagine me, an occasional movie critic, turning off a film after two minutes and writing a review. I mean, according to Netflix’s data, I’ve seen it. Two minutes is enough!
Netflix’s argument was that the new metric is closer to what YouTube and The New York Times counts as a video view. Technically, this is true! If you are measuring for “eyeballs,” then eyeballs you can get for two minutes.
If you’ve been following along, you’ll know that you should never really have a single KPI for content performance, especially not one that can be easily inflated, but hey! Anything to impress shareholders?
Measuring for eyeballs is for chumps. Especially if you care about content. Netflix knows that, and in their Q3 2021 shareholder letter, they shifted to a more accurate, yet still easily manipulated metric.
Want to find a better way to evaluate your web content performance? Take Understanding Google Analytics 4 and learn the key metrics and setup that leading website owners and content professionals use to evaluate and plan for content.
Designed especially for non-marketers and non-data scientists.
The 2021 KPI: Are total streaming hours any better than two-minutes-per-household?
Following the shareholder letter from October 2021, Netflix now counts total streaming minutes of any program—limited series, syndicated hit, movie or otherwise—as its primary reported KPI for content success.
The streaming pioneer also created its Netflix Top10 subdomain, where fans, entertainment industry wonks and investors can dive deep into what's actually being watched. It's a performance of transparency spawned by new competition and a move that syncs TV measurement giant Nielsen's current methodology. With Nielsen and Netflix aligned, measurement of total streaming minutes is more or less an industry standard.
The advantages of using total streaming hours as a KPI:
- With Netflix's current subscriber base, the streaming hours figure always looks big. Everyone else's numbers look like tiny little baby numbers in comparison.
- Time translates fairly easily to dollars.
- It rewards both binge-viewing and repeat watches, two longstanding viewer behaviors that Netflix didn't invent but certainly exploits.*
- It translates across media: whether you spend time watching movies, playing games or streaming a personalized algorithm of short videos, total time watched is a comparison metric.
- It's device-agnostic, whether you watch on a phone or a massive home entertainment system.
- It's culture-agnostic: across the globe, we agree that time is measured in seconds, minutes, hours, days.
- It's familiar. Particularly with the rise of time-tracking and productivity apps, contemporary people, especially investor types, understand comparative minutes spent.
Hours watched is a familiar metric, but it's not excessively meaningful. We understand the limits that hours have, the space they take up in our lives, but in aggregate, at a global scale, they lose any other meaning besides "Netflix big!"
The disadvantages of total streaming hours:
- "Total hours viewed" draws minimal distinction between tv series, movies or other forms of entertainment, handicapping a tightly edited 6-episode series or 90-minute independent films over a hastily produced but labyrinthine soap opera. Serial formats, ensembles of flawed but likable characters, extensive exposition: it's Dickens and Hugo for the streaming era.
- Converting the metric into larger units is existentially triggering: even with a healthy allowance for pure frivolity, knowing humanity globally spent 5,228 years or 5 centuries streaming The Tinder Swindler is a real bummer.
- Netflix makes no distinction between active watching and passive playing... but that's always a challenge to measure.
- Total minutes watched don't actually translate to revenue unless a robust subscriber or advertising mechanism is in place behind the scenes.
- Hours watched as a KPI rewards engagement just like every other growth-only model: by rewarding addictive behaviors, sensationalism, fast production over fact-checking, production shortcuts that sidestep Hollywood industry union standards whenever possible, etc.
Netflix's cumulative global hours watched is performative marketing at its finest: A Goliath number, a varsity letter, and an assertion of data-driven superiority that makes the Hollywood system feel beyond antiquated.
*When I was six, I watched a homemade VCR tape of Who Framed Roger Rabbit? every Saturday morning for months. I wish I could tell the child version of me, You're pioneering a media behavior!
The emerging universal measurement for content engagement
In measuring time spent, Netflix is not alone. With measurement powerhouse Nielsen's ONE product (backed by Disney and Magna), Spotify's embrace of minutes listened, the gaming industry's established embrace of hours per game, and Netflix's switch to a cumulative hours watched as its content KPI, the standardized engagement metric for video and gaming content in the coming decades will likely be time spent per day and total minutes or hours viewed.
That is, unless Netflix changes its mind again in two years and shifts to another metric to distinguish itself from the Nielsen pack.
Netflix and its fellow streaming networks produce different types of content than TikTok or YouTube influencers, but the hours watched metric asserts that Netflix's professional production can easily compete with those user-generated platforms. With 2021's content KPI change, Netflix wants its audience and shareholders to understand that even without a Lannister or Boba Fett, its brand of entertainment can command our time and attention at massive scale.
What does a time-based KPI mean for smaller scale content operations?
Adopting any kind of universal standard for “digital content” is limiting, to say the least. Since the mid-2010s, pivot-to-video folks have said things like “70% of all data consumed online is video” to sell itsy bitsy two-minute marketing videos… but they never couple that stat with the fact that professionally produced, sharply streaming Peak TV has far more reach and eats more resources than user-generated influencer content on TikTok or Snap.
Does that mean short-form video is dead? Not at all. But independent creators and brand content marketers alike may want to reconsider their video content strategies, develop production schedules and measure their efforts with fresh eyes.
By all means: don't start measuring "total hours consumed" until you've built a significant library. Cumulative hours are a decent metric, but Netflix-sized numbers probably aren't within your reach.
Instead, consider how to prove the value of your content in metrics other than base audience size or traffic. Average or estimated time spent per day isn't hard to calculate with web analytics tools, and business stakeholders know that even five minutes per day is a decent amount of mindshare in a world of constant notifications.
Netflix and its data prowess
The 2019 two-minute metric and even the new total hours watched KPI are red herrings that cloud the complexity of the data Netflix uses to make content decisions. Netflix’s holds reams of highly meaningful data at its fingertips, more than any movie studio. That data has been organized from the start, and Netflix shares hints of its data-backed process every now and then, like when they tell you about how they knew House of Cards would be a hit. (I mean, that was always a no shit, Sherlock.)
Netflix’s data informs every creative risk they take, including their confidence in bankrolling stories for diverse audiences that legacy studios and TV networks wouldn’t touch. They have data about how we watch, when we watch, whether we like the shows or not, all that data from the first 10 years of streaming when I rated movies I liked from one to five stars which it does not collect anymore even though I loved that feature.
At the end of the day, Netflix doesn't care what you watch, just as long as you're spending your time with Netflix and renewing your subscription. But I suspect it's also considering the following metrics when evaluating content to produce or acquire:
- Number of households who viewed all of a movie/series (streaming minutes divided by length of the content)
- Average daily time viewed per household (what Nielsen's measuring these days)
- Rate of content consumption per household (# of shows viewed monthly per household/average monthly time spent watching Netflix)
They're creating their own benchmarks, devising their own metrics and understanding patterns of user behavior, optimizing them for profitability. It's challenging, but not out of reach, for smaller organizations to adopt similar systems.
Anyway, content and marketing folks: Gather your data. Consider your content pillars. Benchmark like a pro. And even from your best, most trusted vendors — and friends know that I will go to bat for Netflix as a Great Content Company — take it all with a grain of salt.
The good content companies don’t want eyeballs. The best ones want minds, motherfuckers!
Err… the best ones want time, actually. The best ones measure impact in time.
Interested in learning more about content performance measurement from this author? Sign up for the free newsletter OR