A Measurable Effect: Using Metrics in the Newsroom

By Sybile Penhirin

Metrics have become an inevitable component of today’s journalism. Many websites, such as Chartbeat and Google Analytics, offer various ways for newsrooms to measure and develop their audience. Even people who do not necessarily subscribe to these services can gauge their stories’ success by seeing how well they do on social media such as Twitter and Facebook.

However, there has been little empirical research on how these metrics are produced and how they affect newsrooms’ cultures and journalists’ daily work, said Caitlin Petre, who recently published a report on the topic.

Petre presented the main points of her study, “Traffic Factories: Metrics at Chartbeat, Gawker Media and The New York Times” at a Tow Center event on Thursday night. You can read her report here and the key findings here.

She then invited three panelists to join the conversation: Sam Henig, one of The New York Times digital deputies in charge of broadening the use of metrics throughout the company’s newsroom, Chadwick Matlin, a features editor for FiveThirtyEight who is helping develop that newsroom’s approach to audience analytics; and John Herrman, whose ‘Content Wars’ series for The Awl examines the journalism industry’s metrics-driven moves and counter moves.

When understood and used well, analytics can help newsrooms know and target their audience better, the panelists said. Editors can manipulate different elements, such as headlines or the time a story is published, and then analyse the metrics in real time to determine which option attracts the most readers.

Matlin said he recently changed the photo on a published piece about Deflagate, which proved to be an efficient move to retain readers.

Similarly, The New York Times published the first part of their one-year long investigation on nails salons early on Thursday morning to optimize the number of readers reached.

“It’s very obvious to us that we should be publishing things at times when our readers are coming to us,” Henig said when talking about the publishing process of the company.

This strategy ensures that not only an optimal number of readers read the piece, but there is also a greater potential for the story to be shared and to go viral.

Thanks to analytics, media companies can explore dozens of features to optimize their stories’ spread.

“If you work at a place that is truly analytics-savvy, the thing that is most fluid is form,” Herrman noted.

A place like BuzzFeed, which is very analytics-savvy, seeks new ways of delivering their entertainment and news stories by experimenting with innovative concepts and formats for instance, he said.

“You can categorize that as a slippery slope, but I’m not sure what’s at the bottom, a lot of traffic and a lot of posts that people really like and share online,” he said.

Metrics, used as feedback, can also have a positive impact on writers and editors. In her report, Petre found that journalists sometimes turn to these data as a reassuring reminder of their professional competence.

“If you write online, you can sort of get the feeling sometimes that no one is reading or that you might not even exist, you need to be reminded a lot,” said Herrman when asked why metrics matter.

But data used in newsrooms also come with several underlying issues, Herrman, Henig, Matlin and Petre noted.

One of the pitfalls, for instance, is for the company’s end-goal to become reaching as many “clicks” as possible.

News organisations experimenting with their stories’ features in order to optimize their audience have to be careful that the piece form does not overwhelm its editorial part, Madlin said. A stand alone slideshow on the Syrian conflict might get a lot of visitors but is probably not the best editorial way to tell the story, for example.

“You write to be engaging and you can take that way way too far, analytics provide a lot of temptations to do that,” Herrman said, adding this trap was not so much inherent to analytics but rather to a loss of perspective from the institution’s part.

“That’s the kind of thing that happen in intuitions where you don’t have structures built around analytics, you don’t have someone to interpret it for you,” Herrman said.

To be well understood and used, metrics need to be placed into context and they cannot be interpreted on the fly, Petre noted in her report.

Henig agreed and explained that after their Innovation report, the New York Times brought several specialists – including a former hedge-fund employee, analytic experts and employees from the company’s product side – to form their team handling metrics.

To learn more about Metrics at The New York Times, but also at Gawker Media and Chartbeat, you can download Petre’s report by clicking here.