Research, The Tow Center

Our First Three Discoveries About Audience Metrics at Chartbeat, Gawker, and the New York Times

0

Last year, Buzzfeed published lists of the most tweeted, Facebooked, and searched-for stories across its now-defunct partner network, which included sites like The New York Times, the Huffington Post, the Atlantic. The lists provided plentiful (if unsurprising) fodder for anyone who is worried about the future of participatory democracy in an age when we get more and more of our news from Facebook: the top 10 most-shared stories on the social networking site included “27 Shocking And Unexpected Facts You Learn In Your Twenties,” “What Happens If You Text Your Parents Pretending To Be A Drug Dealer?” and the Times’s wildly successful dialect quiz, “How Y’all, Youse, and You Guys Talk.” But the lists were also striking for a different reason: they underscored the fact that the online habits of news readers are being tracked constantly. While Buzzfeed’s lists focused on clicks and shares, many analytics dashboards go far beyond these measures to include time spent reading, scroll depth (how far one scrolls down a page before clicking on something else), recirculation (how many pages one consecutively visits on a particular site), and visit frequency over time.

The sheer ubiquity of this data makes the longstanding professional argument about whether metrics have any place in a newsroom seem almost quaint. The question isn’t whether metrics should be in newsrooms to begin with – they already are, and they’re not going anywhere. The question being debated nowadays is: now that all this data is available, what are the best audience measures of audience behavior and what should journalists do with them?

Important questions, to be sure. But the premise of my Tow project is that before attempting to answer these normative questions, it is helpful to investigate some empirical ones. How do analytics companies produce metrics? What values, assumptions, and ideas about journalism are embedded within these numbers? How do analytics tools interact with established work routines and organizational dynamics in different types of news organizations? Answering these questions requires that we set aside the idea that data can “speak for itself,” and instead examine the creation and use of news metrics as social processes – that is, as active decisions that are carried out by groups of human beings. Over the past year and a half, I’ve been studying metrics this way by observing day-to-day operations and conducting interviews at Chartbeat, Gawker Media, and the New York Times.

Here are some things I’ve discovered:

  • News metrics are hard to interpret because journalism has multiple aims. The meaning of metrics is relatively straightforward in fields with a singular, easily measurable goal: the Oakland A’s in Michael Lewis’s Moneyball were trying to maximize wins, and everyone agreed on what constituted a win. Journalism is (and has always been) considerably more complicated, because there are many ways a news story can be said to “win”: it can break news, it can prompt a legal change, it can shatter a damaging myth, it can be read by a huge number of people, and so on. This complexity means that news metrics can come to resemble Rorschach tests; as a reporter at the Times put it, “you rarely have an apples to apples comparison….There are so many other things kind of confounding it. It’s very easy for everybody to read their own agenda into the numbers.” At Chartbeat, employees wrestle with the question of how much the analytics dashboard should interpret metrics and recommend actions. Provide too little interpretation and the clients may see the dashboard as insufficiently useful or “actionable”; provide too much and editors may view the tool as a usurper and resent it. In sum, the way in which metrics are interpreted (and by whom) tells us much about the power dynamics and internal politics of the media field.
  • Organizational culture matters – a lot – when it comes to how metrics are used. In other words, the mere existence of data in newsrooms does not tell us much about what journalists do with it. Case in point: the Times and Gawker both use similar Chartbeat dashboards (among other analytics tools) but the organizations use this data in completely different ways. At Gawker, metrics are a prominent presence in the newsroom: large screens displaying real-time traffic rankings of stories and individuals famously loom over writers’ heads in the company’s offices; these rankings are also publicly available online and help determine bonuses. At the Times, access to analytics tools is mostly confined to editors (though this may soon be changing in the wake of the Innovation Report), and reporters’ traffic is not a factor in performance reviews. These different approaches for dealing with the same metrics can be traced back to each organization’s particular history, structure, and culture – yet these factors often get overlooked in conversations about how metrics are affecting news.
  • Metrics serve social and emotional functions just as much as rational ones. We tend to think of the analytics dashboard as a dispassionate tool whose purpose is to provide objective data about reader behavior. For this reason, metrics have gained a reputation as ego-busters, as journalists can discover that their readership is smaller and far less attentive than they imagined. While some find this information to be helpful, if humbling, others can sour on a tool if it only tells them things they don’t want to hear. To avoid this, Chartbeat builds into the ostensibly neutral dashboard opportunities for optimism and celebration, sometimes even at the expense of the data’s utility. For instance, the dashboard’s traffic dial is designed to “break” when clients’ traffic is surging past a pre-set cap. As one Chartbeat employee put it to me, when the dial maxes out “the product is broken, but in a fun way… if you didn’t have that excitement, [the product] wouldn’t work.”

These findings, along with others I’ll introduce in the full report (to be launched in late March) illustrate a basic – though easily overlooked – truth: to know the implications of metrics for journalism, we must first understand how this data is created, interpreted, and used by real people in actual organizations.

The Tow Center

Upcoming Tow Tea: New Beats in Digital Media Thursday, September 25, 4-6 PM

1

Tow Tea
New Beats and Covering Diverse Beats in Digital Media

Thursday, September 25th, 2014
4:00 pm – 6:00 pm
The Brown Institute for Media Innovation

The Tow Center for Digital Journalism is thrilled to kick off our Fall 2014 Series of Tow Teas with an exciting conversation between Zave Martohardjono and David Noriega.  All are welcome and encouraged to attend.

Zavé Martohardjono is a Brooklyn-based trans* artist who works in performance, movement, video, and text. With roots in documentary filmmaking, Zavé became interested in video and media while studying International Relations and Political Economy at Brown University. He went on to grassroots filmmaking and youth media education, later receiving an M.F.A. in Media Arts Production at the City College of New York.

David Noriega is a reporter with Buzzfeed.  Born in Bogotá, Colombia, he moved to the U.S. as a teenager. He graduated from Brown University with a degree in comparative literature in 2008 before pursuing his Master’s at Columbia University Graduate School of Journalism. Noriega is interested in covering social justice issues and the Latin American diaspora.

Open to the public

Announcements, Events, Past Events, Research, The Tow Center

Tow Center Program Defends Journalism From the Threat of Mass Surveillance

3

Knight Foundation supports Journalism After Snowden to ensure access to information and promote journalistic excellence. Below, Jennifer Henrichsen, a research fellow at the Tow Center for Digital Journalism at Columbia Journalism School, and Taylor Owen, research director, write about the expansion of the program.

We’ve long known that it’s easy to kill the messenger. Journalists are murdered all around the world for speaking truth to power. But, it wasn’t until recently that we realized how mass surveillance is killing source confidentiality, and with it, the very essence of journalism. By taking away the ability to protect sources—the lifeblood of journalism—surveillance can silence journalists without prosecutions or violence. Understanding the implications of state surveillance for the practice of journalism is the focus of our project, Journalism After Snowden.

We’re in an age of mass surveillance and it’s expanding. Metadata can reveal journalists’ sources without requiring officials to obtain a subpoena. Intelligence agencies can tap into undersea cables to capture encrypted traffic. Mobile devices, even when powered off, can be remotely accessed to record conversations. The extent of manipulation and penetration of the technology that journalists rely on to communicate with their sources makes it difficult—if not impossible—for journalists to truly protect them. And without reasonable assurances of protection, sources will invariably dry up, cutting off a supply of information about government wrongdoing which for more than a century has been a critical balance of power in democratic governance. And journalism without sources is not journalism at all; it’s public relations for the powerful.

So what can we do? With generous funding from The Tow Foundation and Knight Foundation, the Tow Center for Digital Journalism at Columbia Journalism School seeks to address what we think are three core challenges facing journalism in the age of state surveillance.

First, more journalists and news organizations need to take source protection seriously. They need to conduct risk assessments and embrace digital security tools and techniques. They need to arm themselves with knowledge of their legal rights—or lack thereof—and conduct a thorough audit of how the technology platforms they use retain and release data. And more news organizations should consider implementing technologies likeSecureDrop, an open-source whistleblower submission system, which enables media organizations to more securely accept documents from anonymous sources.

Second, we need to strengthen collaboration between journalists and technologists. Bridging this professional divide is critical to ensuring journalists can reach out to trusted technologists for expertise and technologists can better understand the challenges that journalists face and create more user-friendly tools that address their needs. Journalists also need to be more skeptical when problems with their devices arise. Rather than immediately running to the Apple store to wipe their devices (which can actually hide the problem), journalists should enlist technologists to help determine if there is a more sinister cause than simple equipment malfunction. Researchers and technologists also need to join together to develop a system to collect and anonymize data showing digital attacks against journalists so researchers can analyze these attacks, ascertain potential trends and identify possible solutions.

Third, journalist educators and journalism schools need to discuss how to integrate digital security curricula into their classrooms. Currently, most journalism professors provide ad hoc digital security education—if they do at all. Digital security education needs to become more mainstream in journalism classrooms to ensure emerging journalists are cognizant of the real risks they and their sources face in this changing environment, and to foster the confidence they need to better protect both.

The Journalism After Snowden Project seeks to contribute high-quality conversations and research to strengthen the national debate around state surveillance and freedom of expression. The initiative will feature a yearlong series of events, research projects and articles that we will publish in coordination with Columbia Journalism Review, and it will forge new partnerships with the individuals and organizations that are already doing great work in this space. These will include: a workshop bringing together technologists and journalists in San Francisco, a public lecture by Glenn Greenwald; a lecture series in partnership with the Yale Information Society Project; an edited volume likely to be published by Columbia University Press; a poll on the digital security practices of investigative journalists to be published with Pew Research Center; several research reports on digital security teaching and training for journalists; and a conference on national security reporting in Washington, D.C.

By tackling these challenges together, we’ll help to prevent the death of journalism at the hands of mass surveillance and ensure journalism after Snowden is stronger, not weaker.

Innovation Showcase

Innovation Showcase 2014/ Dropping the Stick

4

Dropping the stick? Even as NYC recycling falls, the city cuts fines in half

NYCrecycling_bannerWith a recycling rate at less than half the national average, New York City is no recycling Mecca.
 
In fact, the city’s recycling rate has been decreasing since 2005. In 2013, it was at 15 percent, less than half of the national average and far from former Mayor Michael Bloomberg’s goal to have it at 30 percent by 2017.
 
Enforcement has also become more lax. City data shows that last year, the city’s fined recycling violators only a third of the levels seen in 2009.
 
While the City Council has raised the fine amount for larger buildings, the total number of fines issued per year has dropped by more than half in recent years.
 
Why has the city dropped the stick on recycling?
INNOVATION SHOWCASE 2014 • MAY 12TH & 13TH, 6PM – 8PM • COLUMBIA JOURNALISM SCHOOL

Innovation Showcase

Innovation Showcase 2014/ Drones At Home

7

Drones At Home

DronesAtHomeDrones at Home is a blog curated by Columbia Journalism School students Olivia Feld, Robert Hackett and Julien Gathelier. Drone technology is transforming many industries. Once the domain of militaries and intelligence services, drones are now being made for commercial and domestic use. The rules and legal regulations regarding this new frontier of technological innovation are still being defined. Drones At Home covers all the latest in drone news, from amazing videos taken in Alaskan glaciers and erupting volcanos in the South Pacific to recent dogfights between hobbyists, businesses and the Federal Aviation Administration. Our series #meandmydrone also showcases drone pilots and their drones.

INNOVATION SHOWCASE 2014 • MAY 12TH & 13TH, 6PM – 8PM • COLUMBIA JOURNALISM SCHOOL

Innovation Showcase

Innovation Showcase 2014 / Browser Topic Reader

5

Browser Topic Reader

Noura Farra, Philip Liou, Joseph Taiwo Orilogbon

BrowserReaderPromo

One way to increase engagement with news articles is to provide users with visual feedback about their favourite news topics and reading habits.  Currently, users spend a lot of time online reading news articles about various topics which can change from week to week. However, they rarely have a convenient and accurate means of aggregating statistics which can enable them to understand their own reading tastes and trends.

By providing analytics to users about their news consumption trends, our tool provides feedback that can alter news consumption positively. For example, a user might realize she is spending too much time reading about “Lindsay Lohan drug addiction” and that she should spend more time reading about other “worldly” events, such as “presidential election 2016”. By providing feedback on reading habits, the proposed tool would increase user engagement with news articles.

The proposed tool we developed is a Chrome browser extension which visualizes a user’s top consumed news topics based on her browsing history. It shows users the breakdown of types of news stories on a daily, weekly, or monthly basis, and allow them to keep track of their news interests, as well as how they change over time. The extension analyzes news articles based on URL history from the browser and presents a summary of the most popular topics for the selected period, in the form of two types of visualizations: a stacked area chart showing evolution of trends over time, and a Treemap which shows the importance of different news topics to users, based on distribution of keywords. The user has the ability to show and hide topics, as well as zoom in on different topics in the Treemap. Finally, we also show the user links to example articles from each topic.

 

INNOVATION SHOWCASE 2014 • MAY 12TH & 13TH, 6PM – 8PM • COLUMBIA JOURNALISM SCHOOL

Innovation Showcase

Innovation Showcase 2014 / Story South Asia

3

 Story South Asia

@Story South Asia , Isheta Salgaocar, Shiwani Neupane, Roshan Ghimire 

StorySouthAsiaStory South Asia started in 2013 when we noticed that we couldn’t find an  in-depth analysis of the region in any publication. We noticed that most South Asian news was very India centric, and wanted to change that, so other nations could get a more equal voice. Today, Story South Asia has more than 600 followers on Twitter and our website has around 15 contributors.

We are mostly interested in opinions, human-interest stories, reviews, interviews, photo-essays, editorials and lists. We are not looking for breaking news – the wires have that covered. We strongly believe in voices from the region: not just numbers and statistics on how many people died. At Story South Asia, we believe in putting a face behind a news article, and we are always looking for contributors. If you are interested, write to us at storysouthasia@gmail.com.

THE PROJECTHOW IT’S MADE

INNOVATION SHOWCASE 2014 • MAY 12TH & 13TH, 6PM – 8PM • COLUMBIA JOURNALISM SCHOOL