What the Tesla Affair Tells Us About Data Journalism

February 21, 2013 by

Consider for a moment two scenarios.

One, a malicious energy reporter tasked with reviewing an electric car decides he is going to fake the review. Part of this fictional narrative, is that the car needs to run out of battery power sometime in the review. He arrives at one of the charging stations, and instead of plugging in, spends a few minutes circling the parking lot trying to drain the battery.

Second, an energy reporter is tasked with reviewing the potential of a new electric car charging network. He arrived at one of the charging location in the dark, and can’t find the charging station. He drives around the parking lot several times looking for it, before finding it and charging his car.

Here is the thing. As Craig Silverman recently pointed out to me, we actually have no idea, based on the interpretation of the review data released by Tesla, which narrative is true. All the data shows is a car driving around a parking lot. And here in lies the principle lesson from the whole Tesla affair: Data is laden with intentionality, and cannot be removed from the context in which it was derived. We do not know, from these data alone, what happened in that parking lot.

David Brooks touched on this very issue in a recent (somewhat overly maligned in my opinion) column on the limits to big data. While his Italian banking analogy felt misplaced, there is actually a large amount of research backing up his general themes. And his point that data struggles with context, is directly relevant to the Tesla dispute:

Data struggles with context. Human decisions are not discrete events. They are embedded in sequences and contexts. The human brain has evolved to account for this reality. People are really good at telling stories that weave together multiple causes and multiple contexts. Data analysis is pretty bad at narrative and emergent thinking, and it cannot match the explanatory suppleness of even a mediocre novel.

In the case of the Tesla review, it is this context that was both poorly recorded by Broder, and which is missing from the Tesla data analysis. This does not mean the analysis is wrong.  But it does mean it’s incomplete.

A couple of further points about the role data played in this journalistic dispute.

First, the early triumphalism against the New York Times in the name of both Telsa and data transparency, were clearly premature. In Tesla’s grand rebuttal, Musk clearly overplayed his rhetorical hand by arguing that the review was faked, but he also overstated both the case he could make with the data, as well as the level of transparency that he was actually providing. Tesla didn’t release the data from the review. Telsa released their interpretation of the data from the review. This interpretation took the form of the graphical representation they choose to give it, as well as the subjective write-up they imposed on it.

What is interesting is that even with this limited and selective data release (ie, without the raw data), entirely different narrative interpretations could be built. Broder and his New York Times team presented one. But Rebecca Greenfield at the Atlantic  provided an even more detailed one. There are likely elements of truth scattered across these three interpretations of the data.  But they are just that - interpretations.

Second, the only person who can provide the needed context to this data is Broder, the reviewer himself. And the only way he can convey this information is if we trust him. Because of his “problems with precision and judgement,” as the New York Times’ Public Editor Margaret Sullivan put it, his trust was devalued. So the missing journalistic piece to this story is lost. Even in a world of data journalism, trust, integrity and journalistic process still matter. In fact, they matter all the more.

Finally, we can’t lose sight of the outcome Tesla wanted from this. They wanted PR for their new vehicle. So amongst all of the righteous indignation, it is worth noting that journalistic principles are not their core objective – good stories about their products are. These may or may not be aligned. This is why, for example, Broder was given significant support and access during his review trip (some of which ultimately proved to be misguided).

An example of this discrepancy surrounds the one clear reality about the Model S (and presumably electric cars in general) that was revealed in the review – they lose significant charge when not plugged in during cold weather. Now, Tesla would rather this fact had not emerged in the review. But it did. And as Steven Johnson pointed out, this has significant implications, specifically for city drivers. For one, it makes parking the Tesla S on the street in the winter (what many urban dwellers would have to do), largely impractical.

So, to recap. The Tesla Affair reinforces that: data does not equal fact; that context matters enormously to data journalism; that trust and documentation are even more important in a world of data journalism; and that companies will continue to prioritize positive PR over good journalism in reviews of their products.

15 Comments

cheap_cialis Sep 16, 2014
coach factory outlet Jun 16, 2014
michael kors outlet louis vuitton outlet coach outlet cheap oakley sunglasses oakley sunglasses outlet louis vuitton handbags coach factory outlet coach outlet store online coach factory michael kors outlet online coach outlet prada outlet air jordan true religion jeans coach factory outlet louis vuitton outlet coach factory outlet oakley sunglasses michael kors coach factory outlet prada handbags coach outlet louis vuitton handbags cheap oakley sunglasses cheap jordan true religion jeans outlet louis vuitton coach factory coach store online coach factory online michael kors outlet coachfactory.com coach factory outlet prada handbags michael kors outlet coach outlet store online coach outlet valentino valentino shoes louis vuitton outlet coach outlet online coach factory outlet online isabel marant Shoes coach outlet store online timberland boots coach outlet oakley sunglasses coach outlet online coachfactory.com true religion outlet coach outlet coach outlet store online coach factory coach factory outlet timberland shoes true religion outlet coach outlet store online coach outlet coach factory coach factory outlet true religion coach outlet store online true religion jeans coach factory louis vuitton coach factory outlet michael kors outlet coach factory outlet polo ralph lauren coach outlet store online tiffany and co coach factory outlet coach outlet online michael kors outlet online coach outlet online coach factory outlet online celine outlet cheap toms coach outlet celine handbags marc jacobs outlet michael kors purses coach outlet ralph lauren coach factory outlet toms shoes coach outlet online supra shoes coach outlet fitflop outlet tiffany jewelry tiffany and co coach factory michael kors outlet online coach outlet michael kors outlet store marc jacobs celine bags coach factory outlet coach handbags outlet timberland boots michael kors factory outlet fitflops coach factory outlet toms shoes louis vuitton handbags oakley sunglasses cheap coach outlet online
Jerry Feb 27, 2013
Look at the 'raw data' from the Tesla test and and decide for yourself: http://www.duckware.com/blog/tesla-elon-musk-nytimes-john-broder-feud/index.html
Les Elkind Feb 25, 2013
Any comments on the relevance of Broder's known prior opinion regarding electric cars to the way he conducted the test drive? In the grey area between intent and innocence observing the action by looking at data, as you point out, offers little insight.
Phil Simon Feb 24, 2013
Interesting piece. To me, the key is this: Tesla didn’t release the data from the review. Telsa released their interpretation of the data from the review. Big difference. Brooks is right. Data always must be viewed in context--and Big Data is no exception. And Twain's quote about damned lies and statistics still holds.
dsm363 Feb 24, 2013
You are basing this on Broder telling the truth. Tesla denies telling him to take off early. Was calling the review 'fake' harsh? Maybe but there are several false statements in Broder's original article that aren't supposed by the data. Data must always be interpreted by humans so not sure why you think that is a problem. As long as the data wasn't fake though it represents what actually happened. You don't mention that no where during the entire trip did Broder bother to fully charge the car making his review of the Supercharging network and the range on the Model S worthless. He also left his last charger after only 32 miles of range needing to go 65 miles but you don't take him to task on that. Tesla would never tell a NY Times reporter to take off on a trip with 1/2 the range they needed in the cold for fear they would run out (which is what happened). Broder either lied or misheard them.
Susanne Feb 23, 2013
Your comments sem to be broken: when you click Post this Comment, the comment is wiped out but never posted,
Peter Feb 23, 2013
Sorry Taylor but you have missed several very important pieces of information/data that has come to light since the publication of the original articles. You write about the over half a mile of driving in circles around the Superchargers saying 'we do not know from these data alone what happened in that parking lot'. You claim that it is not possible to tell if Broder's endless circling was to deliberately try to run down the battery or because the chargers were hard to see in a dark parking lot. You needed to research all the additional data that has become available since the original stories. If you Google the follow-up trip 7 Tesla drivers made with no problems (same for CNN and CNBC) a video would be found that clearly shows that the Supercharger parking lot is actually brightly lit - not dimly lit as Broder claimed - making the Chargers virtually impossible to mess the first time past let alone at least six times past. This video, which is been viewed over12,000 times, seems to have convinced virtually everyone that Broder was clearly lying (check out the comments) and was desperately trying to flatline the car. Please remember, The New York Times, in fact, does not dispute the car computer data itself at all and it was one of the reasons they admit their story was full of errors and deeply flawed. Once you take this into consideration, along with the bright parking lot video and the fact that Broder only charged to 30 miles knowing he had to travel 60, only a very stubborn few would come to the conclusion that there wasn't enough data to reach a reasonable conclusion. Finally Tesla has never tried, in any way, to keep secret that the car may be affected by the cold, but because of the very clever design of the Tesla battery, the effect is actually quite minimal because it is liquidly temperature controlled. In fact Teslas have been working fine and are very popular in extremely cold climates in places like Colorado, Canada, Sweden and Norway for over three years. Again, a bit more research would have uncovered this factual data. Because of a successful trip of the seven Tesla drivers CNN and CNBC there is now plenty of data out there that proves that the Supercharger trip is very easy to do driving above the speed limit with the heat in the 70s and you will still arrive in a brightly lit parking lot with over 30 miles to spare.
Mankel Feb 23, 2013
Why "principle" as "principal", WHY?

Post a comment

We're trying to advance the conversation, and we trust that you will, too. We'd rather not moderate, but we will remove any comments that are blatantly inflammatory or inappropriate. Let it fly, but keep it clean. Thanks.