This is the first in the series of two posts about pageviews. This post will deal with some of theoretical baggage tied up in the metric, while the second will detail some research I’ve conducted on the correlates of pageviews. A year ago today, Aron Pilhofer, head of Interactive News at the New York Times, wrote a blog post that changed my life. In it he reflected on the impoverished status of newsroom analytics, soberly claiming:
”…the benchmarks we use now are so ill suited. They are the simplistic, one-dimensional metrics we all know: pageviews, time on site, uniques. We use them largely because they are there and because they are easy”
The lack of suitable metrics for measuring impact, he argued, was the key to journalism’s survival in a digital environment and the perfect issue for a Mozilla-Knight OpenNews Fellow to address over a yearlong fellowship.
At the time I read this I was a grad student struggling through the process of translating my questions into maths and code. While I had completed some cool projects, it had been three months since I had copy-and-pasted my way though a 5000 line script because I was scared of SQL databases and six since I first opened that ‘scary program called Terminal’ on my MacBook.
So when I read Aron’s eventual pitch - “If you’re an analytics nerd, a news junkie and think it would be neat to spend some time using The New York Times newsroom as your laboratory, we’d like to hear from you” - I was both thrilled and horrified. How could I - neither a hack nor a hacker - compete with the plethora of geniuses that would no doubt apply for such an irresistible position. Like Noah Veltman, my remarkable friend and ‘fellow fellow’, “I had a serious case of imposter syndrome”. This deep sense of self doubt (and perhaps a little procastination) led me to mull over the application to literally the last second.
Since then, I’ve undergone a transformation that is no less than miraculous. In my five-plus months as a fellow I’ve dove deep into the technical and intellectual challenges of impact measurement, reading as much as I could find on the topic, experimenting with the creation of metrics for News Apps, speaking at conferences, and conversing with the brightest minds in the field. I have been continually humbled at the many people working on this problem for no other reason than they think it’s the right thing to do. I’ve also found support in the many innovators and brainiacs I work with at the New York Times and the seven incredible people I’ve shared this journey with.
In this time, I’ve gone from a novice coder with some knowledge of stats to someone who regularly writes map-reduce jobs over terrabytes of data (trust me, if you’re a data nerd, the New York Times is your perverse playground). The freedom of the fellowship has also allowed me to pursue more whimsical projects like building haikubots, experimeting with data sonification, and writing oh-so-many twitter trolls. I’ve also had the priveledge of working with my friends in csv soundsystem to build treasury.io - a daily data feed for the U.S. Treasury.
And after all of this I can say that while my initial fears of technical incompetency weren’t completely unfounded, I was perhaps afraid of the wrong things. To return to the question that launched this crazy adventure in the first place - “what if we measured journalism by its impact?” - I’d be remiss to not complicate the current conceptualization of the problem. This perspective has been deeply informed by my interactions with James Robinson, my friend and mentor at the Times who, in sharing his vast experience in news analytics, has often served as my version of Gene Wilder in Charlie and the Chocolate Factory.
What I think we’ve learned in our experimentation with news metrics is that, more than anything, our work is less about programming than it is about proselityzing. The challenge of changing the approach to metrics in the news room is nothing less than that of sparking social and cultural change. The question we must be asking, then, is the painfully meta one of “how do we measure the impact of impact measurement?” If the tools, methodologies, and metrics we develop are difficult to use, implement, or understand, then the journalists and editors we’re trying to influence will fall back on their well-honed instincts. The key is not to prove whether a story or news organization has ‘made an impact’ but to help journalists make data driven decisions that resonate with their broader goals. This challenge, I think, is more anthropological than statistical, more collaborative than code-based.
So while I think I’ve done alot to tackle the difficult set of questions I’ve been tasked with, in many ways I’ve failed miserably. In my final five months I hope I can do more to build and write things that help more people. But there’ll always be more work. So if you know a bit of code or maths and think insights trump data, then apply to become a 2014 OpenNews Fellow and pick up where I and my other fellows have left off.