Jan. 03, 2012

Page one discussion: metrics behind the wheel?

Recently, a twitter conversation sprang up between Kevin Schaul from the Minnesota Daily, Andy Boyle from the Boston Globe and myself from Voice of America.  It was about page one decision making - how does a front page story get picked?

You should first read Kevin's take: http://www.kevinschaul.com/2012/01/03/editorial-arrogance/


Perhaps I misread some points there and in the conversation. Tweets are optimized for brevity and don't allow for many details or nuances to show up in conversations - but - this happens to be a worthy rational conversation I owed a response to. Perhaps Kevin is playing a devil's advocate on some tangent and will forgive me for any misunderstanding on my part. Should I find myself in his vicinity, I will be honored to buy the first round of drinks.


However, I have to confess that I was confused that Kevin echoed some print and legacy product sentiments that I've run into over the years- sort of a blind distaste for any kind of measurement feedback or criticism, and refusal to acknowledge that online operations have evolved beyond print and in some ways leapfrogged it.

Not that Kevin did it, but this sentiment is usually manifested by sneering at metrics.  But do these people really understand metrics?


My long writeup here was meant to address specific details and draw attention away from oversimplification. Rather than initially attack this conversation head on, I just figured I'd point out a general complexity of web operations:

Quick thought re: Page One - exec. editors meet about what goes on front page for print, why isn't the same editorial judgement used online?
Kevin Schaul
@ Crux of page 1 question is that print and web are behaviorally different products, not just different aspects of same.

However, in resuming this conversation over the following days I took three exceptions to Kevin's points:

  1. His conclusion that web is or should be treated exactly same as print,
  2. inference that no editorial judgement is used online,
  3. assumption that metrics are an egg - and not the chicken.

Kevin's conclusion is to dismiss metrics altogether and have web reflect print process and decisions. This is wrong because it oversimplifies a complex environment in order to make it more convenient to make decisions, and then some. Why dismiss additional information?


Metrics were always here

Before anyone dismisses web metrics, they need to first understand that it's not just a number.  Secondly, that metrics as business research have always gone into the editorial process.

In case you're not buying my point, these above-mentioned "old school metrics" are implemented very slowly, but they do get implemented in editorial policy. It's slow to see, but if you doubt this, revisit how employees are hired, fired, a beat is formed, removed, resources shifted to/from it, or how assignment desks budget items. Yet, unlike sometimes requiring access to the publisher's inner circle, web metrics access is given to virtually anyone.

Whether they use it appropriately is up to them.

To summarily dismiss online metrics without understanding them is akin to throwing away letters to the editor, feedback polls, unplugging metro desk phones, closing down every newspaper's business intelligence office and circulation research. And yes, all of those components go into editorial decision making.


Just as we're a watchdog for readers, readers are watchdogs of our shortfallings.

Metrics can be seen as a measurement, but not only a measurement of success. It's also a measurement of readers' interest, approval and disapproval, and - sometimes - measurement of a failure to do your job. If a newsworthy topic gets adequate coverage but no reach, did you do best that you could to illuminate the issue?

Metrics, a near-realtime aspect of reader feedback, will tell you that. They'll refuse to camouflage your article under blanket readership numbers. They will crush your ego when you least expect it.

Sometimes they'll tell you things you don't want to hear: that citizens may not care about an issue.

But, to blindly dismiss web metrics as a component of an editorial decision process or to assume that an editor would simply pick the best performing topic or article is doing disservice to all of our readers.


Metrics are not numbers. They're patterns.

My only actual point to the conversation is that editors need to be informed. But I was abstract as usual and never explained – informed of what? And why? And when? And how? I had assumed everyone here was talking about observing a very complex system and not just biggest numbers.

Entirely missing from our twitter conversation is how do you use metrics - either appropriately or inappropriately? Since no methodology was cited here, Kevin was presumably referring to raw top numbers - max or aggregate pageviews, or visits.

Doing so conflates multiple dimensions into a single number. If you are covering the greater Baltimore area, you may want to filter out any out-of-state traffic from your metrics report and see what it looks like then. Once you subtract massive drivers like reddit, fark and the Drudge Report, it may be that not many affected citizens discovered the lead poisoning story you thought was performing well. Try again.

Or, if you filter out police and various city department IP address ranges, you may discover that no one else in the state noticed hundreds of city employees making half or more of their salary in overtime. [Shameless database plug, made by yours truly back in '10]


Metrics are also a passive intelligence gathering tool

Metrics told us when Libya unplugged from the internet.  Metrics told us when Google Plus launched, and the Chinese government failed to preemptively block IPV6 traffic to it - and as a result - we got web traffic from China for a time.

The excitement of knowing that we broke through the great firewall of China and its layers of censorship, even by accident, was rewarding and we reacted accordingly to our new traffic spike having taken appropriate advantages of it.

Metrics have also done lesser things for us - such as alert us to adjust staff schedules.  Since we're phase shifted from our Asian audiences, we at times need greater coverage at night (sweet mercy, cricket stories are popular elsewhere during parts of the year!)


Kevin: "why isn't the same editorial judgement used online?"

But editorial judgement is used, just not the same exact kind as with print. His statement partly implies that online is lacking judgment at all - which I'm not sure is what he intended to say.

Similar kind of editorial judgement is applied by honed specialists, though it may not be as visually evident.

In a proper online operation, the product moves a lot faster than in print or radio/tv. There is no luxury of time to have formal meetings in big mahogany furniture conference rooms because there is no arbitrary cutoff point of 9-10 pm for all news to make it into print the next day. There is no 7 o'clock story deadline for writers to be copy-edited.

Collaborative discussions do happen, but they happen faster. As I recall from my Baltimore Sun days, my colleagues had their cubicle partitions lowered so they can discuss while editing.


Print IS specialized

Online departments were doing more-with-less since day one, having traditionally been given limited resources.  Yet when it came to necessary downsizings, they at times took bigger layoff hits than their print counterparts, while not getting equivalent credit for their successes.

Part of this narrative stands up for specialized craft methodologies that are being dismissed out of hand.

My old boss would hire polyglots: production assistants who could copy-edit, copy editors who could use the CMS, section editors who could copy-edit and produce in a pinch.  There were also incredibly creative and competent features specialists who understood competing markets and could sniff out a reader interest void.  All of these people knew photoshop shortcuts better than I do now.

He also promoted a wizard to a deputy managing editor; a person who had a really good feel for readership dynamics. Though that guy was metrics savvy, he only used it to measure success (else all of our pages would have lolcats.)  He exercised nothing but sound editorial judgment first and foremost. It's just that he didn't permit apathy and slackery and for that - you needed a measuring stick.


Measure of success

Under this team, pageviews steadily grew from zero in the Sun's online nascence to about 52 million / month as of last August. That's 52 million pageviews, 4.8 million unique visits, for a print readership of about 340 thousand.


Let that sink in for a second in light of general media trends, as reported by the FCC:

Blue = print, red = online. Print is different than web, so why apply identical decision making to it?


These trends show that print is declining, online is on the rise but is unappreciated despite the fact it overshadows local circulation numbers. Advertising amounts have not adequately shifted from print to web and there are some decision makers that panic. At some point, print will inevitably approach closer to the zero point of its asymptote.

It's a lot easier to form decisions on 300k audience than it is for 4,800 k readers. Days of outsourcing what should be first hand intelligence are over. You need to be able to slice and dice the data onion yourself and in multiple ways - and that's where metrics come in.

But that's not why I made these graphs: it's to show that a transition is in fact happening. If you don't adapt your editorial process to match or meet this transition - and it will need adaptation - you will not survive the modern era and you will find it very difficult to practice journalism.

To say that my old coworkers had successfully diverged from print and became specialized is an understatement. Having witnessed a tail end of this process, I have to reject Kevin projecting a print mindset onto what they accomplished.


Parting thought re: page one.

For some 20-30,000 articles a typical newspaper might publish yearly, they can only feature 365 of them prominently on the front page, for a whole day. And they have to do it for the whole day.

Online presence does not have this limitation and does not have similar constraints. For that difference alone, you might want to reconsider giving online operations the same exact print editorial treatment.


Comments? Thoughts? Crickets?



Find this interesting, or useful? Consider sharing the post.

2 responses to “Page one discussion: metrics behind the wheel?”

  1. [...] TweetUpdate: Edin Beslagic wrote a smart reply here. [...]

  2. Kevin Schaul says:

    Lot's of really great stuff here.

    I'd like to point out that I am certainly not dismissing metrics altogether; they're an invaluable resource for any web editor. My point is I believe they should not replace editorial judgment, but take a more backseat role.

    Also, it's hard to measure success, and while I'm sure your site is a journalistically solid endeavor, I don't believe page views are determinant of success. With online subscriptions becoming inevitable (at least, for now), I see this as short-sighted. BosonGlobe.com cares more about subscriptions than page views, and so likely slides the scale closer to editorial judgment than metrics. Certainly metrics are used to keep subscribers happy and to gain them in the first place, but I'd doubt they trump editorial judgment.

    Andy made a great comment on my blog, though a bit sidetracked: "Focus on the strengths of the two mediums. Make something that requires some user engagement for the web and make something that’s damn awesome to read in print."

    I think the overarching idea is that we have editorial judgment, and we have metrics, and they tend to differ. Which should we trust more?

    It's great to have this discussion.

Leave a Reply

Your email address will not be published. Required fields are marked *

Posts on this blog solely represent my personal opinions and technical experience.

© 2009-2019 Edin (Dino) Beslagic