What Are We Measuring When We Measure Journalism?
The first question asked is rarely the most useful; the last question forces you to find an answer.
When we started to discuss the many aspects of measuring success in journalism and the challenges presented with currently available analytics solutions and industry assumptions about measuring, all questions eventually lead up to “what exactly are we measuring?”
No matter how you measure, how you analyze and how you convey your insights, you need a subject of analysis, and “journalism” is as amorphous a topic as you could ask for.
In order to make any progress we need to define what we want to understand; figuring out those boundaries creates clarity and focus for analysis but also helps surface the qualities of the subject, which additionally hint to what aspects might lend itself to measurement or might be most revealing in understanding it.
Journalism is a process (how it comes about) as well as output (what it produces) as well as implications (the ripples it creates in the world). Those are three very different aspecs and each can be explored fully on its own.
For the work we have been doing on Carebot we chose to focus on the output as the subject of analysis. Journalism as process is endlessly interesting and affects output, and the ripples it creates in the world are vast and affected by the output, but ultimately we wanted to understand the questions about what happens when journalism meets a human and how they respond to this journalism. AKA, do they care?
But what is the output? (Note: We narrowed our exploration to the kind of journalism done today and primarily delivered on the Internet - see footnote). No matter the nature of the work —unexpected events that lead to breking news, hidden mysteries in society that investigative reporting digs up, explanations of contemporary issues that topic-specific desks analyze and elucidate—all journalism is conveyed through storytelling.
Storytelling allows us to focus on something more tangible than journalism, but it’s still not sufficiently concrete. We can also see process, output and implications in storytelling, so what about storytelling? Again, the answer for our purpose is its output: a story. Story as an artifact, the thing created by the process of doing journalism, and in turn the process of storytelling.
With story we find something more concrete and identifiable. Yes, there is still a lot of variety in how a story may manifest in the world, but we can at least understand what that means without much ambiguity and classify those differences, making story a very viable unit for our analysis.
A story has shape, content, duration, rythym, tone, and so much more. A story on the Internet has very specific parameters, we can’t just talk about it abstractly or conceptualy. We know when it starts and when it ends. We know what technology was employed to build, publish and deliver it. The nature of the medium also offers built-in information about the story (metadata on the story itself and data on its use). Story is an ideal unit from that perspective.
So, is a news feature a story? Yes. Is a photo essay a story? Yes. Is a graphic a story? Yes. Is a photo a story? Yes. An illustration? Yes. A cartoon? Yes. Is a news feature with an interactive graphic, photos, embedded YouTube videos and SoundCloud audio also a story? You bet it is.
There are obvious challenges in figuring out an appropriate way to categorize stories so we can make sure we are comparing apples to apples (more on that next time), but the point with using “story” as the basis for the work is identifying a concrete, tangible, minute and distinguishible unit of analysis so our work can be precise, repeatable and understandable.
So, when we talk about measuring journalism, we mean measuring aspects of the stories that represent journalism.
Footnote
Focusing on things delivered via the Internet may seem narrow at first, but consider this means any and all media that is delivered through digital means. Radio and TV may not be included at first, but any story that originates there also eventually find its way online, whether for redistribution, repurposing or archival purposes.