Measuring content strategy: Not a piece of cake
I was recently approached by a member of my team to think through a content issue. We are redesigning a major experience on our website, and the “sacred cows” were ripe for tipping. Did I wish to tip? Why yes, yes I did, thank you.
It’s a minor change. Minuscule. But I was positive it would make the user experience better. It would change the tone for this particular experience from robotic to something more human, and “human” is among our core values. It wouldn’t necessarily make the content easier to understand, but it would certainly bring it more in line with our brand.
But when I started looking into it, it was mentioned the change might not be worth the effort. It was reasoned this change would be a lot of work to ferret out every time it appeared on our sprawling website, and may require reengineering and redesigning.
In essence, even if it was the right thing to do, since there was no way to measure the effect of the new content in terms of conversions, it wasn’t really worth doing.
And this, my friends, made me sad.
At my company (and I’m sure at lots of other companies) changes in content, interaction design, and visual design are usually launched at the same time. Since we rarely launch a content change on its own, we are left to ponder how, exactly, do we measure for our content? It’s like baking an awesome cake and asking the diner, “So, how did that particular brand of flour affect your overall cake-eating experience?”
So what do we do?
I suggest we don’t measure content in terms of conversions as the only measurement. I propose we measure our content against ourselves. Our brand. What we want to accomplish. What we want our company to sound like. Feel like. Taste like (you know, if your company actually bakes cakes or something). We need to recognize content is the voice of our brand, and we have to take responsibility for what we say to our customers and how we say it. I want us to ask ourselves not, “Will this new content make us more money?” but, “Is this new content right for our brand?”
I understand we’re all looking for ways to cut costs, and since content isn’t as visible as visual design, it would be easy to dismiss it. After all, if we can’t always directly measure how much money we’re making off our content, why bother?
Because I think it matters now more than ever.
Just as companies are evaluating how every dollar is spent, so are our customers. Our websites are perhaps our only opportunity to show prospective customers exactly who we are and what we can offer them. Now is not the time to neglect content. Now is the time to make sure that every word is relevant, influential, and enticing. If a customer lands on a website and leaves because the content was irrelevant or unprofessional, we won’t have the chance to say, “Hey! Come back! That’s not really us. We just threw some stuff up there until we have money for good content.” That person is gone, baby. And we may never know it.
So fight the good fight at your company. I’m not sure how my situation will turn out, but I’ll let you know. In the meantime, I’m having a huge piece of cake.
9 Comments
R. Stephen Gracey on June 5th, 2009
Fantastic article. It’s making me wonder how to categorize content by contribution to overall value of the experience. “Filler” needs to be minimized, and it’s easy to pick out among the content treasures, but how can we evaluate the contribution of other content to the experience? Thanks!
Kristina Halvorson on June 5th, 2009
Terrific article!
While I agree with your impassioned call to action, however, my opinion is that we’re a long ways off from this way of thinking. Qualitative measurements are nice to have, but they’re nearly impossible to identify how they directly affect the bottom line.
Yes, fixing our crappy, robotic web content provides a better customer experience. No, we can’t really say how much money it’s going to make us.
To that end, I think content strategists need to start with the metrics. It’s the only way we’re going to grab the attention of the people holding the purse strings.
I get your “how did that brand of flour affect your cake enjoyment” simile, and that’s a challenge. Baby steps, here. Sometimes there *are* copy-only changes. How can we measure them? Even little numbers add up over time.
simon kelly on June 5th, 2009
A good start. We have developed a tool called the storytelling index that not only measures the impact that content has on brand resonance, but more importantly on activity as a result of the content experience. All marketers are going to want to know what happened as a result of the content program and all content marketers should be prepared to be able to answer the question.
leenjones on June 10th, 2009
Excellent food for thought!
I think that content strategists need to get more involved in the full research and testing lifecycle. That way, content is evaluated all the way through the design process, not just as a web analytic. For instance, early interviews with users should talk about content. Usability testing need to include questions and evaluations about content. And so on. Addressing content this way gives us a greater set of “data” than web analytics to support content efforts.
I also like to milk the opportunities to isolate content as the variable affecting conversions. For instance, IHG tested the same Holiday Inn design with “old” and “new and improved” content. Conversion skyrocketed with the new and improved content. We can’t test like this in every situation, but we need to seize the times it makes sense. That way, we can say if content made a big impact here, it likely will make a big impact there.
Keri on June 10th, 2009
Thanks for the thoughts, everyone.
Agreed we need metrics to make our case, but the opportunities are maddeningly few and far between. Colleen, I would LOVE to see the Holiday Inn example and hold it up as what good content can do for conversion.
I understand the company I work for is a business, and shockingly, would like to make money. It’s just hard for me to see bad content existing on the site – no matter how subtle – that I can’t change because of a lack of metrics.
We’ll get there, and yes, baby steps are best (but very hard to do that when you want to speed skate). So let’s get those case studies out there. Let’s collect victories like Beanie Babies or Pokemon cards or whatever the kids are collecting these days.
I’ll do the same here. Now, if only I had access to a South African user experience researcher with a personal interest in good content… Hmm…
Small Business Marketing on August 3rd, 2009
Great insights. Measuring and having the metrics ready for any strategy is difficult but it is the most important to keep the focus for success.
Maggie on August 8th, 2009
That’s a great perspective on how to measure content. This is very interesting to read. Thanks for sharing.
kristengroh on August 14th, 2009
Great discussion! I like the direction leenjones was taking with the Holiday Inn example (I’d also like to see it!). We test site affect structures and functionality have on usability, so why not test the affect content has on user experience and brand perception? I have done this — measuring how a user’s perception of a brand changes based on degrees of exposure to our message, but that was on a more macro, full-site scale (sorry, I can’t share the case study).
There needs to be some version of rapid prototyping available to content strategist that allows to perform micro versions of this for smaller segments of content.
Brand perception measurements are quantitative, so you do end up with numbers, you’ll just need to have stakeholders that realize (or can be convinced) that an improvement in brand perception is important to the business’success.
ContentKeith on June 4th, 2009
Yes, I penned this, and yes, it was (purposefully) basic, but it is a start to measuring content:
http://blog.junta42.com/content_marketing_blog/2009/04/keeping-score-measuring-the-effectiveness-of-content.html
Would love to hear your thoughts.