Measuring conversation is Difficult;
a talk with KD is Priceless
[KDPaine with admirer at Measurement Summit. File Photo]
In the interest of transparency I need to tell you that KD Paine is my friend. She has been my friend for a very long time. We have both hung out in the tech-journalism-marketing-consulting communities for several decades. We both recall when once-weekly tabloids were fast enough and the state of personal communications art was the FAX machine.
I have always admired KD for her temerity, her ability to pick herself up, dust herself off and just get on with life, even when it deals you a blow. She is a breast cancer survivor who started running marathons to raise money to defeat the beast. When her 100-year-old family farmhouse burned down on a cold New Hampshire night, she built a new one that was true to the spirit of the cinders that had housed her parents and grandparents but embraced the comforts of modern times, including a kitchen that can easily feed 100 people, which happens with regularity. When I spoke at the Measurement Summit that she founded, my wife and I were guests in her home.
She would have none of our talk about "Oh, we'll-just-get-a-hotel room." She gave us a tour of her amazing acreage and I discovered it really was a farm. She showed us her bulldozer, the cellar where she preserves her jellied fruit and the river house, where she wrote her latest book, "Measuring Public Relationships" and has already started on another.
This makes it difficult for me to formally interview KD. As the picture above shows, our relationship is up close and personal, but the issue of measurement is among the most complex flex points of social media and business right now. KD is the undisputed expert on this topic, and to NOT interview KD Paine because I know her too well, would be to a disservice to the readers of this survey.
And now, I give you KD Paine.
1. You have been called the Measurement Queen. Just what is that you measure and for whom do you do the measuring?
First of all, I prefer Goddess to Queen, since Queen implies a command and control society and Goddesses are typically credited with giving birth to belief systems and are more inspirational. Besides, there really is a Goddess of all things Measurement named Seshat.
What do we measure?
We measure the impact that media has on the reputation, positioning, messaging, relationships and business interests of our clients. By media we mean everything from Fortune to Salon to Global Neighbourhoods to BizRate to YouTube and Facebook. Typically we look at competitors or peer organizations so the clients understand the relative importance of that impact.
About half our clients are nonprofit and government organizations such as the ASPCA , the American Society of Plastic Surgeons, Georgia Tech and Bonneville Power. The other half are high tech, business to business and business to consumer companies like Facebook, Epson, Raytheon and Georgia Pacific. Our clients typically have titles like VP of Communications, VP of External Affairs, VP of Public or Media Relations.
2. Measurement is one of the enterprise flex point issues when it comes to social media these days. Do you get asked about such issues as the ROI of a blog? How do you answer that question?
Yes, we get asked that all the time. The problem is that the “R” of ROI really depends on the organization. For the ASPCA the ROI can be measured in how many new members are engaged in the organization because of the blog, and ultimately the “R” is the number of new members as well as the amount of new contributions.
For Ingres or SAS, they may or may not be able to draw a straight line from their blog to the sale of a $100,000 software system, but they can certainly track the proliferation of ideas and concepts from the blog into social media and perhaps into mainstream media or directly to customers who subscribe to the blog.
For high tech companies and advocacy groups, I’d argue that the ROI doesn’t matter. Not having a conversation with your customers is no longer an option. It’s like an email or the fax machine. At some point you realize you no longer care about the ROI, you simply can’t do business without it.
For an internal blog the “R” might be lower recruitment costs or reduced turnover or higher morale. It depends on the purpose or goal of the blog. The reality, in far too many cases, is that the reason the organization started the blog was because the CEO or a board member said they had to do it. There are no clear objectives, so the demand to measure ROI becomes a political football tossed back and forth between proponents of social media and the old-line command and control types that once said the internet was a fad that would quickly pass.
3. Media is all about the conversation, which I imagine makes it more difficult to measure than say, a press release. How do you measure social media?
You know the old adage “Measure twice, cut once?” I think today you have to listen a lot, and measure once. It’s all about listening to what your constituencies have to say.
If you video taped a real conversation, you’d capture body language, you’d know who the players were talking to, and who they were going to talk to next. Computers are only beginning to do that. They can count the people in the room, but they have no way of knowing what those people are going to do next. What is a complete waste of time are those organizations that are still counting eyeballs and using text mining to just measure the words used.
To evaluate social media conversations, you listen. You hear the tone. You note what subjects are being discussed, what words are used, what battles are being fought, who is saying what to whom. The only thing you’re missing is the body language, and the social media equivalent of body language is the avatar or photos you use to identify yourself.
What is most interesting to me is that measuring social media draws on research methodologies that are decades old. Basic sociological and psychological research techniques that have been around for decades are being dusted off and put to work measuring social media.
Jim and Laurie Grunig defined how you measure relationships a decade ago. It’s just been easier to measure column inches and impressions than to actually figure out what people think about you as a result of your actions and words. However today, there are no “column inches and impressions” for social media. There are just conversations and relationships. So we are going back to measuring people rather than ink and bits and bytes.
What technology has brought us is the ability to track not just this conversation, but the next conversation and the next. Social mapping systems like BuzzLogic enable you to follow the conversation wherever it goes. That’s the really exciting piece of measuring social media. We’ve been able to evaluate one conversation (think of focus groups) but we’ve never had the tools to be able to track what happens after the participants leave the room. Now we have that technology. It used to be that we measured reach and frequency. We are still measuring frequency, but social network mapping is the new reach.
The biggest problem are all those people out there that are trying to measure social media with tools that were invented in the 1940s to measure TV – panels and eyeball counting mechanisms that are meaningless to today’s consumer.
4. Can you give me a good case study of how you've helped a company evaluate a social media program?
Before there was social media, there was consumer generated media and before that there were newsgroups. We began doing competitive analysis in newsgroups for a leading printer manufacturer (not HP) back in 1995.
At the time, we learned that even though the client was worried about negative newsgroup discussion winding up in mainstream media, the information was, in fact, flowing the other way. After PR launched a product, customers were picking it up and discussing it in newsgroups about two weeks later. This allowed them to determine which messages were actually being heard by their customers. A decade later, our measurement program for them now includes blogs, structured review sites like Amazon as well as traditional media and online sites like Engadget.
We compare the impact that all these forms of outreach have on purchase patterns and what we’re learning is that the greatest influence comes from customers talking to other potential customers in Amazon and BizRate reviews.
We also do extensive research into the impact of social media for Georgia Tech. We look at the discussion about Georgia Tech as well as nearly a dozen other major research universities and determine what’s being discussed, and more importantly, what’s being shared. We look at what Georgia Tech bloggers are saying and what others are saying about Georgia Tech. We also examine the degree to which people are bookmarking and sharing information, so we can identify hot button areas that lend themselves to advancing Georgia Tech’s reputation. Ultimately we’ll be looking at the impact that it all has on applications and requests for information.
For the ASPCA we measure traditional as well as social media and analyze it to determine which topics and which media are having the greatest impact on memberships and donations.
For a major computer manufacturer we did a social media analysis of their bloggers vs. the competitor’s bloggers. What we found was that their bloggers weren’t blogging as frequently as the competition, nor were people commenting as often on their blogs. So they revamped their internal blogging policy as a result.
5. How would you assess the current crop of social media tools. What do you recommend for an individual to use? Is there a different set of tools for an enterprise? How do you see them evolving?
We’ve gotten very good at teaching computers to understand words, the problems is that they don’t understand the nuances of conversations. Computers still can’t tell the difference between sarcasm and irony. And throw in slang and you have an even bigger problem. So computers are good at categorizing conversations as to what they are primarily about, or where they’re appearing, but as to the real impact of the conversation, it still needs a human.
Personally, I use Google Analytics to track the success of my own blog and IceRocket and Sphere to see how well I’m getting my messages out there. We use Compete to determine “reach” Then I put the results into our own DIY Dashboard so I can keep track of trends over time. For my clients we use a variety of capture tools like Critical Mention, Cyberalert and BuzzLogic. depending on the nature and market that the client is trying to measure.
Far more important is the tool that you use to measure relationships. The standard methodology is to present your stakeholders with a series of statements and ask them to what degree they agree or disagree.
This can be accomplished via free resources like SurveyMoney and Zoomerang or even by phone. In an enterprise where you’ve got a significant number of different elements to track and run correlations, you really need some sort of sophisticated database tools like SPSS or SAS. For an enterprise to truly measure its reputation in social media you need a solid mix of human analysts and interpretation with very sophisticated tools. Facebook, HP and Microsoft did extensive research before selecting measurement tools and all three insisted on human analysts. So I ask you if some of the leading players in technology don’t trust computers, why should you?
6. Why do you think an enterprise should allow employees to blog or engage in social networking sites such as Twitter or Facebook?
Because it will make them smarter and it increases social capital. Robert Putnam in Bowling Alone credited World War II and the draft for boosting social capital by forcing people to interact outside of their comfort zone. Facebook and Twitter are today’s version of the draft and life in the trenches. They enable you to see and hear different points of view and perspectives from around the world. It brings you different perspectives – instantly. Besides in Facebook, the ability to ask questions and share ideas is a great way to listen to your customers.
7. I've written that blogging helps companies get closer to customers. How do you measure something like that?
Don’t ask me, ask your customers.
Do a survey and ask them how they feel about your brand or organization. Ask them whether they agree or disagree with a statement like “This is an organization that listens to people like me” or “this is an organization that likes to throw its weight around." Tally up the answers and you get a measure of the health of your relationships. Ultimately, do they feel closer or more alienated? You’ll only know if you ask.
8. Corporate reputation often comes up as one of the issues related to social media and the enterprise. How do you see social media impacting an enterprise that engages in it?
If you accept the premise that engaging in social media enables you to listen more closely to your customers, then it can only improve your relationships and ultimately your reputation. Blogging and social media imply, assuming you do it right, and that you are open and willing to listen. If society believes that you are open and transparent and authentic, research shows that more people will trust you. While trust is only one component of a reputation, I would argue it is one of the most important. Secondarily, depending on what the enterprise is blogging about, social media can help convey other reputation attributes like innovation, good corporate citizenship and community concern.
9. How do you think social media impacts brand?
That depends on what you’re trying to convey with your brand and how you implement your social media strategy. Does being in SecondLife or on Facebook help Coca Cola’s brand? Yes, it conveys that they’re hip and cool. Do they sell more Coke because of it? Probably not, but simply being there says something about their brand. Did being first in Social Media help John Edward’s brand as the first candidate to Twitter? It might have had he actually been doing it, and not faking it. But because it became quickly apparent that it was Joe Trippi twittering and hired bloggers blogging for him, people stopped caring.
It also depends on who your customers are. If your customers are non-techie types raised in the 50s and 60s, social media will have a lot less impact than if your customers are Gen Y-ers. Take a company like Lockheed Martin or Northrup Grumman. It seems inconceivable that they’d blog, since arms dealers probably don’t do a lot with social media. But if they’re trying to hire the best and brightest of today’s college students, participating in social media would do wonders for their brand.
10 Additional comments.
Measuring social media all comes down to measuring relationships. There are far too many media options and far too many messages to try to track what’s being sent out there. If people instead focused on what’s being received, and the impact that has on their constituencies, they’d know a lot more.
--
Shel Israel
writer. consultant. nice guy.
http://globalneighbourhoods.net
650 430 4042
