Firm-Created Word-of-Mouth Communication: Evidence from a Field Test

Back to the Peer Reviewed Journals List

Executive Summary

The authors investigate the effectiveness of a firm proactively managing customer-to-customer communication. In particular, they are interested in proving how, if at all, a firm should go about effecting a meaningful word-of-mouth (WOM) communications program. This is done through two different data collection schemes: a large scale, 15 market test through BzzAgent with a client restaurant chain, and also through a controlled online experiment. The results are somewhat counterintuitive and may change the way web analysts and Marketers should be thinking about WOM and social analysis, particularly if there is a hard monetary investment in the WOM program.

Specially, the researchers are trying to answer 2 questions:

1. What kind of WOM maximizes incremental Sales?

The answer: WOM created by less loyal (not highly loyal) customers, and occurring between acquaintances (not friends). Though perhaps surprising, this result is often found in Marketing program measurement; Sales would occur anyway without the program, especially among best customers. Said another way, the results demonstrate the pitfalls of not using control groups (people not exposed to the campaign) to accurately measure Marketing effectiveness.

2. Which kinds of people are most effective at creating the WOM above?

The answer: “Opinion Leaders” or “Fans” are not as effective in spreading WOM that drives incremental Sales because these efforts are “preaching to the choir”, per #1 above. The networks that opinion leaders or fans have are likely to already know about the Product from pre-existing conversations, and spending money on creating a campaign to reach these people is ineffective because the social communication has already taken place.

In sum, if you want to invest in a WOM program that will drive Sales you would not have received anyway, you want the WOM conversations happening, as the authors say, “where none would have naturally occurred otherwise”.

As is typical of academic research and testing, there is an extensive review of the results of other WOM Marketing studies all the way back to the 1970s upon which the hypothesis for this test was formulated.

Review

This is a classic piece of research that is not only helpful for the evidence and results produced, but also demonstrates a great many ideas and techniques that should be employed in Marketing analysis. Some of the concepts could very well be used to bring more precise definitions and measurement practices for WOM and the social construct in general to the web analytics community.

The discussion of the difference between the need for a persuasive argument versus building awareness is something web analysts should keep in mind so they can make sure they understand the real needs of the Marketer or Product Manager. For products with high awareness already, what is really needed to increase Sales in persuasion of the people already aware, not more awareness. New products with zero awareness obviously need increased awareness.

Per this study, this persuasion versus awareness question affects the choice of who to recruit for WOM campaigns. Loyal customers are the best persuaders and are best used when the product already has high awareness. If you want to drive sales through increased awareness – the goal of many WOM campaigns online – you should be recruiting less loyal customers and encouraging them to talk not to their friends, but to their acquaintances. This approach appears to be contrary to the “opinion leader” or “fan” approach now thought of as best practices. Because of this, a lot of books on social marketing may need to be rewritten, at least as they pertain to generating incremental Sales…

At the very least, some of the discussions around tracking or proving the value of social media need to change given the results of these tests. Seems to me a test like this that is carefully executed using the scientific method is what social advocates have been dreaming of, yet the results don’t lean in the direction these folks generally support. It will be interesting to read their reactions, if any. After all, many online Marketers really don’t care if programs generate profit and are more comfortable following the ancient offline mantra of “any exposure is good”.

As a practical implementation matter, the above suggests changes to the design of many WOM programs and any incentives provided, depending on the goal of the WOM program. The most common program structure – to look for opinion leaders that have lots of “followers” - generates more sales when the product already has high awareness, a situation that requires persuasion. Yet online, this program structure is often used to introduce brand new products. The implication is perhaps this: when launching a new product, the Social programs focused on Opinion Leaders or Fans should be implemented after Advertising has created awareness to maximize their effectiveness.

The more difficult question to answer from an implementation perspective is this: if you want to generate awareness, how do you recruit less loyal customers (not fans) and have them spread the word not to their friends, but acquaintances?

One answer to this question is forced by one of the brilliant ideas in the test design. The authors used BzzAgent in perhaps an unexpected way – not as the hip, cool people who are thought leaders with lots of followers, but as a group of people who were not customers of the product at all and are simply paid to spread the word. It is this group that generated the highest incremental sales.

The opposing group – best customers in the restaurant loyalty program – did not generate as much incremental Sales activity as the BzzAgent crew. This makes sense because among the loyalty program customers, most of their good friends probably were already aware of the restaurant through casual conversation. In other words, the available audience “to be made aware for the first time” was much smaller with loyal customers.

There were some non-controllable issues in the field test that may have affected the outcome, such as different demographics between the BzzAgent people and loyal customers of the restaurant chain. This is common in field tests, but contrary to what we usually see with success stories outlined in by vendors in the online marketing space, these issues were actively researched by the authors and completely disclosed, leaving it up to the reader to decide if these issues nullified the results in any way. Extensive mathematical simulations were run and the potential for any of these issues to significantly affect results discarded.

Personally, I would have preferred to see split market testing, where not all 15 markets in the chain were involved in the test and results compared to this not-involved control group. I’d guess the nature of the work done by BzzAgent and the operating methodology of the restaurant precluded this structure for the test.

Plus, the authors counter these challenges with a very clever solution - creating an online test where they completely controlled all variables. The design of this test was ingenious, but more importantly, the results confirmed those of the field test – the effectiveness of WOM programs depend a lot on whether a product needs awareness or persuasion to drive incremental Sales, who exactly is doing the buzzing, and to whom. Once again, generating conversations where they did not already exist is the key to driving incremental Sales, and this is most likely to occur when less loyal customers spread WOM to acquaintances.

At the risk of repeating myself but understanding the results of this study will be met with a lot of skepticism, this kind of effect is seen in Marketing measurement all the time, and especially so when Marketing to best customers. So the results are not really surprising in any way; it is often difficult to drive incremental Sales when Marketing to best customers because they are highly likely to buy anyway without any Marketing effort, and interactivity simply amplifies this phenomenon. The results of this study fit right into the existing model hand and glove.

The good news for online social buzz advocates is this: online WOM can drive incremental Sales – which is no surprise, given the decades of data from offline studies – and a scientifically designed and executed study proves that. The not so good news is WOM drives incremental Sales when implemented very differently than the way most people currently approach the challenge. The question, as always with Marketing programs like this, is did you make more money that you spent? I’d venture a guess there are a lot of “preaching to the choir” programs going on in social right now that are not profitable.

As a Marketing person, this is what the research means to me. WOM will spread all by itself among Opinion Leaders and Fans, so one should be careful with investing in this area and be clear that persuasion is the result, not awareness. On the other hand, WOM for awareness can be invested in when it’s somewhat intrusive, passed by less loyal customers to acquaintances. In other words, WOM drives awareness when the execution is similar to Advertising, creating conversations where they did not already exist. But this notion of “Intrusive Social” creates somewhat of a Paradox for many of the Social advocates who view intrusive practices to be “anti-Social”. One has to wonder if when it comes to generating awareness, Advertising might be the better way to go.

This research is a great piece of work from a couple of very creative analysts, and well worth your time to review. Also, a hat tip to BzzAgent for being so open with their practices and sharing the data for this very important study.

If you are primarily a Marketing person and more interested in the end Behavior than the Analytics, just skip over the Math sections, as these submissions are peer-reviewed and would not be published if the Math (or test design, for that matter) were faulty. As such, in your pile of research on Social, this piece should be given a lot of weight. Junk science doesn’t make it to publication in the academic journal world – as opposed to many of those blogs you probably read!