Does Chatter Really Matter? Dynamics of User-Generated Content and Stock Performance

Back to the Peer Reviewed Journals List

Executive Summary

Tirunillai and Tellis explore the relationship between User Generated Content (UGC) and stock returns.

The authors define UGC as product reviews and product ratings. This point is important. They make use of traditional product ratings and reviews, not Twitter, Weibo, Facebook, Youtube, or blogs for their analysis. They cite signal-to-noise ratio problems in blogs and videos as a reason to exclude them.

The authors are focused specifically on user feedback in relation to products. They reason that this feedback will influence other consumers in their purchase decisions, which in turn, can have an impact on quarterly sales. They further reason that traders have varying capabilities in understanding and responding this UGC information.

They predict that negative information is more impactful than positive information upon product launch. Marketers tend to highlight the positive features of a product before launch, so that information is known. The negative information, derived from consumer experience, carries considerably more novel information.

They chose only companies where a single product group makes up the bulk of the sales, as opposed to highly diversified firms, and that are listed on US Stock Exchanges. They examined the following companies: HP, Dell, Motorola, Nokia, RIM, Palm, Sketchers, Timberland, Nike, Mattel, Hasbro, Leap Frog, Seagate, Western Digital and SanDisk. They examine reviews and stocks from June 2005 to January 2010. The UGC was sourced from pure UGC sources (not expert reviews) from Amazon.com, Epinions.com, and Yahoo! Shopping. The total number of reviews harvested was 347,628.

They coded for valence (industry term: sentiment) using both a naïve Bayes classifier and a SVM (Support Vector Machine). Where there wasn’t agreement between the two systems, they painfully hand coded reviews that exceeded 10 words in length. They also harvested product ratings (1 stars to 5 stars), and the volume of chatter (the total number of reviews posted by consumers about the products of a firm on any given day).

They include a series of control variables, including analyst forecasts (I/B/E/S database), advertising dollars (TNS Media Intelligence), media citations (LexisNexis/Factiva) and new product announcements.

Taken together with the stock information, there are 1112 trading days of data.

They chose vector autoregression (VAR) as their model of choice, and test for Granger Causality to make an assertion about the directionality of causality. (Expressed more simply, they make use of time lags to assert that the UGC preceded changes in stock value).

They find no instantaneous relationship between any of the UGC metrics and stock return. In other words, there’s a delay between UGC and the stock return. "Positive chatter does not have a significant impact on returns, risk, or trading volume. Negative chatter, on the other hand, influences returns negatively both in the short term…and in the long term.” (p. 208) The magnitude of the impact of a unit shock of negative UGC could erode "$3.3 million over 15 days following the chatter.” (p. 208).

In sum, the authors find a relationship between product review UGC and stock return.

Review

So long as the stock market exists, and managers are incented by it, this area of study will persist. The linking of marketing to stock returns is intoxicating because it draws a straight line between action and reward.

The linkage between product quality, advertising, customer experience, customer decision to write a UGC review, valence of that review, influence of that review on purchase, quarterly sales, and stock price is seven chains long. The public breaks down at three chains and most of us quit after five. That’s an extremely long chain to consider and a high mountain to climb.

Many in Marketing Science have attempted linking various forms of word of mouth to stock performance. Very few of those papers make it through peer review and into publication. The thoughtfulness and consideration that Tirunillai and Tellis put into this paper, succeeding where so many have tried, is what sets it apart.

The point of the paper is not to make stock market predictions. Rather, it was to link a relatively new phenomenon in nature to stock performance.

A good takeaway from this entire study is that, yes, indeed, what people say to one another in product reviews influences stock returns in some significant way, especially when taken over time. There’s good empirical evidence that what consumers say out there has an impact on the bonus envelope, so don’t ignore it.

I recommend this paper to members of the Digital Analytics Association.