What is the Impact of Big Data?

Thursday, October 11, 2012  
Posted by: DAA Administration

Several digital analytics experts that will speak at the DAA Seattle Symposium 2012 tackle this important question.

SEATTLE, Washington – October 9, 2012 – Overwhelmed by the vast amounts of data produced and consumed by your organization? You are not alone. Advances with web technology, the rapid rise of e-commerce, and widespread adoption of web-connected mobile devices allow businesses to draw upon more and more information about customers. While this data blitz creates challenges with data management (e.g., processing and storage), security and interpretation, analytics-savvy businesses are now using this abundance of data to their advantage, especially in the realm of predicative analytics.

We recently engaged several experts who are slated to speak at the DAA Seattle Symposium 2012 to comment on this topic. We asked John Bates of Adobe, Joshua Koran of Turn, Chen Zhao of Marchex Institute and Erika Clemens, Group Product Planning Manager at Microsoft to discuss the impact of “big data” and whether this is more than merely a buzzword.

John Bates, Product Manager of Predictive Marketing Solutions, Digital Marketing Suite at Adobe Systems responds:

“Similar to past business revolutions – Microsoft Windows transforming the office, Google redefining internet search, Apple revolutionizing music – we are entering a new age – the Data Age. The explosion in the volume, velocity, and variety (i.e., “big data”) of data is leading to increased availability and accessibility of information, which is further enhanced by the ever-decreasing cost of data storage and processing. For organizations that strategically exploit this data to drive insights and optimizations, the new age of Data is a significant opportunity. One of the primary sources of “big data” has come from the digital channel. Though there are numerous debates regarding the validity of the hype surrounding “big data”, the fact is that “big data” is here to stay and will only get bigger. Consider the following facts: 90% of the data in the world today was created within the last two years (2011 – 2012), 2.2 million terabytes of new data is created every day, the compound annual growth rate of the big data market is 40%, and the market for big data is expected to grow from $3.2B in 2010 to $16.9B in 2015. The challenge for organizations and especially digital analytics groups is to analyze and act on this ever-expanding mountain of data in such a way as to drive improved performance through real-time personalization across any channel. Big data is more than just a buzzword; big data is the new reality of digital analytics and optimization.”

Joshua Koran, Senior Vice President of Product Management at Turn responds:

“Big data is more than a mere buzzword. Traditionally, we built data warehouses to offload the performance issues of running queries on our transactional data marts. With the rise of distributed data processing, (which underlies the Cloud), we had reduced costs in asking questions of this growing amount of integrated data. However, the current evolution in analyzing big data is around the way we store data within our databases (e.g., nosql) and the processing languages we use to ask questions. Both of these enhancements should make the current stage of data analysis far easier and faster than the approaches we’ve used in the past.”

Chen Zhao, Principal Analyst at Marchex Institute responds:

“The answer, in my opinion, is (as the term “big data” suggests) “only in terms of data, but not in terms of analytics” and let me explain why.

Data changed in volume, velocity and variety as a result of massive digitalization of information and media (Source: HBR October issue. Big Data: The Management Revolution):

  1. Volume: Cheaper storage and faster machines allows us to keep detailed customer behavior.
  2. Velocity: Data is not static. Customers’ profile status changes by the minute. Marketers want “real-time” reporting.
  3. Variety: The multitude of digital marketing channels and therefore a myriad of data sources. The explosion of types of responses, engagements, and interactions that one can capture as data.

On the other hand, the fundamental data analytics techniques we apply today are the same as what we used 15 or more years ago.

Let’s use predictive modeling as an example. Before the internet, predictive analytics are mostly performed in database marketing (think credit card issuers use modeling to select credit worthy customers to send their applications to). Then came the internet in the 90’s and with it came the web site data. Today the data sources extend across multiple devices including tablets and mobile. This vastly increased the amount of “signals” analysts can use in their predictive models, making predictive analytics much more interesting and challenging at the same time. Meanwhile, the types of “response” data also became increasingly digital. One example is phone calls. 20 years ago, the traditional direct marketers used analogue advertising channels (magazines, postcards, flyers…) to drive analogue responses – phone calls and cash register rings. Today the digital marketers use digital channels (web, mobile, etc) to drive digital responses – digital calls, online purchases.

However, we still apply the same predictive modeling techniques – logistic regression, linear regression, hazard models, classification and regression trees, etc. Marketers are still interested in the behavior of the same customers, only that we have more (digital) touchpoints to reach them and we are capturing more of their response behavior from multiple devices.

Is big data just a buzzword?

There are two aspects of “big” – volume and variety. As an analyst, the biggest opportunity here is to take advantage of the “variety” of the data. I do not believe in the value of “volume.” In fact, I have seen companies being paralyzed by the volume (what I call data paralysis). For a statistician, a good random sample can replace the “big data” and provide the same analytical results. Moreover, big data also means “big noise.” I see big data as more of a challenge to finding the right information from data.

On the other hand, the variety of data puts analysts in a much better position to deliver deep insights.

The bottom line is, I would not get excited about the volume of the data but rather, the value is in the variety of the data. To take advantage of data, engineers need to work side by side with analysts and statisticians to truly extract the value of data. Just having mounds of data does not equal value.”

Erika Clemens, Group Product Planning Manager at Microsoft responds:

Big Data is definitely more than a buzzword. Increases in processing and storage capabilities have opened up amazing new possibilities in the field of digital analytics. The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s. For example, decoding the human genome originally took 10 years to process; now it can be achieved in one week. However, with all the possibilities big data brings, it is also challenging to work with because of its size and breadth. It forces us to reconsider data management options and develop new strategies around relational databases, visualization and virtualization and parallel data warehouses. It also requires strong analysis methodologies and frameworks to be defined and followed. With Big Data, sampling is frequently required due to size and processing limitations. I have found that the larger the data set gets, the more important it is to emphasize and adhere to the scientific method. As our technology and capabilities change, our approach to analysis remains the same. When approaching analysis, we still formulate a question or set of questions that lead to a hypothesis, develop a methodology for the data set to be used, analyze the data and prepare the results to prove or disprove the hypothesis. With so many possible questions to be asked of our data, it is important that we are sure we understand how the data will be used and that we continue to foster a culture of data driven decision making.

“Big Data” and other important topics about the future of digital analytics will be covered at the DAA Seattle Symposium 2012 set for Wednesday, November 7 at the Microsoft Conference Center in Redmond, Washington. Registration for members is $10 and $50 for non-members. For more information about the event and speaker line up, visit: https://www.digitalanalyticsassociation.org/?page=seattle2012