Forrester Research says that we are now living in the “Age of the Customer” where “the only sustainable competitive advantage is knowledge of and engagement with customers.”
Aprimo CMO Lisa Arthur writes on Forbes “Why 2013 is the Year of the Marketer” where CMOs are optimistic about the growing relevancy of their marketing organizations because of their ability to exploit data. Marketers and marketing analysts need to ride the Big Data wave in order to stay relevant in their careers.
Marketers Need to Learn Some Data Skills
All of the online product marketers that I personally know have learned to use SQL to run simple ad-hoc queries directly in the data warehouse. They may call upon me and other analytics professionals to help with more complex dashboarding, A/B testing, data mining, and statistical modeling. If the marketers didn’t have much experience with data analysis previously, they definitely see the value now and need to learn fast to adapt.
However, I have not yet met a marketer who is comfortable with running queries directly in Hadoop. Running SQL queries in Oracle or Teradata using a GUI-based SQL tool is within reach of most marketers who are willing to spend a little time to learn, especially if they merely need to extract data into Excel spreadsheets. But dealing with Linux command lines, SSH public/private keys, VI, regular expressions, CRON, and the Java JAR files needed for the effective use of Hadoop with Pig Latin or Hive, seem very daunting for those in the general marketing role. Eventually, the tools to work with Hadoop to analyze big data will get easier and easier, and won’t require such specialized technical skills, so that more mainstream analysts and marketers can tap into the data without as many struggles.
Use Data to Understand the Customer
For now, my role in analytics helping the marketing team is to 1) understand the business challenges, 2) get data from where ever it’s available, 3) analyze and interpret the information, and 4) deliver appropriate recommendations to the business partners. If it’s a recurring opportunity, then there’s much to gain from automating the above process as well.
Marketing is all about understanding the customer. When marketers understand what struggles the customers are going through, what desires and hopes the customers have, and what motivates the customers, then the marketer can truly start building meaningful relationships with the customer. Yes, this sounds a lot like dating.
As marketers start developing product strategies for delivering solutions to the customers, and well before marketers start running campaigns to woo the customers, they need data. More than data, the marketers need insights into the customers–bonus if the marketer knows the customer even better than the customer knows himself. This is where customer analytics play a key role.
We live in an increasingly online world. Customer analysts need to extract meaningful insights out of a wide variety of data sources, many of which may have immense data volumes. It’s both the breadth and depth of data that contribute to the “big data” syndrome that we are trying to tame. For example, the data used in marketing analytics may come from the following sources:
- Transactional ERP systems and order management systems
- Product registrations
- Email Campaign management systems
- Purchased databases about customers and companies
- CRM and Customer Service databases
- Social media
- Web logs
The above data may live in relational databases, or email inboxes, or in external websites. They may be the structured data in an application database or spreadsheet, or the unstructured data in a web log file or in an email discussion forum, or the external data available only via API’s or web crawling. Whoever can tie all these data sources together in order to thoroughly understand the customer’s past, present, and predicted future will be someone greatly sought after. Big Data is not only about the fast growing data volumes, but also the complexity of the multitude of different data sources.
How Does Hadoop Help?
This is where Hadoop provides a solution for Big Data. Hadoop is free open-source software, usually run on clusters of cheap commodity servers, that provides a scalable way to store and retrieve such massive volumes and complexities of data. Although Hadoop is not intended to run critical financial applications, Hadoop is ideal for batch number crunching and data analysis, especially of web logs. At this time, regular Hadoop does not offer the transactional data integrity that a traditional relational database provides (ask a DBA about ACID-compliance).
Hadoop by itself won’t give you all the answers. You’ll still need analysts to run statistical analysis to interpret what happened and why, as well as data scientists to build predictive models of people’s behaviors. However, Hadoop does enable the marketing and analytics teams now to have the ability to leverage Big Data, and allow them to dream big.
See some of the Top Big Data Vendors.