15 – 5 Actionable Lessons from the 2016 Big Data Survey


NewVantage partners is a Big Data consulting firm that runs an annual survey called “Big Data Executive Survey” in which it targets senior Fortune 1000 business and technology decision makers.

In 2016, 73% of the responded were from financial services, 18% from life sciences and rest from different industries. They survey respondents were as follows:
Executive Participation C-Executives 35.9% Head of Big Data 28.6% Head of Analytics 12.5% Senior Technologist 16.1%. Most of the respondants are from large brand names such as BOA, AIG, Citigroup, Wells Fargo, CVS and Harvard Medical school to name a few.

The purpose of this survey is three fold:

1. To understand the role of big data and analytics in the organization
2. Get insights on how big data is being applied across various business units
3. How analytics is changing the industry and the organizations

We have distilled the survey into 5 actionable insights for our listeners so you can apply these insights directly to your business.

Listen to the 5 Actionable Lessons from the 2016 Big Data Survey

Show Summary:
1. Big data is here to stay

Enough has been said about the growth of big data analytics and how it is critical for your organization to be data driven. Big data is now mainstream and the growth in both usage of data for decision making and big data technology has been phenomenal. International Data Corporation has forecasted a 23% CAGR over 2014-2019 with annual spending reaching at 46Bn in 2019. The Big Data Executive survey validates the growth and confirms 62% of the firms have atleast one instance of big data in production. This is double of 31% reported in 2013. 26% of the firms are looking to invest in the excess of $50M in Big Data technology, people and infrastructure in 2017.

32% of the firms have labeled big data projects as mission critical to success of their organization (a 40% increase since 2015 survey).

What’s also interesting is 91% of the time the big data projects have executive buy in. This is pretty telling as far as the big data future holds as there will be no shortage of investments in data projects. If your organization is still thinking about investing in big data then it is the perfect time. There are so many options and solutions in the market which are cost effective and scalable. In our podcast#12 we discussed, multiple free and cheap tools to get started with big data. i.e. BigML, IBM Watson Analytics and Microsoft R. You can also test a full scale managed 10 note Hadoop framework for 15cents an hour at Amazon Elastic Map reduce.

2. Chief Data Officer is a required position.

I remember back in 2008, when the company I was working was starting to invest in analytics. We were hiring our first business and data analytics professionals. Organizations were cautious in spending on data projects and those that took the risk the payoff was exponential. Today, data roles within the organizations are fairly common and C-level positions such as chief data office and head of analytics are opening up.

Organizations are investing in people resources and technological resources alike.
The survey confirms a 350% growth in having a chief data officer role in the organization. 54% of all firms surveyed now have a CDO role filled and 20% of the firms have reported CDO as the primary owner of big data initiatives.

3. Business and technology group alignment is critical

We practice what we preach and we have always recommended aligning the core business objectives with any type of organizational projects. We have seen a significant growth in projects aligned with the business objectives and failure of projects that are rouge and disconnected in nature. Since big data projects have 91% executive buy in, it becomes slightly more easier to get alignment across business and technology groups. The survey reports the single biggest factor for big data adoption is the business and technology integration. 34% of the firms strongly believe in partnership of business and technology leadership to make big data projects successful.

4. Actionable insights and faster response time are key drives of big data investment

The main advantage of using data to drive business objectives is your organization is less likely to fail. Data driven decisions are sort of an insurance policy for your projects and initiatives and they add an extra layer of protection against negative outcomes. Investing in data projects allows organizations to rise above the traditional “because I said so” approach and gives new levers for testing and hypothesizing multiple scenarios. It also increases the response time and the time it requires the organization to understand good vs bad decisions. Marketing is another area where big data has made a significant difference. In podcast#14, we discussed the advantages of predictive and intelligent marketing for those who are interested.

They survey validates, 37% of the firms believe the ability to develop greater insights as the critical driver of big data investments. Participants also agreed on the importance of time of answer and time to decision which comes with the big data implementation.

5. Variety of data is more important than velocity or volume

We are all fascinated by the astonishing growth in the data volume. Infact, IDC predicts, 1.7MB of new data will be created every second by 2020 which equates 40ZB. Gosh I think we will soon run out of labels for the volume growth.
What’s interesting though was as per the survey it is the variety of data and not the volume which is driving big data projects. 40% of the organizations are making investments in Big data projects due to the variety of data. I think I kind of have to agree specially in the marketing world the data is being generated across various fronts. We have text data generated from social media, digital analytics data, call center acoustic data, visual data generated from eye movement technologies, video and audio media data, offline store and beacon technology data, mobile data. It is extremely important to integrate various data sources so we are not making decisions in silos.

This is inline to the survey results which cites 41% of the organizations plan to invest in integration of bid data and data warehouse using a data hub or data lake.
There has also been a 40% growth since 2013 in the usage of analytical sandbox, big data lab or data center for testing and trial purposes. This has resulted in the improvement of speed to actionable insights from data projects.

Now take action and listen to the podcast for a more meatier and detailed action plan.

Resources discussed in this podcast:

Leave a Reply

Notify of