Fast data trumps big data, writes Scott Thomson.

Business has fallen in love with big data, but the old world methods of data analytics have become a bottleneck for all that data offers. It is time to break into the new world analytics that actually begin to make the overwhelming

amount of data being produced by enterprises useful to those who actually need it.

The time has come for the analysis of data, and even the data being analysed to be reassessed. While the explosion of big data has promised new insights into business, the pace at which it can be used is slowed by the application of old world analytics. Old world analytics is slow moving, potentially suffering from the subversion or exclusion of input data by siloed departments picking and choosing the data that suits their agenda best. It can experience hours, days or weeks of latency meaning that the insights, trends and analysis derived from old world analytics also tend to lag indicators of business decisions and can fail to provide meaningful insight into the future direction of the business.

The promise of big data has been to position it as a way to rectify the latency and lag that plague the old way of analysing. But the reality is big data is just as prone to subversion and further latency and has more in common with old world analytics models and further isolates business leaders from real time insights available in enterprise data. It’s big data, but viewed through the same old lens.

The three Vs of big data – approach with care

In 2001 Gartner coined the ‘Three Vs’, what they called Volume, Variety and Velocity – and they are still used as the primary reference on how data capacity is managed and what the constraints are. But there is nothing new about how or what they actually refer to. They are a great way to sell lots of storage, computational power, advanced data analysis, data modelling tools and specialist data scientists. But, at the end of it all, what value is there?

The problem with the Three Vs is that they are set up to look for needles in a haystack, when in reality there are giant logs that lie across every business haystack, which we already understand well as analysts and business leaders. The big logs are the models that drive our businesses and big data is merely the latest name for a range of products that try to improve our understanding of these models.

Forget the Vs, business needs the three Cs and to ask itself six questions

Business leaders need to understand the complexities of big data, and that will lead them to ask the right questions and, in turn, choose the best analysis. The Three Cs are:

1. Context: what are you collecting all this data for? What business problem are you solving?
2. Comprehension: do you know what all your data means? Is it all properly tagged and able to be associated with other data sets?
3. Collaboration: how can anyone else play with this without costing millions of dollars and can we easily associate data from other sources?

Once you understand these three complexities you can begin to assess your business and its needs against them.

1. What industries or parts of your business make the most money?
2. What models drive those industries and businesses? Do any models drive more than one industry or business?How much money are those models worth?
3. What data do you need to build those models?
4. Who has the data you need to build those models? Will they allow you to use it? Do you need raw data or rolled up data? Do you need identifiable data or will anonymous segments do?
5. Who has the data you need to build those models? Will they allow you to use it?
6. How can you collaborate to get the data or segments? How much is it worth to you to get that data? What will you pay? How much will you make?
7. Finally, what tools, processes and skills do you need to put in place to use the data?

You may find that you don’t need a data processing program such as Hadoop, or even data scientists – you don’t want to spend a lot of time and resources on hardware, consultants and technology.

Businesses can lose years of advantage working all that out. And you certainly don’t want a more senior business leader coming around and saying, “What’s all this kit for and who authorised it?” if you don’t have clear business-driven answers to the question.

Put the business first

The model needs to be flipped on its head and, in looking at big data, businesses need to put the business first and not let technology drive the agenda.

In doing this you will most likely discover that what you really need is a way for business leaders to collaborate on disparate data sets with more focused application of data experts. You will want the ability to build and iterate on ad hoc queries quickly and you will want the ability to use your data insights and feedback success scores quickly to drive the success of your business.

What is far more valuable than big data is fast data, accessible data, actionable data and measurable data. This isn’t just about the speed or accuracy of insights delivered; it is also about the accessibility of the insights to business leaders.

You don’t want to have to engage in a six- to 12-week big data technology discovery and delivery cycle to ask each data driven question. Businesses simply can’t afford to wait days or weeks to get data insights to market and months more to find out if data driven initiatives are working.

New world data analytics first and foremost is a shift in thinking about how we manage data and who can access it. Data needs to be streamed into a real time pipe, from all sources including devices, CRM (customer relationship management), social and third parties, then made available to the parts of the business that need to use it. But the technology should support this approach, not drive it.

Slow data – such as product holding information, and geographic and demographic data – can also be loaded into the pipe at the data warehouse of an enterprise, which gives a total picture. With the help of algorithms, the business can then use real-time data as it needs. Obviously it’s a little more complex than that, but it’s a process that removes the lag for business.

The key is that new world analytics systems need to make the real-time data accessible, comprehensible and usable to business users with sandbox style ad hoc query and data visualisation tools. In doing so, they allow a business to scale efficiently beyond the size of a core team of data scientists and technical analysts. The data engine needs to be placed in the hands of the common business user in order for its true value to be realised.

In any industry, no matter how skilled and novel, there is always the inevitable maturity in process leading to greater accessibility and scalability. The era of new world data analytics is upon us and it is time to evolve or fall behind.

Adobe is a Marketing content partner – collaborations with leading organisations on content for the magazine (like this article) as well as exclusive benefits for Marketing Advantage Members. Find out more about membership here.