Type to search

Solving big data’s connectivity challenges

Technology & Data

Solving big data’s connectivity challenges


Craig Law examines the concept of data lakes, and explains why data is only as useful as the insights that can be dredged from it.

It’s no secret that organisations are collecting data at a rate that is faster than ever before, and utilising that data to inform important business decisions every day. Marketing data in particular is a central part of every business, and digital transformation is further helping companies to capitalise on data collected from customers to improve customer service and sales efforts for a more compelling customer experience.

Members of the C-suite, including CIOs and CMOs are just beginning to take note of the impact marketing data can have and are increasingly looking to drive investments around data technology to further that potential. However, data is only as useful as the insights that can be pulled from it, so businesses must consider where to store and how to organise data to maximise the benefits it provides.

One of the latest trends in data is the process of storing data in ‘data lakes’. To put it simply, a data lake is an archive which stores a tremendous amount of raw data in its native format for as long as it needs to be held.

More specifically, data lakes provide massive storage for all types of data, whether structured, unstructured, or semi-structured, that can later be analysed for uses including:

  • 360 degree customer views: Companies can achieve a complete view of customers’ needs, likes, and dislikes through the aggregation of data that is generated through the many touch points a company has with its customer for purchases and service requests.
  • Predictive lead scoring: This methodology predicts the future behaviours of leads by leveraging historical data and predictive analytics through the attribution of numerical qualifiers to those leads.
  • Personalisation: Through organising and applying consumer data appropriately, marketers can use personalisation to target groups of customers with campaigns more directly aligned with their interests and behaviours.
  • Sentiment comparison: Analysing data collected from customers for sentiment comparison enables marketers to understand how products and campaigns are being received. Marketers can use this information to track trends in those opinions.

Related: Rob Morrison on how consumer behaviour can be predicted using the combination of cognitive computing and data analytics »

The difficulty here lies in the fact that marketing data is most often stored in the cloud, which can often present a connectivity challenge between the disparate cloud data sources (such as Salesforce, Oracle CX, Marketo, and others). As the business ingests new types of poly-structured data into lakes, it becomes harder to make sense of it without accessing all of the marketing data stored in the cloud. This can result in a data lake becoming something of a data swamp if the data continues to amass without having a complete view of the marketing systems.

The key to avoiding problems with accessing data between incongruent cloud data sources is improving the connectivity between them so that there is one standard API. This data hub should allow access to anything in the cloud or behind the firewall and should integrate well with all types of data ingestion tools that leverage a common API, such as JDBC. A key component of this is speed: the faster the access point, the better.

Digital businesses that choose the right technology and approach to storing data will become more efficient and be agile enough to adapt to the changes that are constant in today’s business ecosystem. Making the decision to implement embedded connectivity within the technology a business uses provides a competitive edge for that business over others that aren’t harnessing their data as effectively. A simple, streamlined SaaS approach for data connectivity can maximise the potential of an organisation’s data analytics, resulting in a more fluid, omni-channel customer experience and personalised marketing strategies.



Craig Law is managing director ANZ at Progress


* * * * *

Purchase a subscription to Marketing

* * * * *


You Might also Like

Leave a Comment