Beyond Big Data: The extreme data economy

There is more data coming back to and being generated by organizations than ever before and it will only get more complex. This means that organizations need to think about how to simplify their data architectures to grow and thrive in the extreme data economy.

“Those organizations that continue down the path of using legacy big data architectures to solve problems they weren’t designed to handle will ‘fumble in the dark,’” according to Joe Lee, Vice President of APAC and Japan, Kinetica in an interview with Networks Asia.

Kinetica is a San Francisco-based startup (with a $50 million series A funding) and the provider of the fastest GPU-accelerated database. In a nutshell, they allow businesses to analyse high volume, unstructured, and moving data sets faster, using machine learning, BI analytics, and visualisation.

Kinetica allows its customers to process data 100 times faster than a legacy database, at a tenth of the cost associated to investing in hardware. GSK in Europe use their analytics tool to accelerate simulations of chemical reactions and the Lippo Group in Asia allowing the company to consolidate multiple dimensions of customer attributes, including demographics, cross-channel buying behaviour such as their sentiments as gained through social media, and interactions in-store and online

“Those businesses that realize we have moved beyond the Big Data world into the extreme data economy will explore how to build a data-powered architecture that combines GPU-powered databases, location-based analytics and visualization, and machine learning to create a new speed layer on top of their Big Data landscape,” said Lee. “Thus, businesses will be able to use their data to create core intellectual property that not only defends their existing business, but also creates new business models and drives a much deeper understanding of the customer.”

In the following Q&A, Lee further talks about the move from Big Data to an extreme data economy, and its implications on emerging technologies such as artificial intelligence, machine learning, and the Internet of Things.

Weve seen the videos of robots working in amazon warehouses. How far are we from the machine revolution? Do enterprises understand what AI means for now?

We are well on the way to the machine revolution. With the advent of the Internet of Things, we are now deeply entrenched in a world where we are constantly brokering relationships across users, devices and things. We need to build new approaches to edge security, incorporate service-to-service interactions through a microservices architecture, and adopt machine learning and artificial intelligence for better automation. All of these elements are foundational to the machine revolution.

As organizations move in to the extreme data economy, they will need to handle exponentially greater volumes of batch and streaming data, figure out how to operationalize machine learning as part of the core business, and conduct advanced analytics with millisecond response times in order to move from near-real-time data strategies to real-time data strategies.

Data-powered businesses will emerge, ones that are well prepared for applying intelligence to machines and enabling automation.

Across most industries, this is still in its infancy; currently organizations worldwide are hiring data scientists, putting in new GPU-powered infrastructure for massive parallel processing, and determining how best to operationalize AI in their core business.