Real-time Data, Machine Learning, and Results: The Evidence Mounts

By Bryan Kirschner, Vice President, Strategy at DataStax

From delightful consumer experiences to attacking fuel costs and carbon emissions in the global supply chain, real-time data and machine learning (ML) work together to power apps that change industries.

New research co-authored by Marco Iansiti, the co-founder of the Digital Initiative at Harvard Business School, sheds further light on how a data platform with robust real-time capabilities contribute to delivering competitive, ML-driven experiences in large enterprises.

It’s yet another key piece of evidence showing that there is a tangible return on a data architecture that is cloud-based and modernized – or, as this new research puts it, “coherent.”

Data architecture coherence

In the new report, titled “Digital Transformation, Data Architecture, and Legacy Systems,” researchers defined a range of measures of what they summed up as “data architecture coherence.” Then, using rigorous empirical analysis of data collected from Fortune 1000 companies, they found that every “yes” answer to a question about data architecture coherence results in about 0.7–0.9 more machine learning use casesacross the company. Moving from the bottom quartile to the top quartile of data architecture coherence leads to more intensive machine learning capabilities across the corporation, and about14% more applications and use cases being developed and turned into products.

They identified two architectural elements for processing and delivering data: the “data platform,” which covers the sourcing, ingestion, and storage of data sets, and the “machine learning (ML) system,” which trains and productizes predictive models using input data.

They conclude that what they describe as coherent data platforms “deliver real-time capabilities in a robust manner:they can incorporate dynamic updates to data flows and return instantaneous results to end-user queries.”

These kinds of capabilities enable companies like Uniphore to build a platform that applies AI to sales and customer interactions to analyze sentiment in real-time and boost sales and customer satisfaction.

Putting data in the hands of the people that need it

The study results don’t surprise us. In the latest State of the Data Race survey report, over three quarters of the more than 500 tech leaders and practitioners  (78%) told us real-time data is a “must have.” And nearly as many (74%) have ML in production.

Coherent data platforms also can “combine data from various sources, merge new data with existing data, and transmit them across the data platform and among users,” according to Iansiti and his co-author Ruiqing Cao of the Stockholm School of Economics.

This is critical, because at the end of the day, competitive use cases are built, deployed, and iterated by people: developers, data scientists, and business owners – potentially collaborating in new ways at established companies.

The authors of the study call this “co-invention,” and it’s a key requirement. In their view a coherent data architecture “helps traditional corporations translate technical investments into user-centric co-inventions.” As they put it, “Such co-inventions include machine learning applications and predictive analytics embedded across the organization in various business processes, which increase the value of work conducted by data users and decision-makers.”

We agree and can bring some additional perspective on the upside of that kind of approach. In The State of the Data Race 2022 report, two-thirds (66%) of respondents at organizations that made a strategic commitment to leveraging real-time data said developer productivity had improved. And, specifically among developers, 86% of respondents from those organizations said, “technology is more exciting than ever.” That represents a 24-point bump over those organizations where real time data wasn’t a priority.

The focus on a modern data architecture has never been clearer

Nobody likes data sprawl, data silos, and manual or brittle processes – all aspects of a data architecture that hamper developer productivity and innovation. But the urgency and the upside of modernizing and optimizing the data architecture keeps coming into sharper focus.

For all the current macroeconomic uncertainty, this much is clear: the path to future growth depends on getting your data architecture fit to compete and primed to deliver real time, ML-driven applications and experiences.

Learn more about DataStax here.

About Bryan Kirschner:

Bryan is Vice President, Strategy at DataStax. For more than 20 years he has helped large organizations build and execute strategy when they are seeking new ways forward and a future materially different from their past. He specializes in removing fear, uncertainty, and doubt from strategic decision-making through empirical data and market sensing.

Data Architecture, IT Leadership