Episode 2: The Challenge of Data: Using Search to Accelerate Results That Matter

Ask anyone who’s lost an online auction or abandoned a shopping cart in today’s hybrid & multi-cloud world, speed matters for transactions and business decisions –      nanoseconds count.

As cloud migration continues apace, accessing and using the data that runs and informs your applications has become a challenge for organizations of all sizes. Cloud Search company Elastic takes the challenge of the observability of data availability and security head on.

Defining the challenge

For a typical data team, 80% of time is spent on data discovery, preparation, and protection, and only 20% of time is spent on actual analytics and getting to insight, says IDC. Data silos, legacy data management tools and skill sets, and the impact of the COVID-19 pandemic all have hobbled organizations’ efforts to unify, share and analyze data for forecasting and better business decisions.

“It’s astonishing how much inefficiency exists across the industry,” says Brian Bergholm, Senior Marketing Manager in the Cloud Product Marketing team at Elastic. “In fact, based on some recent survey work that Elastic conducted with Wakefield Research, we found that 81% of knowledge workers say they have a hard time finding documents under pressure.”[1] 

That inefficiency has some hard costs associated with it, Bergholm says, which are manifested in three trends noted by Elastic.

“One, information is becoming harder to find, and this inability to find information actually costs the average enterprise $2.5 million per year,” he says. “Second, enterprise IT is becoming harder to keep performant, and system downtime costs the average enterprise $1.5 million per hour. Third, cyber threats are becoming harder to prevent, and these data breaches cost the average enterprise $3.8 million per incident. Adding these all together accrues to millions of dollars of unnecessary costs.”

Components for data success

To tackle these costly inefficiencies, organizations are turning increasingly to an integrated approach as opposed to a suite of point solutions. Bergholm points to three advantages of an integrated solution: speed, scale, and relevance.

“From a speed standpoint, you can find matches in milliseconds within both structured and unstructured datasets,” says Bergholm. “You can scale massively and horizontally across literally hundreds of systems, and the most important aspect is that you can generate highly relevant results and actionable insights from your data.”

Integrated solutions also increasingly take advantage of technologies like artificial intelligence and machine learning, he says.

“We’ve also built machine learning in so it’s in the suite, and these capabilities can be leveraged across all three solution areas: search, observability, and security.

The security mandate

When talking about data challenges, security is a prime consideration. There are two sides to the coin: data security, and leveraging data for security intelligence.

First, any search solution itself must be secure and compliant. “Elastic takes data sovereignty very seriously” says Bergholm. “We’ve invested to ensure Elastic is operating in compliance with the principles of GDPR, and, in fact, Elastic Cloud is available in 17 Google Cloud regions. This allows you to place applications where the data lives and supports local data sovereignty and governance requirements.”

Second, an integrated search approach can be applied to the security data that’s collected routinely by organizations.

“By using advanced search analytics, you can leverage petabytes of data, enriched with threat intelligence to glean the insights you need to protect your organization,” says Bergholm. “Search also helps mitigate cyber threats by exposing unfolding attacks by correlating diverse data. We use machine learning algorithms and natural language processing capabilities and other tools to better understand context and meaning from a wider array of data types and formats, and all of this helps your SEC ops teams to quickly identify issues.”