The Challenge

A large, global Data Management and AI software company with over 5,000 customers across more than 100 countries had seen unprecedented growth (more than 30% year over year) in telemetry data from their multi-cloud infrastructure being sent to the Elasticsearch Observability and SIEM Platform. The growth of this data contributed to a multi-million dollar price tag for Elasticsearch.

Their security and DevOps teams also reported several gaps in their security and compliance efforts owing to very noisy telemetry data being fed to Elasticsearch. Finding actionable data within Elasticsearch became increasingly difficult in a growing sea of data with no analytical value. They also found several incidents where personally identifiable information (PII data) showed up in unexpected fields harming their ability to comply with their internal standards as well as privacy regulations like GDPR and PCI.

The Solution

The company began searching for an observability pipeline to do log management for data that had increasingly grown over the past few years. They wanted a solution that could automate the optimization of this log data, reduce noisy log volume, and help them comply with security and privacy policies. They chose Observo AI and its AI-powered observability pipeline. They were up and running with Observo AI in a few hours and fully implemented and optimized their observability pipeline within a week.

Optimizing log data with Observo AI is automated. By applying data type-specific transforms, Observo’s AI-based smart summarization can reduce the volume of log data including VPC Flow logs, CloudTrail logs, and App Logs. They reduced log volume by more than 70%, helping them cut Elasticsearch costs and reduce noisy data to surface meaningful insights better. 

Observo AI proactively detects sensitive and classified data, allowing the company to secure it through obfuscation and hashing. The company used pattern recognition from the Observo AI solution to discover all sensitive data, even in unexpected places like open text fields.

Results

“Observo.ai really transformed how we do Log Management. We were able to reduce costs while supercharging our incident detection & response capabilities." 
Gary L, VP, Cloud & Product Security

Telemetry data optimization via Observo AI’s observability pipeline significantly impacted Elasticsearch costs and issue resolution. The company was able to reduce its total Elasticsearch Observability and SIEM costs by more than 40%. Reducing noise helped them detect and resolve issues much faster, helping them reduce Mean Time to Resolution (MTTR) by more than 40%.

By using the AI-powered compliance and sensitive data discovery features of Observo AI, they closed many of the gaps in their security and compliance efforts. By protecting all of their sensitive data and PII, they have much more confidence in their ability to comply with GDPR and other privacy regulations.

Learn More

For more information on how you can save 50% or more on your observability costs with the AI-powered observability pipeline, Read the Observo AI Whitepaper, Elevating Observability with AI.