(Macroecon/Shutterstock)
The concept of dynamic observability has emerged to exploit the full breadth combined observability data, including logs, metrics, and traces. Observability tool provider Sumo Logic has also embraced the concept, but with its own twist.
Observability emerged as a new concept several years ago to reflect the idea that, by combining logs, metrics, and traces into a single environment, developers and site reliability engineers (SREs) could glean more insight from the functioning of complex systems. In short, by combining the various types of data monitoring, observability backers expected new insights to emerge that would be more than the sum of its parts.
Like other monitoring tool vendors, Sumo Logic also started town the observability road. The SaaS company has been helping customers to monitor their applications, servers, storage, and other systems since it was founded in 2010. However, at some point, the journey toward observability nirvana took a turn.
The problems with observability stem from tough challenges inherent with each of the different observability data types, says Bill Peterson, senior director product marketing for observability at Sumo Logic.
“The original premise of observability was we’re going to combine all of that and allow you to analyze it,” Peterson says. “But what folks found out was it’s really hard.”
Sumo Logic surfaces insights based on observability data (Image courtesy Sumo Logic)
The first problem was the size of the data, Peterson says. Logs, metrics, and traces generate a veritable tsunami of data (dare we call it a data-nami), which creates storage challenges of its own. The fact that metrics are just samples of your data, rather than a full reflection of your system, is another challenge in piecing together a full picture, he says. Third, traces are completely reliant on developers to code into the application, which is often never done.
“So the original premise of observability was I was going to capture all that, to allow you to do reporting and know what’s going on,” Peterson says. “It never really happened. It never really fulfilled the promise of what folks hoped observability would be.”
While obtaining full observability into your applications through logs, metrics, and traces sounds good, it hasn’t really panned out. For example, about four out of five incidents logged in retail settings are customers letting companies know that the website isn’t working, Peterson says. That’s a strong indication of failure.
“We’re kind of flipping that model on its head and saying, no, we have to have a system that’s continuously looking at these levels, creating service maps, and identifying the breakdown,” he says. “A real-time system of insight on the health of your IT stack.”
The approach that Sumo Logic opted to take with its next generation observability solution–which it opted to call dynamic observability–is one based on logs as the ground truth. Metrics and traces are nice when they work, but they’re not adding enough to the equation at the end of the day, Peterson says.
“What we’re doing at Sumo is looking at dynamic observability from the notion that it all kind of starts and ends with the logs,” he says. “Logs are where all of the detailed, granular data on what’s happening in an application exists.”
Mo Co-Pilot aims to surface insights from log data
Sumo Logic thinks it has an advantage with logs because of the way it stores and processes them. For the most part, customers aren’t charged for ingesting logs into their SaaS-based observability solution (beyond a nominal charge). Instead, the company charges primarily for the insights generated from the logs.
Secondly, Sumo Logic supports unstructured logs, which some vendors do not. Unstructured logs, which can resemble just a series of numbers generated from some component of an application, can be given a schema upon reading into Sumo, Peterson says.
The third key element of its new dynamic observability system, which the company unveiled at the re:Invent 2024 conference in December, is the utilization of generative AI technologies to accelerate the analysis of data and the delivery of insight.
“We’re calling dynamic observability a system of insights,” Peterson said. “What we’re building on top of that is natural language processing and GenAI tools to actually create a system where you can use all of your data to monitor, observe, track, report, analyze, respond to incidents, etc. within Sumo Logic.”
At re:Invent, the company unveiled Mo Copilot, an AI copilot built on Amazon Bedrock that’s designed to assist with complex query creation, automatic generation of insights from security and performance incidents. Mo Copilot is being tested by customers and is slated to be released to GA later this year.
The hope with Mo Copilot is it will lower the barrier to observability. Instead of having to know the Sumo Logic query language, customers will be able to drive the tool using natural language. GenAI will be able to automate the query development, analysis, and response tasks, which will speed up the mean time between resolution (MTBR), Peterson says.
“What we’re doing with dynamic observability is getting ahead of all of that,” he says. “We still create live service maps and identify, hey, you just had a server go down in Indiana. But then what we’re doing is looking at that server and saying, ‘Okay, that server failed because some patch that went on it three days ago is bad, and we need to fix that patch.’
“But what we can then do is look at your entire network and based on the logs, predict what other servers are going to are going to fail over time because they also got that bad patch,” Peterson continues. “So it allows us to reduce MTBR, reduce downtime, allow companies to actually have a plan to go out and address this.”
Sumo Logic isn’t the only monitoring vendor chasing the dynamic observability goal. Elastic, Dynatrace, and Datadog, among others, are making moves in the space. As companies get more familiar using GenAI and copilots, and figure out where exactly they fit in, the potential benefits of this approach could be quite significant.
Related Items:
How Sumo Logic Turns the Event Data Tsunami into Continuous Intelligence
2025 Observability Predictions and Observations
Explosion of Observability Data from Cloud Reaches Tipping Point, Dynatrace Says
