Analyze Every Logline with Processors

We all know that log data contains tons of insight. Even more so when you can extract metrics from the log content, things like response times or status code. Doing so not only provides real-time visibility into the behavior of your applications but also allows you to track behavior over time and alert on anything abnormal.

In this video, we're going to walk through Edge Delta's processors and explain how they can deliver this new analytical value from your log data. But before we start, what is a processor? An Edge Delta processor is a regex-based monitoring logic that analyzes your log data. It does so in an automated manner at the agent level. Edge Delta supports several different processor types. In this video, we're going to focus on dimensional and cluster processors.

Our dimensional processors allow you to extract dimensions from your logs and convert them into time series metrics. For example, one of the KPIs I care about is API response time. Now, let's go into my Agent Configurations and create a processor that tracks this dimension over time. Here in Agent Configuration, we see the definition of our processors as well as how each and every processor is used in our agent's workflow, to better understand and to start monitoring your particular KPI, in our example here, we use response time. We first need to define the name of the processor. We use API response time here, where we're checking for large spikes in our API response time. We also specify a pattern, this is super important. It gives us the capability of extracting where response time is occurring from our log events. Once we've defined the processor, all we need to do is reference that processor in a workflow where we can quickly and easily begin to have our agent analyze our data and create our metric in real time.

Now, let's go to the metrics explorer. Edge Delta maps out the response times over time, so I can easily track how this KPI fluctuates and if it's behaving as we'd expect. Edge Delta automatically baselines these metrics and alerts for your team when anomalous behavior occurs. Here we have an example of Edge Delta's anomaly detection in action, where the agent has identified an anomaly in the API response time metric. Every time Edge Delta recognizes an anomaly in your environment for any given application or service, in this example here, it's the access service, whenever we see an anomaly, we'll let you know and alert you on the relevant data.

These metrics also come in handy if your team relies on metrics-based dashboards for real-time monitoring. Typically, in these situations, log data is searched reactively when issues occur, helping with investigations, but offering very little real-time value. Now, you can pull in insights from your log data into your metrics dashboard so you get the full analytical value of these data sets.

Our cluster processor first finds common patterns, then it groups similar log events together. If you're using Edge Delta to monitor your applications, these patterns help you reduce the noise of your log data and more quickly understand each event. If you're using Edge Delta to create an observability pipeline, these patterns can be used to control the data you route downstream. For example, these three info logs take up 30% of my total log volume. It's probably not necessary to stream all that raw data into my observability platform. Edge Delta also supports Top-K processors to show the frequency of log events and the ratio processors to show the proportion of similar events against non-similar events.

Explore insights like these by deploying Edge Delta for free at edgedelta.com.

(logo whooshing in)