Observability teams need to structure and refine their telemetry data to enable effective downstream analysis, but rising data volumes from numerous sources make this process incredibly complex. Each distinct data stream requires its own set of unique processing steps, which is difficult to manage across highly distributed cloud-native environments.
Edge Delta addresses this challenge by providing AI-powered processing recommendations when data is in motion and at rest. First, our intelligent Telemetry Pipelines automatically provide pre-index recommendations for parsing, enriching, masking, and optimizing data in flight. Then, our Observability Platform identifies low-value data streams and suggests targeted reduction strategies to reduce volume (and associated costs) without impacting visibility.
Edge Delta’s new intelligent Recommendations Feed brings all of these processing recommendations together in a single, holistic view, making it easier for teams to manage and iteratively act on recommendations across their entire infrastructure — at any scale.
What Is the Recommendations Feed?
Edge Delta’s Recommendations Feed aggregates all pipeline-related processing recommendations generated across your environment into an intuitive, unified display. This feed enables teams to reduce data volumes and costs, prevent unmasked PII from moving downstream, and enforce telemetry best practices across their infrastructure without switching contexts. It can be filtered by recommendation category, type, associated pipeline, and state, enabling teams to prioritize the changes that deliver the most value for their organization:
Each recommendation targets a specific data stream, and includes an embedded “Create Processor” button to instantly implement the processing change within the appropriate pipeline:
Once this button is clicked, Edge Delta automatically opens the associated pipeline, adds the recommended processor, and applies the change in preview mode. Users can then verify the expected behavior using Live Capture and officially deploy the change:
The Recommendation Lifecycle
Each recommendation in the feed goes through a defined lifecycle that helps users track progress and take timely action. Here’s what each state means and how it affects your workflow:
- New:
- Newly generated recommendations that haven’t been viewed yet are marked with a blue dot.
- Pro Tip: Check for new recommendations regularly, as they often represent new optimization opportunities based on recent traffic patterns.
- Active:
- Once a user opens or interacts with a recommendation, it transitions to the “Active” state. This indicates that the recommendation is under consideration but has not yet been applied or dismissed.
- Note: Active recommendations expire after seven days to ensure the feed stays relevant and manageable. If you don’t act on them, they’ll automatically be removed from the feed.
- Configured:
- When you apply a recommendation by adding the suggested processor to a pipeline, it transitions to the “Configured” state. This gives you visibility into which suggestions you’ve implemented.
- Pro tip: Use the list of “Configured” recommendations to track your optimization efforts over time.
- Dismissed:
- If a recommendation doesn’t apply to your use case, you can dismiss it. It won’t appear in your “Active” feed, but it will remain accessible under the “Dismissed” tab if you want to revisit it later.
- Pro Tip: Dismiss similar low-impact recommendations in bulk to clean up your feed.
A Closer Look at Edge Delta’s Intelligent Recommendations
The Recommendations Feed includes recommendations generated by two distinct parts of the Edge Delta Platform.
In-Pipeline Recommendations
Edge Delta’s Telemetry Pipelines continuously analyze data in transit and generate intelligent, real-time processing recommendations. By examining payload structure and metadata, the pipelines suggest contextually relevant processors tailored to the data. These recommendations surface directly within Live Capture, enabling teams to enforce compliance, reduce data volumes, and enrich metadata pre-index:
Pattern-Based Recommendations
The Edge Delta Observability Platform generates intelligent filter and sample recommendations on low-value, high-volume log data at rest based on log pattern analysis. The system tags log patterns indexed in our Platform that constitute at least 1% of the total log volume from a given source node. Each tagged pattern is then deconstructed, hashed, and analyzed without exposing any raw log content. An AI model scores these patterns based on factors such as overall volume, recurrence, usefulness, and data sensitivity, and makes a reduction recommendation (i.e., filter or sample) if appropriate.
For example, a repetitive debug log representing 12% of a service’s volume might trigger a “Filter Out” recommendation to reduce ingestion costs:
To enable this functionality, ensure your pipelines include Log to Pattern processors. Edge Delta will automatically recommend adding a Log to Pattern processor to streams that currently lack one.
Conclusion
Modern observability depends on real-time data processing to power downstream analysis and ensure data quality at scale. But as data volumes grow and sources diversify, manually managing these processing steps becomes more complex and error-prone.
With the launch of Edge Delta’s new Recommendations Feed, teams gain a centralized view of every intelligent processing recommendation across their infrastructure. This makes it easy to manage and act on organization-wide optimization opportunities in just a few clicks, which leads to enhanced data quality, reduced costs, and improved compliance — with minimal overhead.
Want to explore our Recommendations Feed for yourself? Check out our free Playground environment, or book a demo with a technical expert.
