🤖 Meet OnCall AI, our observability copilot that makes troubleshooting easy. Read announcement.

Skip to content
Product

Introducing Compound Nodes: Group Together Processors for Easier Management

Feb 29, 2024 / 3 minute read

Learn how you can simplify observability pipeline management with Compound Nodes. The new feature combines multiple processors into a single node.

.

Compound Nodes simplify observability pipelines by allowing you to manage several processors as a single node. This feature is currently in beta – try it out today.


Today, we’re excited to announce Compound Nodes – a new feature that simplifies observability pipeline workflows. Compound Nodes enable you to group together individual components within Visual Pipelines. As a result, you can avoid managing several discrete processors for each subset of data.

A simpler pipelines experience

Often, observability pipelines are one massive configuration. Within the configuration, users apply several different processors for each subset of data passing through the pipeline. For example, you may take one dataset and…

  • Mask PII within the loglines

  • Capture ERROR and WARN level logs

  • Extract 400s and 500s as a metric

  • Summarize all data into patterns

  • Filter out DEBUG-level logs

…and more.

Before, each of these processors would exist as its own node within the Visual Pipelines interface. As a byproduct, observability pipelines could quickly become complex and hard to understand. Especially when you’re a large team, processing data from several different sources and routing it to many destinations.

Compound Nodes enable you to combine multiple processors into a single node. In other words, you can manage the components of your pipeline in bite-sized functions versus one big monolith.

Here's an example of what a pipeline might look like BEFORE Compound Nodes...

.

...and here's one AFTER using Compound Nodes:

.

Much cleaner! We designed Compound Nodes with enterprise organizations in mind. This feature makes it easier to support and scale pipelines spanning hundreds of teams across thousands of data sources.

Create in one place, reuse anywhere

Additionally, Compound Nodes enable reusability. Our Node Library allows teams to build and store Compound Nodes in a catalog that’s accessible to other teams. Perhaps you want to pre-process Nginx logs in the same way across multiple pipelines. Now, you can do so in a matter of clicks.

Manage observability functions by team

Compound nodes also provide an easy way for pipeline administrators to group observability functions by application teams or business units. With compound nodes, you can denote specific sub-pipelines that different teams are responsible for. So they can meet their observability needs without impacting others’ work or having to understand every area of the pipeline.

Moving forward, we will provide several more capabilities related to individual team ownership and management. Stay tuned as we move closer to general availability.

Get Started Today

Compound Nodes are available today in beta. If you already have an Edge Delta account, you can access Compound Nodes by visiting https://app.edgedelta.com/pipeline/compound-nodes.

.

Riley Peronto

Director of Product Marketing

.

Alan Murphy

Principal Product Manager

Filed Under

Share

Stay in Touch

Sign up for our newsletter to be the first to know about new articles.