Compound Nodes simplify observability pipelines by allowing you to manage several processors as a single node. This feature is currently in beta – try it out today.
Today, we’re excited to announce Compound Nodes – a new feature that simplifies observability pipeline workflows. Compound Nodes enable you to group together individual components within Visual Pipelines. As a result, you can avoid managing several discrete processors for each subset of data.
A simpler pipelines experience
Often, observability pipelines are one massive configuration. Within the configuration, users apply several different processors for each subset of data passing through the pipeline. For example, you may take one dataset and…
Mask PII within the loglines
Capture ERROR and WARN level logs
Extract 400s and 500s as a metric
Summarize all data into patterns
Filter out DEBUG-level logs
…and more.
Before, each of these processors would exist as its own node within the Visual Pipelines interface. As a byproduct, observability pipelines could quickly become complex and hard to understand. Especially when you’re a large team, processing data from several different sources and routing it to many destinations.
Compound Nodes enable you to combine multiple processors into a single node. In other words, you can manage the components of your pipeline in bite-sized functions versus one big monolith.
Here's an example of what a pipeline might look like BEFORE Compound Nodes...
...and here's one AFTER using Compound Nodes:
Much cleaner! We designed Compound Nodes with enterprise organizations in mind. This feature makes it easier to support and scale pipelines spanning hundreds of teams across thousands of data sources.
Create in one place, reuse anywhere
Additionally, Compound Nodes enable reusability. Our Node Library allows teams to build and store Compound Nodes in a catalog that’s accessible to other teams. Perhaps you want to pre-process Nginx logs in the same way across multiple pipelines. Now, you can do so in a matter of clicks.
Manage observability functions by team
Compound nodes also provide an easy way for pipeline administrators to group observability functions by application teams or business units. With compound nodes, you can denote specific sub-pipelines that different teams are responsible for. So they can meet their observability needs without impacting others’ work or having to understand every area of the pipeline.
Moving forward, we will provide several more capabilities related to individual team ownership and management. Stay tuned as we move closer to general availability.
Get Started Today
Compound Nodes are available today in beta. If you already have an Edge Delta account, you can access Compound Nodes by visiting https://app.edgedelta.com/pipeline/compound-nodes.
