🤖 Meet OnCall AI, our observability copilot that makes troubleshooting easy. Read announcement.

Skip to content
Announcements

Meet Hybrid Log Search: Query from SaaS, Keep Data in Your Environment

Jan 23, 2024 / 5 minute read

Introducing Hybrid Log Search – a new logging experience that enables you to run queries from Edge Delta SaaS, without data leaving your environment. 

.

Today, Edge Delta launches Hybrid Log Search. Now, you can use Edge Delta’s SaaS app to query data that resides within your own data store.

Interested in participating in the Private Beta? Sign up here.


Log management products historically fall into one of two categories: SaaS and self-hosted. SaaS logging tools provide superior management ease. However, teams in regulated industries often have to keep data within their environment.

Hybrid Log Search reimagines traditional architectures to deliver the "best of both worlds." More specifically, it combines the management ease of SaaS with the data protection and cost-efficiency of self-hosting:

  • All the log search components live within your environment to ensure data never leaves your “four walls.”

  • We’ll confirm you’re a Hybrid Log Search user from our web interface. From there, all requests are routed to a local API only.

  • When you authenticate from our web interface, you’ll also receive the latest and greatest front-end updates.

  • The only thing that leaves your environment is health data, which we’ll use to help simplify platform management.

Deploying Hybrid Log Search is as simple as running a couple of commands. From there, you can query data that resides in your own cloud data store from Edge Delta's SaaS interface.

Why Self-Hosted and Federated Search Products Don’t Cut It

Self-Hosted Platforms Are Complex to Manage and Scale

There are two main reasons to self-host a log management platform:

  • If you work in a highly regulated industry, you likely can't forward logs to a SaaS observability tool.

  • Once your data exceeds a certain threshold, it becomes cost-prohibitive to centralize logs in a SaaS platform.

Organizations that meet one or both of these requirements typically use self-hosted logging platforms. However, there are several shortcomings to these platforms as well.

Self-hosted logging tools are complex and time-consuming to deploy, manage, and scale. They require dedicated resources to maintain availability during peak demand. Plus, if you generate lots of data, you may need to reduce retention to work within resource constraints. This means you sacrifice long-term visibility.

Federated Search Products Can’t Support Most Troubleshooting Use Cases

Other SaaS vendors offer federated search or "search in place" products to drive cost-effectiveness. These products don’t require you to deploy search infrastructure within your environment. Instead, they integrate with your environment and return queries to the vendor-hosted backend. As a result, you'll incur latency and poor performance.

Federated search products may be helpful for running one-off queries at a very low cost. But, they aren’t designed for frequently queried datasets that are essential for troubleshooting.

A Simple, Secure Search Experience

.
Edge Delta's hybrid deployment model combines the benefits of SaaS and self-hosted log management platforms.

Edge Delta Hybrid Log Search solves the challenges listed above. We designed this product specifically for medium-to-large enterprises working in highly regulated industries.

Our goal is to provide a new log search experience. One that protects your data and removes the management burden of legacy self-hosted offerings.

  • Control where your data resides to meet security and compliance requirements.

  • Reduce TCO by leveraging our optimized log storage and eliminating egress charges.

  • Minimize ongoing maintenance via support and consulting.

Additionally, you will realize the other benefits inherent to the Edge Delta platform. This includes adopting the OpenTelemetry schema out-of-the-box.

How it Works

.
This reference architecture depicts the components and data flow of an Edge Delta Hybrid Log Search deployment.

You can deploy Edge Delta Hybrid Log Search using a Terraform package and a Helm chart. Here’s an overview of the components you’ll deploy:

Log Search Service

The Edge Delta Log Search service consists of an on-premise Edge Delta API and a hybrid token. The API enables communication between the Log Search service running in your environment and your other local resources (log store, Amazon S3, etc.).

Log Data Store

The Terraform package also includes a column-oriented log data store. This data store will read queries that are sent from the API. Then it will return the results from an Amazon S3 bucket.

Amazon S3

Again, all of your logs will reside in your own Amazon S3 resources (we’ll have more storage options in the future). The deployment includes two Amazon S3 buckets – one for storing your log files and one for database operations.

Other Resources

In addition to these resources, you will deploy other components for health monitoring, data collection, compression, and more. You can read more about the deployment in this whitepaper.

"How will I ensure my data is private?"

Post-deployment, Edge Delta runs as a single-page application in your browser and verifies you are a Hybrid Log Search customer. Once we confirm you are a Hybrid Log Search customer, all subsequent requests are sent from your browser to the on-premise API only. In other words, no data is ever returned to our SaaS backend.

Save Your Spot in the Private Beta

Edge Delta Hybrid Log Search combines the ease and scalability of SaaS with the data protection and cost of self-hosting. If you work in a highly regulated or generate large volumes of data, we highly encourage you to try Hybrid Log Search for yourself.

You can sign up for the Private Beta here.

.

Riley Peronto

Director of Product Marketing

Filed Under

Share

Related Posts

Stay in Touch

Sign up for our newsletter to be the first to know about new articles.