Founded in 2005, Box is a cloud content management company that securely connects enterprises to their people, information, and applications. Over 115,000 enterprises trust Box to house their most critical content.

“We operate at a scale that is large. We're approaching an exabyte of data,” explains Ben Kus, Box’s CTO. Building and maintaining a platform of this nature presents many important challenges that require a strong engineering culture.

One of these challenges is observability. To deliver the experience users expect, the Box team has to ensure content remains safe, secure, and available. “Observability is one of the most critical things when you have an enterprise-grade solution like Box,” says Kus. Box uses several observability tools throughout the organization to support these needs.

Exponential data growth drives complexity and cost

Over the past few years, Box experienced exponential telemetry growth as a byproduct of supporting so many customers and running large-scale, distributed systems. By 2022, the company was generating several hundred terabytes of log data every day. With this level of growth came challenges for Box’s engineering team.

First, it was extremely difficult for engineers to grasp the data being created. Kus notes, “Of course, we’re supposed to know what logs we generate and what services make these logs. But, our systems are so sophisticated and complex – it makes it very hard.”

Second, capturing such large data volumes drove up Box’s observability costs. Lastly, as a byproduct of the first two challenges, the team began investing serious time and effort trying to determine the most cost-effective format and destination for each dataset.

In other words, Box’s engineering team was forced to shift focus away from their core responsibilities and onto managing telemetry. “We don't want to be in business to make our logs more efficient,” Kus explains. “We want to be in business to provide new capabilities to our customers.”

Edge Delta helps Box future-proof its observability strategy

In 2022, Box began evaluating Edge Delta. Soon into the engagement, the value of Edge Delta became apparent. “This is not just about doing what you used to do in the past, and doing it a little bit better,” Kus remarks. “This is a new way to see this world of how we collect and manage our observability pipelines.”

Central management simplifies observability pipelines

One of the specific improvements Kus and his team experienced was gaining a central view to manage log collection, pre-processing, and routing across all data sources. Having this view allowed different teams to understand what data the rest of the organization was creating and analyzing.

“We had grown to the point of complexity where it was hard to get a handle on everything that was going on,” Kus says. “Edge Delta reduced all of the complexity, from trying to figure it out amongst many different teams and engineers, into one area where we could manage centrally.”

Edge processing enables new levels of cost efficiency

By pre-processing data at the edge, Box was able to more easily identify and capture the data they truly needed. “The key for us with Edge Delta – and also just across our observability strategy – is to make sure that we had the right control points.” Kus continues, “We want to make sure that we're collecting the right things – the right metrics, the right logs, the right traces and doing it efficiently.”

By adopting this approach, Box has been able to reduce it’s observability spend. Across some datasets, Box has seen anywhere from an 80% to a 99% reduction in log ingestion.

Automation removes monitoring toil

Lastly, Edge Delta has helped the Box team get back to focusing on their core product. “When we got Edge Delta, we were able to spend dramatically less time on managing logs because the tool handled a lot of it for us.” Kus continues, “And even if we needed to change something, it was a much smaller project.”

Box brings costs and operational efficiency into the future

By pushing data processing upstream, Edge Delta has helped Box build an observability strategy that will provide both cost and operational efficiency moving forward. As Kus puts it, “For me Edge Delta means we have a more capable, standardized, efficient infrastructure around how we handle observability. And if we don't have that right, then it's hard to do anything else.”