Box Controls Observability Cost and Complexity at Scale

Founded in 2005, Box is a cloud content management company that securely connects enterprises to their people, information, and applications. Over 115,000 enterprises trust Box to house their most critical content.

“We operate at a scale that is large. We're approaching an exabyte of data,” explains Ben Kus, Box’s CTO. Building and maintaining a platform of this nature presents many challenges that require a strong engineering culture.

One of these issues is observability. To deliver the experience users expect, the Box team ensures content remains safe, secure, and available. “Observability is one of the most critical things when you have an enterprise-grade solution like Box,” says Kus. Box uses several observability tools throughout the organization to support these needs.

Exponential data growth drives complexity and cost

Over the past few years, Box experienced exponential telemetry growth as a byproduct of supporting so many customers and running large-scale, distributed systems. By 2022, the company was generating several hundred terabytes of log data every day, which created data management issues for Box’s engineering team.

First, it was extremely difficult for engineers to grasp the data being created. “Of course, we’re supposed to know what logs we generate and what services make these logs," Kus says. "But, our systems are so sophisticated and complex that it makes it very hard.”

Second, capturing such large data volumes was driving up Box’s observability costs. Lastly, as a byproduct of the first two challenges, the team began investing serious time and effort trying to determine the most cost-effective format and destination for each dataset.

As a result, Box’s engineering team had to shift too much of its focus onto managing telemetry and away from their core responsibilities. “We don't want to be in business to make our logs more efficient,” Kus explains. “We want to be in business to provide new capabilities to our customers.”

Edge Delta helps Box future-proof its observability strategy

In 2022, Box began evaluating Edge Delta. Soon into the engagement, the value of Edge Delta became apparent. “This is not just about doing what you used to do in the past, and doing it a little bit better,” Kus remarks. “This is a new way to see this world of how we collect and manage our observability pipelines.”

Central management simplifies observability pipelines

One of the specific improvements Kus and his team experienced was gaining a central view to manage log collection, preprocessing, and routing across all data sources. Having this view allowed different teams to understand what data the rest of the organization was creating and analyzing.

“We had grown to the point of complexity where it was hard to get a handle on everything that was going on,” Kus says. “Edge Delta reduced all of the complexity, from trying to figure it out amongst many different teams and engineers, into one area where we could manage centrally.”

Edge processing enables new levels of cost efficiency

By preprocessing data at the edge, Box was able to more easily identify, capture, and prioritize the data they really needed. “The key for us with Edge Delta — and also just across our observability strategy — is to make sure that we had the right control points,” Kus says. “We want to make sure we're collecting the right things — the right metrics, the right logs, the right traces — and doing it efficiently.”

By adopting this approach, Box has been able to reduce it’s observability spend. Across some datasets, Box has seen anywhere from an 80% to a 99% reduction in log ingestion.

Automation removes monitoring toil

Along with the monetary benefits, Edge Delta has also helped the Box team get back to focusing on its core product. “When we got Edge Delta, we were able to spend dramatically less time on managing logs because the tool handled a lot of it for us,” Kus says. “And even if we needed to change something, it was a much smaller project.”

Box brings costs and operational efficiency into the future

By pushing data processing upstream, Edge Delta has helped Box build an observability strategy that will provide both cost and operational efficiency moving forward.

“Edge Delta means we have a more capable, standardized, efficient infrastructure around how we handle observability," Kus says. "And if we don't have that right, then it's hard to do anything else.”