Inter Reduces Observability Costs by 60% and Gives Teams More Control Over Data

Overview of Inter

Inter was founded in 1994. Then called Intermedium, the company supported real estate credit operations, such as loans and mortgages. In 2015, Inter pivoted, becoming the first 100% digital bank in Brazil. It has since become a “super app,” offering everything from banking and investing to e-commerce and even food delivery.

Since this shift, the company has seen massive growth. In 2016, Inter supported over 80,000 customers. Today, that number has ballooned to over 28 million customers. Their team consists of 3,400 employees, 1,000 of which are engineers.

Rafael Bruno de Almeida is a Senior Tech Manager on Inter’s infrastructure team. As he puts it, “I’ve been with the company for six years, but it feels like 60 given the growth we’ve seen.”

The Challenge: Centralizing Massive Data Volumes Increases Costs and Slows Analysis

Almeida and his team support observability for all of Inter’s internal users. This includes infrastructure, security, software development, and application teams. Each team uses different tools for analyzing their observability data, including Amazon CloudWatch, New Relic, and Splunk.

Before adopting Edge Delta, the company faced a couple of challenges related to observability. First, Inter had difficulty deriving real-time insights while centralizing the massive volume of data they generated. To solve this challenge, Almeida and his team sought a new kind of observability architecture.

“We learned from an early age that centralizing all data is not good in a high-density environment,” explains Almeida. “The platform becomes so big and heavy, and trying to batch process all the data just doesn’t work. When something breaks, everyone loses the information they need.”

Second, the team looked for a solution that would help them reduce observability costs. Naturally, observability costs grew with Inter’s data volume. As Almeida frames it, “To sell a new solution internally, we needed to prove we could save money.”

The Solution: A Distributed Approach to Observability

Once Almeida and his team began evaluating observability tools, they were immediately drawn to Edge Delta’s distributed architecture. All of the software Inter builds in-house runs on distributed, Kubernetes-based architectures because the team values both the efficiency and stability of doing so. Emulating this approach with observability made a lot of sense for the company. “With everything, we’re thinking about how we can take a more asynchronous or real-time approach. Edge Delta helps us do that,” explains Almeida.

From here, Almeida needed to understand how Edge Delta could help Inter support its different teams. Today, Edge Delta enables two unique use cases:

• Log Search & Analytics: Serving as a standalone tool to help teams derive real-time insights from logs, and run ad hoc queries.
• Observability Pipelines: Helping teams collect, pre-process, and route data to other observability platforms.

Almeida saw the opportunity to pursue both use cases.

For immediate cost savings, the company would replace Amazon CloudWatch – the infrastructure team's existing log management tool – with Edge Delta Log Search & Analytics. In addition to saving money, Edge Delta helped the infrastructure team troubleshoot issues more easily. “Edge Delta brings us good alerts and insights in a simpler manner to help us troubleshoot the whole issue,” notes Almeida.

To help the security, development, and application teams gain more control over their data, Inter adopted Edge Delta Observability Pipelines. “I wanted to give other teams the flexibility to choose where they analyze their data. Edge Delta gives us that flexibility,” explains Almeida.

Before adopting Edge Delta, the different teams often ingested a significant amount of noisy or low-value data. Edge Delta has helped the company solve this challenge through the Observability Pipelines use case.

As Edge Delta collects and analyzes data, it also routes a copy of the raw data to archive storage (in this case, Amazon S3). This capability enables Inter to access the exact subset of data they need on every platform, instead of ingesting a ton of noise.

“The team can simply access the information they need from the S3 bucket from a particular point in time.” Almeida continues, “They can build their own filters and see the piece of information that they like.”

Both initiatives have been a huge success. Inter has seen over 60% savings in observability costs since adopting Edge Delta.

What’s Next

Given the success of this initial project, Inter is planning additional use cases with Edge Delta. In the next phase, Inter will explore data optimization for the other observability platforms. “We’d like to transform logs into metrics to build visualizations while sending less data to those tools,” notes Almeida.

He summarizes the project on a high note: “Edge Delta is a great company with great leadership. You’re going to talk to them, and they’ll listen,” remarked Almeida. “Plus, they support your business in a modern, distributed way.”