Most observability vendors use a “centralize-then-analyze” approach, meaning your data must be centralized in their platform before users can search or run analytics on it. This approach worked well before the rise of cloud and microservices. However, now, applications now generate significantly more log data. This creates challenges, such as:
High Costs: Observability platforms are becoming prohibitively expensive and a simple mistake, like leaving debug logs on, can cause a massive overage.
Limited Visibility: Teams must sample, filter, or drop loglines to contain costs, inhibiting visibility.
Poor Performance: As datasets continue to grow, it becomes increasingly difficult to support real-time analytics.