HomeFind your caseOur dashboards time out as our data grows
Case · Enterprise
A common situation · not a client storyOur dashboards time out as our data grows.
Product and data teams serving analytics, inside the company or to customers, from Postgres or MySQL, or paying more each month for a cloud warehouse.
What you're probably seeing
If this sounds familiar, you're in the right place.
- Dashboards that loaded in seconds now time out, sometimes in front of customers.
- Event and log tables have grown far beyond what the database was set up for.
- Nightly rollup jobs run longer every week, and sometimes into the working day.
- Read replicas were added to cope, and now they lag.
- The warehouse bill rises with every new dashboard and every new user.
The usual ways out
What teams usually try. Each works somewhere. None works everywhere.
| The way | Works when | The catch |
|---|---|---|
| Indexes, partitioning and query tuning | A few slow queries dominate | Buys time; growth catches up |
| Materialized views and pre-built rollups | The questions are known in advance | Every new question needs a new rollup, and freshness suffers |
| Postgres extensions for time series or sharding | You want to stay on Postgres | More to operate, and analytics still competes with transactions |
| A managed warehouse (BigQuery, Snowflake) | Analysts run bursty, ad hoc queries | Cost per query, and latency for customer-facing dashboards |
| ClickHouse, self-hosted or managed | High-volume events, fast dashboards, many people querying at once | A new system to run; joins and updates work differently |
| A cache in front of the database | The same views are requested again and again | Stale data, and the slow query is still there |
What other companies did. Their published results, not ours.
We read each company's own write-up and asked one question: would our diagnostic's tests have led to the same call? We judge the approach, not the result.
| Company | What they did | What they published | Source |
|---|---|---|---|
| LangfuseOpen-source LLM observability platform processing tens of thousands of events per minute. Dashboards and filters on Postgres became painfully slow for enterprise users. Ingestion was also hitting Postgres IOPS limits. | Postgres query tuning was tried first and was not enough. They then moved tracing data to ClickHouse after proofs of concept. They also put a Redis queue and async workers in front of ingestion and stored payloads in S3. | Prompts API p99 fell from 7 seconds to 100 ms. About 90% of updates were written within 10 seconds. Dashboard targets were set (p99 under 1 s for the past week, under 4 s for longer ranges), but the text does not say whether they were met.They noted: Row updates are costly in ClickHouse. Duplicates lingered for minutes to hours, which made short-range dashboards nearly unusable until they combined FINAL with other methods. Joins exhausted node memory. Changing an ordering key meant a full rewrite that could take days. They also misjudged timelines. | From Zero to Scale: Langfuse's Infrastructure EvolutionUndated |
| LagoOpen-source usage-based billing platform ingesting millions of events per minute. Analytical queries on Postgres were blocking the ingestion queue. | Kept Postgres for app data and moved only billable events to a distributed, self-hosted ClickHouse (Altinity Operator on Kubernetes). Events arrive from Kafka through a materialized view. They compared ClickHouse Cloud, self-hosting and the operator first. | Not stated (they say only that the strategy succeeded and they have not looked back)They noted: Staying on Postgres with vectorized executors and triggers needed a lot of setup. A columnar Postgres option (Hydra) launched after their decision and might have changed it. OLAP stores handle updates and deletes poorly. | Using Clickhouse to scale an events engineFeb 2024 |
| ilertIncident management platform with customer-facing analytics on BigQuery. Dashboards took several seconds on average and up to 20 seconds for the largest customers. Repeated dashboard queries were billed again each time. | Weighed tuning BigQuery (BI Engine, reservations, materialized views) and Druid, Pinot, StarRocks and TimescaleDB. Moved to a self-run ClickHouse cluster fed by CDC through Kafka, using ReplacingMergeTree for rows that change. | P95 dashboard query latency fell from about 10 seconds to under 2 (about 5x). Per-query billing was replaced by a fixed cluster cost. Savings were not quantified.They noted: They now run replication, ZooKeeper, upgrades, backups and capacity planning themselves. The cluster is sized for peak and underused otherwise. Ported queries gave plausible but wrong results until checked against test data. Leaving Google Cloud was part of the reason. | From BigQuery to ClickHouse: How we made our analytics 5× fasterJul 2026 |
| Cloudflare (Digital Experience Monitoring team)Three-engineer team on a Zero Trust monitoring product. Inserts grew from 200/sec to 1,000/sec, and tables reached billions of rows. Plain Postgres slowed for large customers querying 7+ day ranges. | Rejected ClickHouse because its ingestion path would add about five new components, and their many small per-device batches suit MergeTree poorly. Stayed on Postgres with the TimescaleDB extension (hypertables, compression, continuous aggregates). | Benchmarks on 500 million to 1 billion rows showed 5x to 35x faster queries. One dataset compressed from 1616 GB to 49 GB (about 33x). Earlier, plain-Postgres pre-aggregates had given up to 1000x on dashboard queries.They noted: Sparse indexes needed an explicit orderby setting, which they found only after the evaluation, and the defaults were not always right. They argue every added component carries operating cost. | How TimescaleDB helped us scale analytics and reportingJul 2025 |
Our read
Reasonable call. One thing showed up late. The options chosen fit each team, but Langfuse's ClickHouse move ran into update, join and freshness costs only after the switch.
Langfuse tuned Postgres and ran proofs of concept. Even so, it found after the move that slow deduplication made short-range dashboards nearly unusable, that joins ran out of memory, and that ordering-key changes took days. A test of the real slowest queries, including updates and short time ranges, on a copy of the data would check those points before committing. ilert caught silently wrong ported queries by checking against test data, which is the same kind of check. Cloudflare weighed its write pattern and its three-person team against ClickHouse's extra components and stayed on Postgres, close to what our freshness and team-capacity tests weigh.
What we would test first: Run your slowest real queries, including updates and short time ranges, on a copy of your data on each shortlisted option, with the cost at your scale.
What the diagnostic would test
The tests that rule the wrong ways out.
On your own systems and data, read-only first. The number of tests sets the diagnostic's price, quoted before it starts.
- 1
Your twenty slowest queries, and how often each one runs.
- 2
Data volume and growth, table by table.
- 3
How fresh each dashboard's data has to be.
- 4
How many people query at the same time, at peak.
- 5
The same queries on a copy of your data, on each shortlisted option, with the cost at your scale.
- 6
What your team can run on its own after we leave.
This week, without us
Three things you can do yourself.
- 1
Turn on slow-query statistics (pg_stat_statements on Postgres) and list the top queries by total time.
- 2
Write down which dashboards customers see, and how fresh each one needs to be.
- 3
Check whether analytics queries run on the same database as your transactions.
Analytics databases: ClickHouse, Postgres and warehouses
ClickHouse Toolkit: three tools we wrote for running ClickHouse in production →
Sound like you? Bring it to a 30-minute call. We'll tell you which tests your case needs, and what they cost.
This page describes what teams in this situation commonly face. It is not a client story.