Real-Time Analytics Dashboard Architecture: From Raw Events to Sub-Second Insights

The Problem with Traditional Dashboards

Most dashboards are batch-based: they query a database, cache the result, and refresh every 5–60 minutes. For operational use cases – water pressure monitoring, factory throughput, hospital queue management – this is unacceptably slow.

Architecture for Sub-Second Dashboards

[Events] → Kafka → Flink/Spark → ClickHouse → API → WebSocket → Browser
                                      ↓
                              [Pre-aggregated
                               materialized views]

ClickHouse: The Right Tool for OLAP at Speed

ClickHouse can aggregate billions of rows in milliseconds:

SELECT
  toStartOfMinute(event_time) AS minute,
  meter_zone,
  sum(flow_m3) AS total_flow,
  avg(pressure_bar) AS avg_pressure
FROM meter_events
WHERE event_time > now() - INTERVAL 1 HOUR
GROUP BY minute, meter_zone
ORDER BY minute DESC;

WebSocket Push vs. Polling

For real-time dashboards, always prefer server-sent events or WebSocket over polling. Polling at 1-second intervals with 1,000 concurrent users generates 1,000 requests/second – a DDoS against your own backend.

Conclusion

The stack that works for us: Kafka (ingestion) + ClickHouse (OLAP) + Redis Pub/Sub (WebSocket relay) + React (frontend). Total end-to-end latency from event to pixel: under 800ms.