How do I know if a service is safe to horizontally scale as stateless?

Check for externalized state (sessions, cache, files), idempotent handlers, and zero dependency on local disk or in-memory sticky data. If a single request can land on any replica and succeed, you’re stateless enough.

What’s the first move to scale Postgres without a full re-architecture?

Introduce PgBouncer in transaction mode, add read replicas, and route read-only queries there. This resolves connection storms and offloads 30–60% of read traffic in most CRUD apps.

Should I scale on CPU or latency?

Use CPU as a guardrail, but scale primary on load proxies users feel: RPS, in-flight requests, and backlog/lag. Latency is noisy for direct autoscaling but perfect for SLOs and alerting.

When do I need to shard writes?

Only after you’ve exhausted read replicas, pooling, and query/index tuning. If write QPS saturates a single primary or you need multi-region writes with low RTO, consider Vitess/Citus or a distributed SQL option like CockroachDB.

Performance-optimization · Oct 16, 2025 · 10 minute read

The Autoscaler That Blew Our SLO: Horizontal Scale for Stateless vs Stateful That Actually Works

If your p95 slipped while the cluster doubled in size, you scaled the wrong thing. Here’s how we design horizontal scale that protects user-facing latency and revenue—across stateless frontends and the stateful beasts behind them.

Back to all posts

The Autoscaler That Blew Our SLO: Horizontal Scale for Stateless vs Stateful That Actually Works

Key takeaways

Implementation checklist