How do I know if a service is safe to horizontally scale as stateless?

Check for externalized state (sessions, cache, files), idempotent handlers, and zero dependency on local disk or in-memory sticky data. If a single request can land on any replica and succeed, you’re stateless enough.

What’s the first move to scale Postgres without a full re-architecture?

Introduce PgBouncer in transaction mode, add read replicas, and route read-only queries there. This resolves connection storms and offloads 30–60% of read traffic in most CRUD apps.

Should I scale on CPU or latency?

Use CPU as a guardrail, but scale primary on load proxies users feel: RPS, in-flight requests, and backlog/lag. Latency is noisy for direct autoscaling but perfect for SLOs and alerting.

When do I need to shard writes?

Only after you’ve exhausted read replicas, pooling, and query/index tuning. If write QPS saturates a single primary or you need multi-region writes with low RTO, consider Vitess/Citus or a distributed SQL option like CockroachDB.

Performance-optimization · Oct 16, 2025 · 10 minute read

The Autoscaler That Blew Our SLO: Horizontal Scale for Stateless vs Stateful That Actually Works

If your p95 slipped while the cluster doubled in size, you scaled the wrong thing. Here’s how we design horizontal scale that protects user-facing latency and revenue—across stateless frontends and the stateful beasts behind them.

Alex Rivera

Principal Engineer, GitPlumbers

Alex has led scale-ups at marketplaces, banks, and SaaS platforms for 20 years—through monoliths, microservices, and now AI-fueled traffic spikes. Ex-AWS, early SRE at a unicorn you’ve definitely doomscrolled about.

If your autoscaler isn’t anchored to SLOs, it’s just an expensive random number generator.

Back to all posts

The Autoscaler That Blew Our SLO: Horizontal Scale for Stateless vs Stateful That Actually Works

Key takeaways

Implementation checklist