How do I pick a realistic latency target?

Start from observed conversion elasticity and device mix. If mobile dominates, target lower p95s on API endpoints (600–800ms) and 2–2.5s LCP for primary pages. Use current p95 as a baseline and improve in 10–20% increments rather than jumping to unrealistic p99s.

Should we optimize p95 or p99?

Optimize p95 for day-to-day experience and use p99 to detect tail risk. Most revenue correlates with p95; p99 is a stability signal. If p99 drives paging, add circuit breakers, timeouts, and retries to cap tails without gold-plating the entire stack.

Do we need a FinOps tool to measure cost per request?

You need cost allocation data, not necessarily a pricey tool. Start with Cloud CUR + tags and Kubecost. Export to Prometheus and compute CPR alongside your latency metrics. Upgrade tooling later if needed.

Will autoscaling fix my latency?

Only if compute is the bottleneck. In practice, it often isn’t. Autoscaling amplifies whatever you have: slow queries, chatty calls, cache misses. Fix those first, then let HPA/VPA keep you right-sized.

Is serverless cheaper for this?

Sometimes. Serverless shines for spiky, IO-heavy tasks where you can keep functions warm and avoid cross-service chatter. But for hot, high-RPS paths, containerized services with tuned autoscaling and caching are typically cheaper at scale.

Performance-optimization · Oct 21, 2025 · 10 minute read

The Latency Budget That Cut Our Cloud Bill 38% Without Slowing Users

A battle-tested framework to tie p95s to dollars, not dashboards — and ship faster without torching your cloud budget.

Alec Donovan

Partner, Performance & Reliability at GitPlumbers

20 years of scaling web and data systems at Shopify, Twitter, and a couple of unicorns you’ve definitely rage-quit from. Now fixing AI-assisted and legacy software so teams can ship safely.

If your SLOs don’t drive a go/no-go decision in a deploy, they’re just wall art.

Back to all posts

The Latency Budget That Cut Our Cloud Bill 38% Without Slowing Users

Key takeaways

Implementation checklist