How do I pick SLO targets without a ton of historical data?

Start with your customer promise and incident history. Pick conservative targets per tier (e.g., Tier-1 at 99.9%), then monitor the actual error/latency distributions for 2–4 weeks. Calibrate based on real burn; it’s normal to revise early.

What’s the fastest way to implement burn alerts if we don’t use Sloth?

Create ‘good’ and ‘bad’ SLI counters in Prometheus and use the multi-window thresholds shown here. You can wire them manually into Alertmanager or use Sloth to generate the rules from a simple YAML SLO spec.

How do error budgets apply to batch jobs?

Define SLOs in terms of timeliness (e.g., 99% of runs finish within 60 minutes of schedule) and freshness (data no more than 24 hours stale). Burn when deadlines slip, and gate rollouts on those SLIs the same way you would for APIs.

What about third-party SaaS dependencies?

Model them as dependencies with their own SLIs and an allocated portion of your budget. If a vendor burns your budget, you have leverage: mitigation plans, credits, or architecture changes (caching, bulkheads).

How do I prevent alert fatigue with more indicators?

Use indicators as inputs to SLO burn, not separate paging alerts. Page on burn, annotate with leading indicators, and send non-paging heads-up alerts to a low-noise channel for trend watching.

Reliability-observability · Oct 4, 2025 · 10 minute read

Stop Burning Budgets Blind: Designing Error Budget Allocation by Service Tier (and Wiring It to Rollouts)

Vanity SLOs won’t save you. Here’s how to tier your services, watch the right leading indicators, and plug error budgets into triage and deployment automation.

Alex Mercer

Partner / Reliability & Observability

20 years building and rescuing distributed systems at scale. Ex-Adobe, ex-Twitter, and the person companies call when SLOs turn into politics. At GitPlumbers, Alex leads engagements that turn telemetry into safer shipping.

If your SLOs don’t change deployment behavior, they’re just compliance theater.

Back to all posts

Stop Burning Budgets Blind: Designing Error Budget Allocation by Service Tier (and Wiring It to Rollouts)

Key takeaways

Implementation checklist