Do we need Istio to do progressive delivery?

No. Argo Rollouts supports NGINX Ingress and SMI too. We picked Istio for precise traffic splitting and because the client already needed mTLS and traffic policies. If you’re on EKS with AWS Load Balancer Controller, NGINX can be simpler.

Won’t canaries double our costs?

During the rollout window you run extra pods, typically a 10–30% overhead for minutes. In our case it added ~2–3% to monthly compute. We saved far more by avoiding Sev1s and overtime.

How do you handle database migrations?

Use expand-contract: add nullable columns first, ship code that can read/write both (behind flags), then flip flags and backfill, and finally drop old paths. Treat schema changes like code—progressively rolled out and reversible.

What if our metrics are noisy?

Start with coarse thresholds and short intervals. Use per-route labels and compare canary vs stable deltas. You can also use Argo Rollouts’ webhooks to call a custom scorer if PromQL isn’t enough.

Case-studies · Oct 13, 2025 · 9 minute read

The Canary That Stopped the Friday Night Pager: Progressive Delivery That Cut Change Failures by 78%

A fintech SaaS was burning its error budget every sprint. We rebuilt their pipeline around progressive delivery with Argo Rollouts, Istio, Prometheus, and LaunchDarkly. Change failure rate dropped from 23% to 5%, MTTR fell from 4 hours to 45 minutes, and deploys went from weekly to daily—without blowing up compliance.

Alex Reeves

Principal Consultant, GitPlumbers

20 years building and rescuing software at scale—ex-Spotify platform, led SRE at a unicorn marketplace, and has battle scars from monolith stranglings and AI-assisted regressions. Alex helps teams ship faster without burning the pager.

We didn’t make engineers “more careful.” We made rollbacks automatic, and suddenly Friday nights got quiet.

Back to all posts

The Canary That Stopped the Friday Night Pager: Progressive Delivery That Cut Change Failures by 78%

Key takeaways

Implementation checklist