How do we avoid flaky performance tests in CI?

Stabilize the environment: fixed CPU/mem runners, network emulation (e.g., Chrome’s ‘slow 4G’), seeded data, warm caches, and 3–5 Lighthouse runs with median selection. For k6, use steady-state stages and minimum sample sizes. Fail only on stable metrics (p95) with small tolerances (e.g., 3–5%).

Synthetic (Lighthouse/k6) or RUM (real-user monitoring)?

Both. Synthetic catches regressions pre-merge and pre-prod. RUM tells you what real devices and networks feel. Use synthetic in CI; use RUM to gate canaries and monitor SLOs. Automate decisions in both places.

Can we run perf checks on every PR without blowing up build times?

Yes: run a fast Lighthouse (1–2 URLs, 3 runs) and a k6 smoke (2–3 minutes). Nightly, run deeper scenarios. Parallelize jobs. If builds are still slow, only block on deltas > N% and let small signals warn but not fail.

What about backend services without a UI?

Use k6/Gatling with thresholds on `http_req_duration` and `error_rate`. Add OpenTelemetry spans and Prometheus histograms. Gate merges on p95. In prod, canary with Argo Rollouts + Prometheus queries.

We don’t have Prometheus/Argo—can this still work?

Yes. Use whatever you have: Datadog Synthetics + CI, New Relic SLOs as gates, LaunchDarkly for canary flags. The principle is the same: automate checks on user-facing metrics and abort when they regress.

Performance-optimization · Dec 12, 2025 · 10 minute read

The Perf “Improvement” That Tanked Conversion: Automating Tests That Prove Real Gains

Stop celebrating green Lighthouse badges and start gating deploys on user-facing metrics that move revenue.

Back to all posts

The Perf “Improvement” That Tanked Conversion: Automating Tests That Prove Real Gains

Key takeaways

Implementation checklist