A difference you can see is not a difference you can prove.

100ms against 103ms, five runs a side. That is “3% slower” whatever the spread is — and at a 0.2ms run-to-run sd it is p < 0.0001 with the effect known to within 0.6ms, while at 4ms it is p = 0.27 and the interval runs −2.8ms to +8.8ms — it contains a speed-up. The percentage is identical in both. The spread that decides between them is the one number the report does not carry.

The engine is free and needs no account. Paste the results — one variant per line, the baseline first. The smallest effect a suite can detect is fixed before it runs: MDE = (z + z) × cv × √(2/n), so five runs a side at 4% noise resolve 7.09% and a 3% regression is invisible by construction — the power against it is 21.9%, and four times in five the job reports nothing and is believed. A suite of k benchmarks alarms on noise: 20 comparisons at alpha 0.05 turn a clean tree red 64.2% of the time. And a tail percentile is an order statistic — from 50 runs the p99 is the largest run, and 80% against 70% is not separable at a hundred tasks each. Every lane ships a worked example that costs nothing. Writing the report needs credits.