Release confidence, powered by evidence

A pass rate is not release confidence

Take a sprint that looks healthy on the surface: 1,248 test cases, a 96% pass rate, 78% automated, 94% of requirements covered.

Now add the detail that a pass rate hides — twelve open defects, three of them critical. The suite is green because it is not testing the thing that is broken.

A green suite tells you what you checked. It says nothing about what you did not.

This is why quality reporting so often fails at the moment it matters. Each number is true, none of them is a decision, and the person who has to sign off is left combining them by instinct.

Four signals that predict a bad release

Confidence is not one metric. It is the relationship between four, and no single tool usually holds all of them:

• Test results — what passed, what is flaky, and what has been skipped quietly for three sprints
• Code coverage — where the suite actually reaches, and which recent changes it does not
• Defects — open severity mix, age, and whether they cluster in one area
• Production incidents — what escaped last time, and whether the same surface is changing again

Individually each is a status. Together they are a prediction. A change landing in an area with thin coverage, an ageing critical defect and a history of production escapes is not a 96% risk — it is the risk, and it is visible before the release, not after.

The same evidence, read differently by each role

Quality is the clearest case for role-shaped intelligence, because the same underlying evidence answers a different question for every person in the delivery chain:

• Tester and QA — which risks are unmitigated before the release, and where coverage is thinnest against what changed
• Scrum master — whether the quality debt in this sprint is about to become the constraint in the next
• Product owner — whether the acceptance criteria that mattered are actually verified, not merely ticked
• Developer — which specs are flaky in the area being touched, before rather than after the pull request
• DevOps and release manager — whether the evidence needed to sign off exists, and what is still missing

A single quality dashboard shown to all five gives four of them somebody else’s view. The evidence should be one source; the reading of it should belong to the role.

From signals to a decision

The useful output is not a richer dashboard. It is a smaller number of decisions, each with its reasoning attached.

A quality health score that says which signal is dragging it down. Risk hotspots that name the area rather than the metric. A release confidence figure a QA lead can defend in a go/no-go, because the evidence behind it is one click away.

And when the answer is no, the next action should already be specific: re-run these two flaky checkout specs, cover this changed path, close this critical before sign-off.

Quality that survives contact with the deadline

Every quality process works when there is time. The test is what survives the week the date is under pressure.

Processes that depend on someone assembling evidence by hand are the first thing dropped. Evidence that assembles itself from the work — coverage against what changed, defects against what shipped, incidents against what broke — is still there when it is least convenient and most needed.