The Coverage Paradox

A coverage report says 95%, and the room relaxes by exactly five points of worry. But stop on the number for a moment and ask what it is a percentage of. It is 95% of a whole that a human drew, before any measurement began, when the space was least understood. The instrument we built to soothe our uncertainty about whether we looked at enough opens by assuming the boundary of that uncertainty. You cannot report 95% until you have already declared, with total confidence, what 100% would have been.

The Silicon Paradoxes · Article 01 of 11

Coverage is one of the most trusted ideas in hardware engineering, and it earns that trust cleanly at the small scale. It converts a vague dread, "did we look at enough?", into something a schedule can hold: a number, a target, a gate. The dread is worth converting. Nobody wants to tape out on a feeling. The trouble is buried in the arithmetic itself, and it is not a bug in any tool. It is a property of what a ratio is.

A ratio needs a denominator. To produce one, you must first commit to the set of things worth counting, the population, the frame, the whole. And you must commit to it at the beginning, which is precisely the moment you know the design least well. So the very first act of measuring completeness is an act of definition, performed under maximum ignorance and never revisited by the measurement that follows. A coverage number, read strictly, is not a statement about the design at all. It is a statement about how faithfully you filled a space you had already decided was the whole space.

And before going further, discard the habit that quietly narrowed this word. Coverage is not a verification feature. It became attached to design verification only because that is where we first learned to write the denominator down, but the question it asks belongs to the whole life of the chip. Specification, architecture, synthesis, placement, clock tree, timing, power, physical checks, packaging, board, firmware, the field it will finally run in: every one of them selects a universe, makes assumptions, exercises some conditions and ignores others, and so each has a boundary between what it examined and what it did not. Read at its true size, coverage is not a percentage at all. It is the name for that boundary, wherever the boundary is drawn, and its primary job is to describe the edge of what we know we looked at. That job was never the property of one stage.

The number cannot audit its own denominator

Here is the sharp edge of it. Coverage is computed against the boundary and can never be computed about the boundary. Every percentage lives inside the frame it was given and has no vantage point from which to question that frame. A report can read complete and still be blind, not because the counting was wrong, but because the thing counted was the wrong set, and nothing in the counting is built to notice.

This is not a thought experiment. On the cva6 core, an independent reference static-timing engine analysed 8,898 of 15,583 timing checks. The report on those 8,898 was correct. Every arc it examined, it examined properly. But the other 6,685 were an entire class, recovery and removal checks on asynchronous set and clear pins, and they were not failing quietly in some corner of the results. They were never in the denominator to begin with. You cannot miss what was never counted. A coverage percentage over the 8,898 would have been immaculate and would have told you nothing about the 6,685, because the 6,685 were outside the universe the number was measuring. The report was correct inside a universe that was itself incomplete.

Sit with the shape of that, because it recurs everywhere in this series. The failure was not that a check failed. The failure was that a category of checks was absent from the population, and completeness measured over the population reported completeness, exactly as designed. The measurement did its job. Its job simply never included auditing whether the population was the right one.

The number was true. The frame was small. No amount of accuracy inside the box tells you the box was drawn in the right place.

There is a discipline that follows directly from this, and mature reports observe it. When a report has never collected a family of metrics, it should write ABSENT, not 0%. The distinction is the whole ethic of the thing. Zero is a measurement: it says we looked and found none. Absent is an admission: it says this was never in our field of view. A report that prints 0% for a category it never sampled is quietly lying with a true-looking number, because it dresses an untouched region in the costume of a measured one. A report that prints ABSENT is doing the harder and more useful thing, telling you where the map ends.

The last candid line a report can print is the one naming what it never looked at. ABSENT is not a low score. It is the edge of the measured world, drawn where a person can see it.

Everything downstream is compression, and compression forgets

The denominator problem is where coverage begins to slip. What happens next is a second, larger loss, and it is structural to how we run projects at all. A modern chip is decided by an enormous number of small facts, and human beings cannot hold an enormous number of small facts. So the flow does the only thing it can: it compresses, again and again, until what is left is small enough to fit in a meeting.

1. Millions of local decisions: every gate sized, every net routed, every arc timed, every via placed.
2. Thousands of transformations that rewrote those decisions: resize, buffer, reroute, legalize, optimize.
3. Hundreds of reports, each summarizing one view of the result.
4. A few key performance indicators lifted from those reports: worst slack, total violation, utilization, count.
5. Five coloured cells on a dashboard.
6. One phrase: tapeout ready.

Every arrow in that descent is a deliberate act of forgetting. That is not a criticism; it is the point of the arrow. You compress because the uncompressed thing is unusable by a human mind. But compression is lossy by definition, and lossy means the output is missing something the input contained. So the question that should hover over every green cell is not "is it true?" The cell may be perfectly true. The question is what did it forget in order to become small enough to be green? A summary is an index to evidence or it is a substitute for evidence, and the difference between those two decides whether the green cell is a doorway or a wall.

The metric survives; the genealogy does not

Compression does something worse than lose detail. It loses provenance, the history of how a result was produced, and provenance is often the only thing that separates a result you should trust from one you should not. Two outcomes can be numerically identical and be different objects, and the number is engineered to make them look the same.

Consider a hypothetical, and hold it clearly as hypothetical, not a WIOWIZ run. Two physical layouts of the same block arrive at the dashboard, and on the dashboard they are twins. Worst negative slack: positive twenty picoseconds. Total negative slack: zero. Design-rule violations: zero. By every column the review looks at, they are the same row in the same spreadsheet, and the spreadsheet would let you ship either one.

Now look underneath, at the history the row deleted. The first layout reached that state with roughly eight thousand buffers, distributed naturally across the block, and it has been stable since just after clock-tree synthesis. Nothing dramatic happened to it late; it converged and stayed converged. The second reached the identical numbers with roughly thirty-one thousand buffers, several thousand of them inserted late in the schedule, concentrated in a handful of blocks, arrived at through repeated churn of resize and buffer and reroute as the tools fought the same regions over and over. One design settled. The other was wrestled to the ground in the final hours and pinned there.

Which one would you rather own six months into production, when a corner shifts? Obviously the first. But nothing on the dashboard encodes "the first." The genealogy, the fact that determines how each design will behave under the stresses you did not simulate, was compressed out on the way to the KPI. The metric is not false. It is impoverished. It preserved the outcome and discarded the evidence of how the outcome was produced, which is exactly the evidence you would want when the outcome is later put under pressure.

Coverage should be a structure, not a scalar

If a single percentage cannot audit its own denominator and cannot preserve provenance, then the fix is not a better single percentage. It is to stop pretending completeness is one number. Coverage wants to become multi-dimensional, and the useful move is to name each dimension by the question the scalar was hiding. Each of these is a separate axis of "did we look enough," and collapsing them into one figure is what let the blind spots hide.

Check coverage

Of the checks that could exist for this design, which of them ran? The cva6 8,898-of-15,583 question, made permanent and visible instead of implicit.

Constraint coverage

What is unconstrained, defaulted, or excluded: false paths, multicycle paths, generated clocks, case analysis, disabled arcs? Which part of "clean" came from the physical design improving, and which part came from redefining the problem until it was clean?

Spatial coverage

IR drop and rule violations are fields across an area, collapsed to a scalar. One hotspot or five hundred? Are the worst regions overlapping with clock trunks, SRAM, and congestion, or scattered harmlessly?

Transformation coverage

What fraction of the design was materially changed, and where, when, and why? A heatmap of churn: which blocks were rewritten hardest, so that even under all-green KPIs, high-transformation regions still earn a second look.

Stage-to-stage genealogy

The worst-slack trajectory across synthesis, placement, clock-tree synthesis, routing, and final engineering-change orders. Did closure arrive smoothly, oscillate, or get bought with late fixes? A final value hides its own trajectory.

Rule coverage

Which rule families were applicable and in fact triggered, not merely "passed"? A rule that never fired on a design it did not apply to is ABSENT, not passing, and the distinction from section two extends directly to here.

Gate-level coverage

What behaviour unique to the gate and timing view was exercised: X-propagation, reset release, back-annotated corners, timing-check activation? Or did the gate-level run merely re-execute the same functional stimulus, slower, and cover nothing the register-transfer view had not already covered?

Notice that none of these is a new tool. They are all recoveries of information the flow already generated and then compressed away. The data existed at the bottom of the descent; it was destroyed on the way up. A structured coverage view is, more than anything, a refusal to throw away the evidence you paid to produce.

Green does not mean stop asking

None of this makes the coloured dashboard stupid. Green, yellow, red is necessary compression, and the human need it serves is legitimate: nobody can navigate a hundred reports by reading all of them, and a visual index that says "start here" is a service. The colours are not the problem. The problem begins at one specific moment, when the summary quietly stops being the index to the evidence and starts being consumed as the evidence. The instant a green cell means "you may now stop asking," instead of "nothing here is currently demanding attention, click to see why," the compression has inverted from a help into a lid.

A better contract for the colour is small and strict. Green should mean: nothing here presently requires your attention, and the reasons are one click away. It should never mean: do not ask another question. Every green metric deserves an attached confidence anatomy, the denominator it was measured over, the constraints that shaped it, the transformations that produced it, the trajectory it took to arrive. A green cell with no anatomy behind it is not evidence of completeness. It is evidence that someone drew a small enough box.

A number can move while the engineering stands still

Once you see coverage as a ratio over a declared frame, a quiet hazard becomes visible: the number can improve without the engineering improving at all, and usually with nobody acting in bad faith. When a metric gets better between two runs, there are at least seven reasons it might have, and only one of them is the one everyone assumes.

1. The design improved. The underlying engineering got better. This is the one we mean.
2. The measurement boundary changed, and a region left the field of view.
3. A constraint changed, and a path that used to count now does not.
4. The denominator changed, and the same numerator now divides into a smaller whole.
5. The tool stopped reporting something it used to report.
6. A path was excluded as false or multicycle, correctly or not.
7. A rule became inapplicable and dropped silently out of the count.

Six of those seven produce a better number and a chip that is exactly as good, or as fragile, as it was before. Only the first is the underlying engineering getting better. And here is the uncomfortable part: on a bare dashboard, all seven look identical. They are the same green cell moving in the same encouraging direction. Without the anatomy, a team cannot distinguish an improvement it earned from an improvement it defined into existence, and the second feels precisely as good as the first right up until silicon disagrees.

Good at outcomes, poor at evidence

Step back from timing and rules and dashboards, and a single sentence sits underneath all of it. Engineering has become very good at measuring outcomes and surprisingly poor at preserving the evidence of how those outcomes were produced. We can tell you the worst slack to the picosecond. We struggle to tell you whether that slack was earned or engineered, converged or wrestled, measured over the whole or over a convenient part. The outcome is captured with great precision. The provenance of the outcome evaporates.

This is where coverage could become something far larger than a percentage, and closer to its original promise. Not a scalar that asks you to trust it, but a structure that hands you the evidence it was built on:

One percentage asks for trust. A structure earns it, by making inspectable the path from the millions of small facts to the single word at the top.

That is a version of coverage that could recover the evidence the compression threw away: what existed, what was checked, what was not, what was excluded, what changed and where and when and why, which assumption permitted each exclusion, and what evidence stands behind the final verdict. Notice it is not one number. It is a genealogy. It answers "how complete" by refusing to pretend completeness is a scalar, and instead showing you the shape of the thing you built confidence out of.

The compression paradox

And now the trap closes, in a shape this series will keep returning to. The more complex a chip becomes, the more aggressively we must summarise it, because a human mind is a fixed-size vessel and the chip is not. But the more aggressively we summarise, the more of the complexity that decides confidence disappears from the summary. The two forces do not balance. They compound in the same direction, and the direction is away from visibility.

So we arrive at the sentence that should be printed on the wall of every signoff room. The more evidence we generate, the less of it any human can inspect. We produce more reports, more checks, more logs, more waveforms, more coverage than any prior generation of engineers, and precisely because there is so much of it, the fraction a person can look at shrinks toward nothing. We do not lose the evidence. We drown in it, and drowning and losing feel identical from inside the meeting. Coverage was supposed to be the answer to "did we look at enough," and it turns out coverage is subject to the same compression as everything it was meant to police.

We will stop here, at the edge, without pretending to have crossed it. The failures that live outside every frame we know how to draw belong to a later article in this descent. This one lands on something smaller and more exact, which is enough to unsettle any dashboard you will read this quarter. A denominator can be measured perfectly without ever having been conceived perfectly. The percentage watches only the measuring. It has never once watched the conceiving, and the conceiving is where completeness was decided, before a single thing was counted.

And one more thing the number hides, which a later paradox will pull on: a coverage figure is always local to the stage that defined it. Coverage closed in verification is a statement about the verification model and nothing else; it does not follow the intent forward. A design can carry a green coverage number out of DV and into synthesis, placement, timing and signoff, and at every boundary the denominator is redrawn by someone new, for a different purpose, in a different language, with no obligation to the one before it. The green you earned upstream does not travel. It was only ever true of the stage that measured it.

Before you accept the next 95%, ask the one question the number was built to keep you from asking: ninety-five percent of what, and who drew that whole, and when, and what did they not yet know? The figure is a faithful account of how well we filled a space. It is silent, and will always be silent, on whether the space was the right one to fill.