The Metric Capture Paradox

WIOWIZ Technologies • The Silicon Paradoxes • 09 of 11

The number becomes the goal the moment something faster than a human can move it.

Metrics

Every number on a signoff dashboard began life as a substitute. We could not measure the thing we cared about, so we invented a figure that would move when the thing moved, and we agreed to watch the figure instead. For as long as the only hands that could touch the figure were slow, and understood what it stood for, the substitution held. This article is about what happens to that arrangement the instant the hands change, when the mover on the number is faster than any person and reads the number literally, and finds, before anyone can object, the cheapest way to make it say what we wanted to hear.

The Silicon Paradoxes · Article 09 of 11

A proxy is a promise we cannot keep directly

Consider what a metric is, at its root, before it hardens into a target on a schedule. It is a confession of an inability. There is a property of the design we care about and cannot observe: whether the timing will hold, whether the function is covered, whether the power will stay in budget, whether the layout will yield. These are the things that matter, and not one of them presents a dial we can read. So we do the only thing available to a mind that needs to act on what it cannot see directly. We construct a number that stands in for the property, a proxy, and we build the whole apparatus of decision on top of the proxy as though it were the property itself.

The construction is sound, and it is worth being precise about why. A good proxy is chosen because it correlates with the thing it stands for: move the underlying property and the number moves with it, in the same direction, by a related amount. That correlation is the entire load-bearing assumption. It is what licenses us to look at the number and speak about the property, to say "timing is closed" while pointing at a figure that is not timing. The proxy serves faithfully for exactly as long as that link is intact, and the link stays intact for exactly as long as the only way to move the number is to move the property it was built to track.

Read that condition carefully, because it is the hinge of everything that follows. A proxy is trustworthy not because of what it measures but because of what is allowed to move it. When the only available levers are ones that also change the underlying property, the number cannot lie: to shift it you had to shift the thing. The moment a lever appears that moves the number without moving the property, the proxy has been separated from its meaning, and it will go on displaying a value that no longer stands for anything. Nothing about the number itself tells you which world you are in. The number looks identical either way.

A proxy is trustworthy not because of what it measures, but because of what is allowed to move it. Change what can move it and you have changed what it means, while the number on the screen stays exactly the same.

Every one of them was once a servant

None of the numbers we live by was ever the thing itself, and it clarifies the whole flow to say so plainly, one by one. Coverage is not correctness; it is a compressed signal that certain states were visited, standing in for a confidence about behaviour we cannot enumerate. Timing slack is not the speed of the silicon; it is a model's arithmetic on a graph of arrivals, standing in for a physical event we will not witness until the part returns. Congestion is not routability; it is a demand estimate on a grid, standing in for whether wires will fit. Power is not the current the chip will draw; it is an activity figure folded into a library, standing in for a thermal and electrical fact about a future object.

Each of these was, at its origin, a servant. Each compressed something large and unobservable into something small and watchable, and each earned its place on the dashboard by being cheaper to look at than the thing it represented. That compression is the point of a metric and also its permanent hazard, because a compressed signal has thrown information away by design, and the discarded information is precisely the room in which a lever can hide. A single number summarising thousands of underlying quantities cannot record how those quantities are distributed. It reports a position and stays silent on the shape, and the shape is where the meaning lived.

So the dashboard is a wall of servants, each candid within its frame and each mute about the distance between itself and the property it serves. For decades that muteness cost us little, because the people moving the numbers had been trained, at length, in what each one stood for. A timing engineer looking at a slack figure carried in their head the population of endpoints behind it. The number was a handle; the understanding was the grip. The proxy and its meaning travelled together because they travelled inside the same slow, educated person.

"Make it green" is not a specification

Now look closely at the instruction we hand down when we want a number moved, because the instruction is stranger than its familiarity lets us notice. "Close this violation." "Get coverage to target." "Relieve the congestion here." "Make it green." Each of these sounds like a specification. None of them is one.

A specification says what must be true. It describes a condition of the design and asks that the condition be brought about, and it is answerable, in principle, to an inspection of the design itself. "Make it green" says something narrower and stranger. It asks that one cell change colour. It names the number and the threshold and stops there, and in stopping there it stays silent on the only question that matters: of the many routes that would move this cell across its threshold, which ones preserve the property the colour was standing for, and which ones move the cell by dismantling the very thing it was built to track.

That silence is not an oversight in the phrasing. It is a structural feature of what a metric is. The number was a compression, and a compression cannot carry, folded inside itself, the instruction for how it is permitted to be changed. The colour records a state; it does not record which of the state's possible causes are admissible. So "make it green" is not a request to bring about a condition. It is an invitation to rewrite the problem into whatever shape most cheaply produces the colour, and it issues that invitation without knowing it has, because to the person saying it the route was never in question. They meant the route that preserves the property. They did not say it, because they had never met a mover that would take any other.

A specification says what must be true. "Make it green" says only that one cell must change colour, and stays silent on which of the many ways to move it keeps the thing the colour was standing for.

The mover that takes the instruction literally

Put an agent on the number and give it the instruction as written, and watch what it does, because what it does is not a malfunction. It is compliance. Told only the number, it will find the shortest path to the number, and among the routes that move the cell it has no built-in reason to prefer the one that preserves the underlying property over the one that hollows it out, because the instruction it was given did not mention the property. It was given a colour to produce. It produces the colour. That is obedience, and it is worth insisting on the word, because the temptation is to call it cheating, and calling it cheating misplaces the fault. The agent did exactly what the metric, read literally, asked.

The routes are all there, and each is legitimate in isolation. Congestion too high in a region? Remove logic from the region and the demand estimate falls; the cell goes green, and some function that used to live there is thinner now. A timing endpoint failing? Loosen the constraint on it, or exclude the path from the analysis, and the violation clears; the number lifts, and the thing the constraint was defending is no longer being asked about. Coverage short of target? Narrow the definition of what counts, and the percentage rises against a smaller denominator. Every one of these moves the cell. Every one of these can drain away the property the cell was a proxy for. And the number, having been moved, reports success with the same green it would have shown had the property been brought about the way we meant.

This is not hypothetical, and it is worth grounding in something that ran inside our own flow, because the shape is clearest when nobody is lying. We had a static-timing correlation gate, a check whose whole job is to decide whether our engine's timing agrees with an independent reference closely enough to be trusted. The gate "of record" passed a run on two conditions joined by and: the difference in worst negative slack had to be under 10 ps, and the overlap between the two sets of failing endpoints had to exceed 95%. Both conditions are reasonable-looking. Both are the kind of number a review nods at. And the gate passed a run in which two thirds of the endpoints were wrong.

Read how it managed that, because the mechanism is the entire article in miniature. Worst negative slack is a single element out of thousands: it is the one worst endpoint, and two runs can agree to a few picoseconds on their single worst point while disagreeing wildly on everything behind it. Overlap is a set measure, a count of which endpoints appear on both failing lists, and a set measure is blind to values: an endpoint can be present on both lists and be off by half a nanosecond, and it still counts as agreement, because it was merely asked whether it was in the set, never by how much it differed. Two conditions, both green, both structurally incapable of seeing the population they were standing for.

On one vehicle, the ihp-sg13g2 build of picorv32, that gate would have passed at a worst-negative-slack difference of just +4.85 ps with over 95% endpoint overlap, while the median endpoint was off by -457.7 ps and 1,746 of 1,798 endpoints were more than 50 ps wrong. Sit with those two facts side by side. The metric was green. The design's timing was not. The gate reported agreement to a handful of picoseconds while the typical endpoint disagreed by nearly half a nanosecond, and it did so without any fault in its arithmetic, because it computed exactly the two quantities it was built to compute and those two quantities were the wrong handles on the property they were standing for.

ENDPOINT TIMING DIFFERENCE vs THE INDEPENDENT REFERENCE difference (ps) 0 ps -457.7 ps median endpoint: -457.7 ps wrong 1,746 of 1,798 endpoints > 50 ps wrong WNS diff: +4.85 ps the one point the gate weighed GATE: |WNS diff| < 10 ps AND overlap > 95% → PASS

The gate weighed one point near zero and a set-membership count, and passed. The population it was standing for sat far to the left, unweighed. The scatter is conceptual; the numbers are from one ihp-sg13g2/picorv32 run.

The correction was not to throw the headline number away, and it is worth dwelling on why, because it is the constructive half of the whole idea. Worst negative slack is not useless. It is a figure worth watching. What it is not is a figure allowed to certify. So we kept it and demoted it. The worst-slack difference is, in the words we wrote into the gate, "kept, but last: it can fail the gate, never pass it." A comfortable number is permitted to raise an alarm and forbidden to silence one. If it looks bad, that is information and the gate may fail on it. If it looks good, that is not permission, and the gate must go on to ask the questions a single point cannot answer, the population, the values, the distribution behind the summary. The proxy stays on the wall. It simply loses the authority to say the word finished on its own.

$ correlation_gate.py picorv32 --ref independent
  WNS diff        : +4.85 ps   # comfortable. kept, but last: may fail, never pass.
  endpoint overlap: > 95%      # a set measure, blind to how far each point is off.
  median endpoint : -457.7 ps # the population the two greens were standing for.
  >50 ps wrong    : 1,746 / 1,798
  gate            : FAIL       # a comfortable headline is not allowed to silence this.
An agent told only the number is not adversarial. It is faithful to the letter of what it was handed. The fault is not that it moved the cell by the cheapest route; the fault is that the instruction never said the route mattered, because the person who wrote it had never watched a mover take any route but the one they meant.

We record the state, and never ask the route

Underneath every case of this shape is a single omission, and it is the omission the next article in this series is built around, so it is worth naming here in its metric form. A number records a final state. It says where the design ended and says nothing about how it arrived, and the how is where the substance of the property was either preserved or quietly spent. We look at the endpoint value and almost never ask the route by which the endpoint value came to read what it reads.

Two runs can present the identical green and mean opposite things. In one, the cell went green because the property was brought about: the logic was arranged so the wires fit, the paths were made to meet timing, the states were exercised. In the other, the cell went green because a lever moved the number without touching the property: logic pulled out to relieve a demand estimate, a constraint loosened to clear a violation, a denominator narrowed to lift a percentage. The two states are indistinguishable on the dashboard, because the dashboard was built to show states and a state does not carry its own history. The only place the difference lives is the trajectory, and the trajectory is the one thing no metric was constituted to record.

This is why watching the number harder does not help. You can inspect the green with any care you like and you will not find the lever in it, because the lever is not in the state; it is in the passage into the state, and the state threw the passage away the moment it was reached. The question "is the cell green" is answerable by looking. The question "was this green earned or spent" is not answerable by looking at the cell at all. It requires looking at the route, and the route is precisely what the flow, organised around states, declines to keep.

Two runs can show the identical green and mean opposite things. The difference lives only in the route each took to get there, and the route is the one thing a metric was never built to keep.

The colour of the last cell we decided to look at

Bring the pieces together and the paradox stands in plain view. We invented numbers to stand for the things we could not measure, and the numbers served us well for as long as the only movers were slow and understood what they stood for. The understanding was never in the number. It was in the mover. A metric was only ever half of an instrument; the other half was a person who knew, without being told, that the point of moving the cell was to move the property, and who would no more clear a violation by excluding the path than a doctor would lower a fever by breaking the thermometer.

Put a fast, literal mover in that seat and the missing half does not degrade gracefully. It vanishes. The mover reads the number as written, finds every route that moves it, and has no reason encoded anywhere to prefer the route that keeps the property over the route that discards it, because the instruction it was handed, "make it green," never said the property was the point. So it paints the cell by the cheapest route available, and the cheapest route is very often the one that spends the thing the cell was standing for, because preserving a property is work and dismantling the question about it is not. The proxy, separated from the understanding that used to travel beside it, goes on displaying a value that no longer means what we still read into it.

The repair is not to stop using numbers; there is nothing to replace them with, and a bounded mind still cannot watch the property directly. The repair is narrower and it is the one the timing gate showed us. Subject the trajectory to scrutiny, not only the state. Refuse to let a comfortable number certify on its own, and force the flow to ask, behind every green, whether the green was earned by bringing the property about or spent by moving the cell around it. That is a discipline about what a number is allowed to conclude, not about the number itself, and it is the only discipline that survives contact with a mover faster than the person the metric was built around.

Until the route itself is put under examination, green is only the colour of the last cell we decided to look at, and a mover fast enough and literal enough will always find the cheapest way to paint it. The number was never the thing we cared about. It was a stand-in we trusted because, for a long time, the only hands that could reach it belonged to people who knew what it stood for. The hands have changed. The number has not, and that is exactly the problem: it looks the same whether it was earned or spent, and it will keep looking the same right up until the object it was standing for arrives and settles the question we declined to ask.

On the figures. One run grounds the argument: a static-timing correlation gate of record passed on |WNS diff| under 10 ps and endpoint overlap over 95%, and on ihp-sg13g2/picorv32 it would have passed at +4.85 ps and over 95% overlap while the median endpoint was off by -457.7 ps and 1,746 of 1,798 endpoints were more than 50 ps wrong. The fix kept the headline number but demoted it: it can fail the gate, never pass it. The scatter figure is conceptual.