Perspective
Evidence
3 minute read
When an unsupported number carries the answer
Every decision rests on a few numbers. Most of them have no record of where they came from.
By David Doyle, Principal, DDA
A schedule, a forecast, a policy claim, or an investment case depends on inputs. Some inputs carry the whole result. If one of them is wrong, the answer changes. The analysis then depends on a number that nobody has checked.
DDA starts from that fact. We do not try to resolve every input. We find the inputs that the decision depends on, and we check those.
Three kinds of weak input
The first kind has no source. A figure appears in a report, and the report gives no table, date, or author. The number can be correct. Nobody can show it.
The second kind is a proxy that reads like a direct measure. A count of apprenticeship registrations is a real number. It measures entry into training. A plan can use it as if it measured available workers. The proxy is honest until someone treats it as the thing it approximates.
The third kind is a figure that conflicts with another. Two sources give different values for the same quantity, and the report uses one of them and does not say so. The conflict often comes from a different definition, a different geography, or a different year.
Each kind needs a label. DDA uses three. ACTUAL means a direct extract from a named primary source. PROXY means a derived figure with its steps shown, and a statement of how it differs from the quantity needed. FLAG means we looked for a public source and found none, so the record states the gap and the assumption in its place.
A worked example
Take a common claim: apprenticeship registrations show that labour is available.
The claim has five conditions behind it. Registrations are recorded. The registered workers complete training. They stay in the region. They enter the required occupation. They are available in the project window.
A registration count supports the first condition and nothing else. The count is a PROXY for entry into training. The other four conditions have no support in the count, and each gets a FLAG. The conclusion, that labour is available, depends on four conditions that the evidence does not establish.
The result is a more precise statement than "the claim is wrong." The claim rests on one supported condition and four open ones. The register shows which four, and it shows what evidence will close each one.
Trace the consequence
A weak input matters when the decision depends on it. The test is simple. Change the input across a plausible range and watch the conclusion. If the conclusion holds across the range, the input carries no weight, and DDA leaves it unresolved. If the conclusion flips, the input carries the answer.
A schedule can depend on the number of certified workers in one trade. A fiscal forecast can depend on a growth rate. A policy claim can depend on the definition of an outcome. In each case DDA solves for the value at which the decision changes. That value shows how much error the decision can absorb.
Some uncertainty needs no work. A number that moves by a large amount and changes nothing does not need resolving. Other uncertainty needs work at once, because a small movement changes the decision.
What to resolve first
DDA orders the open inputs by their effect on the decision. The first input to resolve is the one with the most weight and the cheapest path to evidence. Each open row gets a resolution path: the source that holds the answer, the organization that keeps it, and the observable signal that shows early if the input is moving the wrong way.
Signals matter because some inputs cannot be resolved before the decision. A permit, a financing close, an equipment order, or a hiring wave gives evidence that arrives over time. The analysis states which signals will change the conclusion and by how much.
What this changes for the reader
A report that shows its inputs gives the reader a way to challenge it. A hostile reader can pick any figure and reach its row. That access is the standard. A finding that survives it has earned trust, and a finding that fails it shows the problem early, when correcting it costs little.
Send DDA the forecast, the schedule, the claim, or the case that depends on a number. We will find the input that carries the answer and show what supports it.
Related thinking
The trade gap hidden inside a workforce number
A worked case where a regional total hid a shortage in one trade at one phase.
Evidence reconstruction
The register, its status labels, and the rules that keep every figure traceable.