Goodhart's Law: The Moment You Turn Documentation into a KPI, It Hollows Out

• Target audience: Executives, PMs, and organizational change leads grappling with whether qualitative initiatives should carry KPIs
• Prerequisites: A working sense of Pattern F from “Implementation Guide for Organizational Context Supply Capability” (the 12 cross-cutting threats)
• Reading time: ~13 minutes (full read) / ~4 minutes (key points)

Overview

The moment you make “ADR count,” “wiki page count,” or “postmortem count” a KPI, only the volume goes up while the substance hollows out. The “one decision per file” principle collapses, and “ADR-flavored dummies” get cranked out for the metric — this is a textbook case of Goodhart’s law.

The contemporary formulation, from Marilyn Strathern in 1997¹: “When a measure becomes a target, it ceases to be a good measure.” Building organizational context supply capability is a question of quality, but a KPI slides easily into being a question of quantity.

This article digs into Pattern F from the sister piece “Implementation Guide for Organizational Context Supply Capability” as a standalone deep-dive. The frame is the organizational application of Goodhart, KPIs as health indicators, and the recognition that for some domains “no KPI” is the right answer.

The history of Goodhart’s law

Charles Goodhart’s original proposition (1975)

The British economist Charles Goodhart originally framed it in a monetary policy context²: “Once an economic indicator is adopted for regulatory purposes, the relationship between that indicator and economic reality breaks down.”

Strathern’s contemporary formulation (1997)

Marilyn Strathern applied it to education and social policy in 1997 and produced the version that’s now widely cited¹: “When a measure becomes a target, it ceases to be a good measure.”

Application to organizations

Mapping this onto an organization’s context supply capability:

• “ADR count” becomes a KPI → volume rises, substance hollows out
• “Wiki page count” becomes a KPI → page count rises in parallel with stale-document burial
• “Postmortem count” becomes a KPI → “we’ll be more careful next time” becomes a ritual
• “1on1 completion rate” becomes a KPI → people set a 30-minute timer and end it formally
• “Training completion rate” becomes a KPI → people leave the video playing while doing other work

In every case, the moment the metric becomes the target, the substance drops out.

Why volume KPIs and qualitative initiatives are a structural mismatch

Qualitative initiatives inherently involve subjective judgment. “Is this document useful?” “Is this culture healthy?” “Did this 1on1 go deep?” — the answer depends on who is observing, what context they’re in, and when. The moment you convert this into something objective and quantitative, the substance falls out.

Structural mismatch between volume and quality

What you can measure (volume)	What you actually want to measure (quality)
ADR count	Whether ADRs improved decision quality
Wiki page count	Whether necessary information can be found
Postmortem count	Whether learning reduces the next incident
1on1 completion rate	Whether reports grew and problems surfaced
Training completion rate	Whether trainees can apply it on the job

The relationship between volume and quality is not linear. Up to a point there’s positive correlation; past that, targeting volume drops you into an inverse-correlation zone where quality goes down.

Typical Goodhart runaway patterns

Behavior changes that show up in KPI-driven organizations:

• “Crank out dummy ADRs” (ADR-count KPI)
• “Leave duplicates and stale wiki pages in place” (page-count KPI)
• “Open a postmortem for every minor incident” (postmortem-count KPI)
• “Wrap up 1on1s in 15 minutes” (completion-rate KPI; duration isn’t measured)
• “Leave the training video running” (completion-rate KPI)

These are not malicious. They are the rational behavior that the KPI is incentivizing. Goodhart’s law says: don’t blame people, look at the structure.

Directions for response

1. Treat KPIs as health indicators, not targets

Treat KPIs as health indicators, not targets to hit:

• Not “10 ADRs per month → healthy” but “ADRs are part of how we look at decisions → healthy”
• Watch the number; don’t chase the number
• If the number drops, investigate why; don’t set a target to push it back up

This is the same principle as a blood pressure reading. The number indicates a state of health, but “raising or lowering the blood pressure number” is not the goal (unless you’re already in a pathological state).

2. Watch “reference count” and “reuse count,” not volume

• Not the count of ADRs but how often they are referenced (links from other documents, access log)
• Not the number of wiki pages but how often new hires access them within 30 days
• Not the number of postmortems but the 30-day completion rate of action items
• Not the completion rate of 1on1s but the execution rate of decisions surfaced in 1on1s

These measure qualitative health. Unlike volume, they’re hard to inflate.

3. Some domains are fine with subjective evaluation

Aggregate indicators (see the article on three aggregate indicators) can be subjective:

• Manager self-assessment
• Affirmative response rate on anonymous surveys
• A check on “does the texture feel different?”

Rushing toward objectivity invites number-fudging (Goodhart). Subjective evaluation works on the assumption that you’re following the time series. It’s not suited for cross-organization comparison — and that’s deliberate.

4. A quarterly meeting that questions the KPIs themselves

Hold a quarterly meeting that questions the KPIs themselves:

• “Are we actually changing by chasing this number?”
• “The number is dropping but the texture feels better, or vice versa. Which is right?”
• “Has the indicator stopped functioning as an indicator?”
• “Is there a domain where having no KPI would be better?”

A culture that can question its own KPIs is the last line of defense against KPI runaway. As Jerry Muller’s “The Tyranny of Metrics”³ observes, the worship of metrics has itself become a fragility of modern organizations.

5. Domains where “no KPI” is the right answer

You don’t need a KPI for every domain. Domains where “no KPI” is the right answer:

• Cultural change progress (quantification is counterproductive)
• The quality of 1on1s (judge through observation and dialogue)
• Trust building (the time horizon is too long)
• Depth of learning (judge in the application context)
• The maturity of the paired negative (judge case by case)

“If we can’t measure it, we can’t manage it” is wrong. There are domains where observation, dialogue, and case analysis are sufficient management.

Anti-patterns

Pattern	What happens	Response
“KPI everything”	A breeding ground for Goodhart runaway	Don’t set KPIs for qualitative initiatives
Highlighting volume KPIs in executive reports	The pursuit of volume gets organized	Report access logs and reference counts instead
Writing OKR Key Results as volume metrics	OKR itself runs into Goodhart runaway	Allow KRs to be qualitative and observational
Penalizing dropped KPIs	Incentivizes falsification	Treat KPIs as health indicators
Treating objective indicators as superior to subjective ones	You lose the ability to measure quality	Recognize the legitimacy of subjective evaluation

Summary

• Goodhart’s law: when a measure becomes a target, it ceases to be a good measure (Strathern 1997)
• Setting volume KPIs on qualitative initiatives drops you into an inverse-correlation zone where volume runaway lowers quality
• Treat KPIs as health indicators, not targets
• Watch “reference count” and “reuse count,” not volume
• Some domains are fine with subjective evaluation (assuming time-series tracking)
• Hold a quarterly meeting that questions the KPIs themselves
• Some domains are best served by “no KPI”

Related articles

• Implementation Guide for Organizational Context Supply Capability: From Facing Problems to Repair — the parent article
• Three aggregate indicators: measuring context supply capability beyond individual KPIs — indicator design that avoids volume
• Documentation theater — the harm of turning page count into a KPI
• Operational deep-dive on blameless postmortems — the harm of turning incident count into a KPI

References

1. “Improving ratings”: Audit in the British University System — Marilyn Strathern, European Review, vol. 5, no. 3 (1997). [Reliability: high] ↩︎ ↩︎²

2. Problems of Monetary Management: The U.K. Experience — Charles A. E. Goodhart, in Monetary Theory and Practice (1984; original paper 1975). The original source for Goodhart’s law. [Reliability: high] ↩︎

3. The Tyranny of Metrics — Jerry Z. Muller, Princeton University Press (2018). ISBN: 9780691174952. [Reliability: medium-high] ↩︎