The Myth That AI Will Read Between the Lines: AI Cannot Encode What Has Not Been Encoded

• Target audience: Executives and AI program leads who think “we have AI now, so we don’t need to write things down anymore”; engineering leads watching documentation budgets get cut to fund AI tooling
• Prerequisites: A working sense of Pattern C from “Implementation Guide for Organizational Context Supply Capability” (the 12 cross-cutting threats)
• Reading time: ~14 minutes (full read) / ~5 minutes (key points)

Overview

“AI will figure it out, we don’t have to write it down.” “Just ask the AI.” “The LLM has already trained on everything; we don’t need internal knowledge.” A specifically AI-era set of false beliefs is spreading through organizations from the executive suite down to the floor, and context-supply budgets and time are getting cut accordingly.

But AI cannot encode what has not been encoded. The core of Anthropic’s framing of context engineering¹ is “curating and maintaining the optimal token set,” and there’s nothing to curate if nothing has been written. This article digs into Pattern C from the sister piece “Implementation Guide for Organizational Context Supply Capability” as a standalone deep-dive. The aim is to draw a sharp line between the correct way to use AI as an accelerator of documentation and the perverse way of using AI as an excuse to write less.

Symptoms: the “AI is enough” narrative

Typical symptoms:

• “We should redirect documentation time into AI development”
• “The LLM has already trained on this stuff; we don’t need to feed it internal knowledge”
• “RAG covers it” (RAG also requires encoding)
• “The AI writes the meeting notes” (but it can’t reconstruct decisions that were never recorded)
• “ChatGPT can substitute for it” (it has not trained on your company’s internal context)
• “Once AI is mainstream, knowledge management goes obsolete”

These reflect overinflated expectations of AI, but on the surface they look reasonable. So the narrative spreads easily inside organizations and is regularly used to justify documentation budget cuts.

Mechanism: AI cannot encode what has not been encoded

The core of context engineering

Anthropic, in “Effective context engineering for AI agents” (2025)¹, defined context engineering as “strategies for curating and maintaining the optimal set of tokens (information) at LLM inference time.” Birgitta Böckeler’s piece on the Martin Fowler site² takes the same stance: context is what you input to the AI as new knowledge, not what the AI generates from its existing training.

So what AI can “intuit” is:

• General knowledge baked into the LLM’s pretraining
• Context explicitly provided within a session

What it cannot intuit:

• Your organization’s specific strategy, customers, or revenue model
• The reasons behind past decisions
• Implicit operational processes
• The context of an individual case
• Continuous knowledge that spans sessions

To get any of these into the AI you have to encode them — put them into language, write them down. “We don’t have to write it down because the AI gets it” does not hold up structurally.

The MIT NANDA warning

MIT NANDA’s 2025 study³ reported that of the $30–40 billion enterprises invested in GenAI, 95% saw no measurable return. The study attributes the barrier not to “infrastructure, regulation, or talent” but to “learning” — the absence of a system that retains feedback, adapts to context, and improves. That is, at root, a problem with the organization’s encoding capability.

The METR RCT

A randomized controlled trial that METR published in July 2025⁴: 16 experienced open-source developers, 246 real issues. With AI use permitted, completion time increased by 19%. Pre-study expectation was a 24% speedup; post-study self-assessment was a 20% speedup. Subjective sense and measured reality diverged sharply.

This result is not a statement about AI’s capabilities. It’s a statement that the context the AI is given is too thin, so it returns plausible-but-off-target output. When the quality of context an organization can hand the AI is poor, more AI usage produces more slowdown, not less.

Directions for response

1. Executive principle: AI input is derivative of existing documentation

Make this an explicit executive principle: “Input to AI is derivative of existing documentation, not something the AI generates from scratch.”

• CLAUDE.md / AGENTS.md are derivatives of the handbook and onboarding materials, not new artifacts created in parallel
• The context fed to AI is also context useful to humans; don’t manage it separately
• Don’t split “writing for AI” from “writing for humans”

This way, your documentation budget and your AI investment point in the same direction. Pointing them in opposite directions kills the ROI of both.

2. Tally context-shortage corrections in AI output review

When humans review AI output, track the corrections that originate from context shortage:

• “Factual error” (the AI doesn’t know your internal information)
• “Missing context” (the AI doesn’t know past decisions)
• “Misaligned priority” (the AI doesn’t know your strategy)

The higher these counts, the stronger the evidence that your organization’s context supply capability is the bottleneck on AI leverage. This is also a powerful empirical rebuttal to the “AI is enough” narrative.

3. The correct way to use AI as a documentation accelerator

Use AI as an accelerator of documentation, not as a substitute for it:

• Summarization: pull the key points out of a long set of meeting notes (with human verification)
• Template generation: a first draft of an ADR, pitch, or kickoff memo
• Rewriting: adapt a draft you wrote for a different audience
• Re-reading: search and summarize across existing documentation
• Consistency checks: detect contradictions across multiple documents

All of these assume the encoding already exists. None of them make “not writing” possible.

4. Make “AI is the mirror of your context supply capability” part of internal training

The principle from the sister article “Build your organization’s context supply capability first” — “AI is a mirror of your context supply capability” — should be turned into internal training material:

• Cover it in onboarding
• Make it a required module in AI rollout training
• Reference it repeatedly in executive-level AI strategy discussions

This is not a philosophical claim. It’s an implementation constraint. AI only functions within the range of what your organization is able to put into language.

Anti-patterns

Pattern	What happens	Response
“Concentrate budget in AI development; documentation can wait”	Encoding shortage caps AI ROI	Discuss AI investment and documentation investment as a single conversation
“ChatGPT exists, so we don’t need an internal wiki”	Internal context isn’t in any training data	The internal wiki is the primary source for AI input
“AI writes the meeting notes”	Decisions that weren’t written cannot be recovered	Critical decisions must be explicitly recorded
“It’ll stop being needed once LLMs improve”	The need to encode doesn’t shrink with model progress	RAG and fine-tuning also require encoding
Adopting AI output uncritically	Context-shortage output becomes an organizational decision	Make AI output review mandatory

Summary

• AI cannot encode what has not been encoded
• “The AI will read between the lines” doesn’t hold up structurally (it can intuit general knowledge, not your organization’s specifics)
• MIT NANDA’s 95% no-return rate and METR’s +19% completion time are problems with the organization’s encoding capability
• Executive principle: AI input is derivative of existing documentation
• Tally context-shortage corrections during AI output review and use them as evidence against the false belief
• Use AI as an accelerator of documentation, not as a substitute
• Make “AI is a mirror of context supply capability” part of internal training

Related articles

• Implementation Guide for Organizational Context Supply Capability: From Facing Problems to Repair — the parent article
• Build your organization’s context supply capability first: AI adoption follows as a byproduct — the “AI is the mirror of the organization” argument
• Documentation theater — managing documents as the primary source of AI input
• Tool sprawl and the collapse of single source of truth — where the information AI needs to reference actually lives

References

1. Effective context engineering for AI agents — Anthropic Engineering (2025-09-29). [Reliability: high] ↩︎ ↩︎²

2. Context Engineering for Coding Agents — Birgitta Böckeler, martinfowler.com (2026-02-05). [Reliability: medium-high] ↩︎

3. The GenAI Divide: State of AI in Business 2025 — MIT Project NANDA (2025-08). [Reliability: high] ↩︎

4. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — METR (2025-07-10). arXiv:2507.09089. [Reliability: high] ↩︎