Reports›Where Did We Decide That?

Report

Where Did We Decide That?

A small tool that answers "where did we decide that?" from a repo's own notes in a few hundred tokens, shown on 748 of Python's own decisions.

Elephant, drawn by Tamara Schneider

By · November 2026 · 5 min read · The code on GitHub

A trail of decisions

When we're moving fast building things, we end up creating a trail of decision records: why this library and not that one, why the table has two columns, why a feature was cut. With coding agents the trail grows faster, because every session makes decisions, and an agent can write a note about each one in seconds.

Python has kept a trail like this since 2000. Each proposal (a PEP) opens with a short summary its authors wrote at the time, and when one decision replaces another, the record says so. That habit is why anyone can still find out why Python does what it does.

Working as a community, Python wrote about 29 a year. Working alone with coding agents, I wrote 98 in one month.

In a repo run by coding agents, decisions pile up as markdown: decision records, explainers, design notes, handoffs. Each is cheap to write, and an agent can write one at the end of every task.

Python's PEPs are the same pattern, written slowly: 748 records in 26 years, about 29 a year, each with an author-written Abstract and Replaces / Superseded-By headers. Working alone with coding agents, I wrote 194 decision records between February and November 2026, 98 of them in August, and 202 explainers in one repo over the same months. Mine are short (a median of about 400 tokens, against 3,600 for a PEP), and a PEP carries a community's debate that a short note doesn't. The shape is the same, and the pile grows faster than anyone rereads it.

Demonstrated at AI Tinkerers Seattle, Dev Tools track, Nov 3 2026; featured in the AI Tinkerers Community Spotlights newsletter.

The bigger the trail, the more it costs to ask

The larger the project, the more it costs to find out what you've already decided. An agent that can't find the note decides again, sometimes differently. An agent that goes looking for it reads whole files to find one sentence, and you pay for every word it reads.

The cost of “where did we decide X?” scales with the size of the corpus, not the size of the answer. A grep agent reads the top few matching files whole, and those tokens stay in context for every turn after. On python/peps (~3.7M tokens at chars/4) that came to between 28,840 and 85,208 tokens per question.

Two ways people handle it

1. Search everything and read

Let the agent search every file for the question's words, then read the best few matches from top to bottom. It needs no setup, and on a small project it's fine. It gets more expensive as the project grows, and it can't tell you when the answer isn't there: it reads something either way.

Grep the question's terms, rank files by hits, read the top three whole. Zero setup and no index to go stale. Cost is linear in what it reads: about 3× more than ctx on a 6k-token notes folder, 10× on 551k, 110× on 12M (19 real questions across 7 repos, from the README). No relevance floor, so it always returns something.

2. Move the decisions into a database

Put every decision in a proper database, with fields for what was decided, when, and what it replaced. Searching is fast and exact. But the decisions move out of the project into a system somebody has to run, and the agent that made the decision has to fill in a form somewhere else instead of writing a paragraph next to the code.

Structured records (a table, or an issue tracker) give exact queries and real links for supersession. They also move the record out of the repo: a schema to maintain, a service to reach, writes that don't appear in the pull request, and agents writing through an API instead of the markdown they already produce. The record stops being reviewed alongside the change that made it.

Keep the markdown, index the headers

Keep it simple. Five pieces:

  1. Markdown notes. The decisions stay as short notes in the project, next to the code, where people and agents already write.
  2. A header. Each note starts with a few lines: a name, a title, and one sentence saying what was decided and why, written at the time by whoever decided it. If one decision replaces another, the header says so.
  3. An index of the headers. ctx reads only those headers into one small file, rebuilt in a second or two. The notes stay the record; the index can be thrown away and rebuilt.
  4. Search the index first. Before the agent goes looking, it asks ctx. The answer is one line per match: which note, and its one sentence. The agent opens that note and nothing else. When nothing matches, ctx says so and keeps a list of the questions it couldn't answer: the notes still to write.
  5. The corpus. You tell ctx which folders hold notes (decisions, explanations, conventions shared across projects), and it checks that they agree with each other.
  1. Markdown in the repo is the source of truth, reviewed in the same diff as the change it explains.
  2. Frontmatter: id, title, summary (one sentence, written when the decision was made), optionally tags, terms, supersedes / superseded-by. ctx lint fails a note without a summary. ctx never writes summaries, because “a summary invented later by a model that was not there is a guess dressed as a record.”
  3. ctx index puts the frontmatter, not the bodies, into SQLite FTS5 with Porter stemming and BM25. The index is disposable: delete it, rebuild it, and every answer comes back identical. No embeddings, no server, no daemon.
  4. ctx get applies a relevance floor (a hit must match two of the question's words, else it is a logged miss or up to three weak: leads) and a token budget (--budget 800 fills to about 800 tokens, then lists what it dropped), and prints id — title — path#heading — summary. It never reads bodies at query time.
  5. Corpora in .ctx/config.yaml: explainers, decisions (with the supersession chain, ctx history <id>) and shared (across repos). ctx lint checks the frontmatter contract, reciprocal supersession and cycles.

What you get

1. Answered once, not eight times

Your agent stops rediscovering what's already been decided. In one of my own projects (details changed), agents asked the same question eight times in one week: how do I add a second client? The answer was a note written once, in September. Each time, ctx handed back its one line, and nobody worked it out again.

An answer is one line per note: id, title, path and summary. In my own project (details changed) the same question came up eight times in a week, and each time ctx answered with this note's line, 47 tokens, against about 6,145 for reading around the repo:

---
id: 004-a-client-site-is-a-row
title: A client site is a row
summary: A second client is one organization row — no module, no deploy
---

2. The current decision, not an old one

When a decision has been replaced, ctx gives the current one, and shows the old one only when you ask for the history. An agent working from an old decision builds the wrong thing correctly. ctx also checks that your notes don't contradict each other.

Superseded records are excluded by default; --history and ctx history <id> show the chain. ctx lint catches one-sided supersession links, cycles, and an accepted record that something claims to supersede.

3. It says when it doesn't know

An agent that can't find a decision makes it again, sometimes differently. When ctx finds nothing, it says so instead of offering something close, and it keeps the question. That list is the notes still to write.

The relevance floor (a hit must match two of the question's words) turns noise into a logged miss, or up to three lines labelled weak: that are leads rather than answers. It never pads the answer to fill its slots.

4. The session stays light

A search that reads whole files fills the agent's memory, and everything it reads stays there for the rest of the session. One line per answer leaves room for the work, so the agent gets to it sooner and the session runs longer before it fills up.

Tokens read stay in context for every later turn. On the PEPs, grep-and-read took 28,840 to 85,208 tokens per question; ctx took 104 to 857. The index is SQLite over the frontmatter only: 748 documents in 1.3 seconds, with no server to start.

Tested on 26 years of Python's decisions

I asked ctx eight questions about Python's own decisions. Seven times, the first answer it gave was the right one, in a single line, at about a hundredth of what it costs an agent to search and read. The saving grows with the pile: small on a few dozen notes, large on hundreds. At the rate coding agents write them, hundreds come quickly.

On python/peps (748 PEPs, ~3.7M tokens at chars/4), 7 of 8 everyday questions returned the right PEP as the top hit, at 80× to 280× fewer tokens than a grep agent that reads its top three files. The eighth is a vocabulary miss, explained in 2 below.

1. Ask Python where it decided things

Python's language changes are decided in documents called PEPs. Each one starts with an abstract its authors wrote when they proposed it. ctx indexes that kind of summary, so Python's history is a test anyone can rerun.

Each question went two ways. Grep-and-read is what a coding agent does when it has nothing better: search every file for the question's words, then read the best few matches from top to bottom. ctx looks only at the summaries and hands back one line per match. The numbers are tokens, the text the agent has to read before it can answer. Tokens are what you pay for, and what fills the agent's memory.

Each PEP has RFC 822 headers (Status, Replaces, Superseded-By) and an Abstract. A 27-line converter maps them to ctx decision frontmatter. The summary is the first paragraph of the authors' Abstract, verbatim, and the title stands in for the 64 PEPs without one. Final, Accepted and Active map to accepted; Draft and Provisional to proposed; the rest keep their status and drop out of default answers. Nothing is generated. ctx index: 748 documents in 1.3 seconds.

QuestionTop hitctxgrep-and-read
assignment expressions walrus operatorPEP 572: Assignment Expressions18752,404
make the global interpreter lock optionalPEP 703: Making the GIL Optional85785,208
structural pattern matching match statementPEP 634: Structural Pattern Matching26569,958
f-strings literal string interpolationPEP 498: Literal String Interpolation64652,000
async and await coroutinesPEP 492: Coroutines with async and await26132,095
type hints function annotationsPEP 526, then PEP 484: Type Hints50165,063
remove dead batteries from the standard libraryPEP 594: Removing dead batteries53656,609
dataclassesPEP 681: Data Class Transforms (not 557)10428,840

Tokens are characters ÷ 4 on both sides. The grep agent greps the question's words (four letters or more, minus stopwords) and reads its top three files whole. Real agents often read more.

ctx's map of the 748 PEPs, asked about assignment expressions: two PEPs returned, PEP 572 first, in 187 tokens.
ctx's map over the 748 PEPs, asked the first question. Each dot is a PEP; the two red ones came back. A separate run from the table, so the grep figure differs slightly.

To rerun it, download pep2ctx.py and, with ctx installed (below), run:

git clone --depth 1 https://github.com/python/peps
python3 pep2ctx.py peps/peps decisions
mkdir .ctx && echo 'corpora: {decisions: {type: decisions, path: decisions}}' > .ctx/config.yaml
ctx index && ctx get "assignment expressions walrus operator"

2. A miss, and what caused it

Asked “dataclasses”, ctx returned a later PEP about data classes, not the one that introduced them. The original's authors wrote “Data Classes”, two words. Ask “data classes” and PEP 557 comes first. ctx searches the words the authors used. It doesn't guess what you meant.

FTS5 with Porter stemming doesn't split compounds, so dataclasses matches only documents containing that token. PEP 681's summary does (“introduced the dataclass”), PEP 557's doesn't. The fix belongs in the note, not the engine: a terms: [dataclasses] line, added by whoever owns the record. ctx's relevance floor (a hit must match two of the question's words, else it is a logged miss or a weak: lead) is why it says so rather than padding the answer.

3. It knows what was replaced

PEP 333 defined how Python web servers talk to web apps. PEP 3333 replaced it. Ask about it and ctx gives you the current one. Ask for the history and it shows you the chain.

ctx get "WSGI web server gateway interface" returns pep-3333 first; pep-0333 is excluded by default because it is superseded. With --history, pep-0333 appears with superseded-by: pep-3333. ctx history pep-3333 prints the chain with <- current.

ctx's map asked about WSGI: PEP 3333 is returned, 222 tokens against 67,615 for grep.
Asked about WSGI, ctx returns PEP 3333, the current one. Grey dots are superseded PEPs.

4. It checks the record itself

ctx also checks that the notes agree with each other. On Python's PEPs it found 20 problems across 14 PEPs: a PEP still marked Final although a later PEP says it replaces it, or a replacement recorded on one PEP and missing from the other.

ctx lint checks reciprocal supersession, cycles, and an accepted record that something claims to supersede: 20 errors on python/peps. Example: PEP 333 is Status: Final with Superseded-By: 3333. These are findings about the headers, not bugs in ctx.

Try it

You don't need a standards body. Install ctx, and paste two sentences into the file your agent reads at the start of every session (CLAUDE.md, AGENTS.md or similar):

When you make a decision, write a short note with a one-sentence summary of what was decided and why. Before searching the code for context, run ctx get "<your question>".

From then on the agent keeps the record as part of finishing each task, and the next session reads it before anything else. When ctx finds nothing for a question, it logs it, and that list is the notes still to write.

uv tool install git+https://github.com/ivantohelpyou/ctx
cd your-repo && ctx init      # .ctx/config.yaml, .ctx/ROUTING.md
ctx index && ctx get "token budget"

In CLAUDE.md or AGENTS.md: Before grepping for context, run ctx get "<question>" and read .ctx/ROUTING.md. Notes need frontmatter with an id, a title and a one-sentence summary; ctx lint fails a note without one. Other frontmatter (Hugo's, say) maps with map: in the config.

Where it fits, and where it doesn't

It's for repos where decisions pile up in notes faster than anyone rereads them. It isn't a search engine for prose, and it doesn't replace reading code: it finds the note that explains, then gets out of the way.

It suits hundreds to thousands of notes with author-written summaries. It doesn't search bodies at query time, so if the summary doesn't say it, ctx won't find it. A small corpus gains little: on a 6k-token notes folder the saving was about 3×.

Follow Model Citizen Developer on LinkedIn →

Field Notes by email. New reports, floor models and talks, as they land.

No tracking pixels. Unsubscribe in one click.