2024 was the year of prompt engineering. The whole discipline was about persuading a model to want the right thing. As GLM-5.2, DeepSeek V4 Pro, and Kimi K2.7 all got better at simply following instructions, that lever kept losing value: a well-phrased prompt on top of the wrong context still produces a wrong answer, only a more confident one. What is left to engineer now is context: which units of memory get pulled from storage, how they are ranked against a query, and what actually survives to fit inside the small share of the window a model spends acting instead of reading.

Context engineering divides into four jobs, and this project treats them as ordered rather than parallel: write context down outside the window so it survives the session, select the few units a given task actually needs, compress them so the model spends its budget acting instead of reading, and isolate what remains across agents and sandboxes so one mistake stays cheap. Getting them right, in that order, is worth more from here than any further model shopping, and the gym's own numbers already show it. Three sessions were spent benchmarking DeepSeek against GLM against Kimi while the layer underneath all of them was surfacing the right document less than a quarter of the time. The model was never the binding constraint.

Who does what

Three models, deliberately separated so no one of them can mark its own work. The split is the reason a score here means anything.

STUDENT · GLM-5.2Sits the exams. Retrieves from the Brain, explores a sandboxed copy of the repository at the exam's own commit, and submits a diff. Cheap and flat-rate, which is the whole point: the project exists to find out whether a cheap model with a good Brain writes code like this repository's.

TEACHER · GPT-5.6Grades. Reads the student's diff beside the reference implementation and scores five fixed axes, citing exact lines. It never writes code for the project, so it cannot grade its own work.

AUDITOR · ClaudeBuilds the harness, checks that the numbers mean what they claim, and reports honestly when they do not, including when the fault is its own. It does not sit the exams and does not set the grades.

How it works today

One training cycle is the loop this whole platform exists to run, and it always executes in the same order:

1. Schedule picks the next exams from real git history. Each one is a task the repository actually faced, with the commit that solved it held back as the reference answer.
2. Student (GLM-5.2) receives its retrieved context plus the always-on standing rules, explores the sandbox, and submits a diff.
3. Exam sandbox applies the diff and runs the gates. A gate judges the change, never the age of the checkout: it compares against the same exam's baseline run and reports unavailable when it cannot assess the commit at all.
4. Teacher (GPT-5.6) scores five axes against the reference answer, every score carrying a citation to a real line.
5. Harvest asks one evidence-bound question per genuine gap the grade exposed.
6. Curate runs every proposal through deterministic contradiction, duplicate, provenance, and one-concern gates.
7. Only what survives curation is written back to the Brain, which the student draws on again at the start of its next exam.

What actually reaches the student

The loop above is the process. This is the substance: everything the student can see, and where the result goes. Six layers are assembled into one prompt; the top three never change, and only the two Brain layers differ from exam to exam. The student reads that window, writes a diff, and never sees the real commit it will be judged against.

The task layer is deliberately thin right now. Each exam also has a written description of its required behaviour, integration points and acceptance criteria, and approving one appends it to that layer. All 35 are held in draft on purpose: the current question is what the student can do from a plain task statement, and handing it a specification would change both the difficulty and the search query mid-measurement. They stay drafted until this baseline is understood, and turning them on becomes its own experiment rather than a quiet drift.

The number that matters most right now

Whether the Brain actually surfaces the right unit for a task is the direct measure of select, and it is now measured two ways. The bars below are the in-situ measure: a locked 13-exam relevance panel scored during real graded work, rather than the model's own citation count. It only moves when a paid cycle runs, and it has not been re-audited since GLM became the daily driver, so the figure below is the last independent measurement on record. The retrieval-quality section on the dashboard is the isolated measure: 34 curated queries with known answers, run against the live index for free, and it is what a ranking change is judged by day to day.

Phases

These are the four buckets above, in build order, plus where the harness itself is going. Each is graded by the same loop before the next one starts. That is why compress is still marked next even though it already has measured evidence in hand: the evidence says the change helps, and the measure cannot yet prove it does not also cost something.

1. 01
   Write: keep context outside the windowCurated lessons, ADRs, and exemplars persist in Postgres instead of living in a prompt. Built: every accepted lesson survives contradiction, duplicate, provenance, and one-concern gates before it is written.
   DONE
2. 02
   Select: pull back only what the task needsHybrid retrieval: vector search over bge-m3 fused with a Postgres full-text arm by reciprocal rank. Measured against the 34-case gold set on the dashboard, which is what makes this the current phase rather than a finished one.
   ACTIVE
3. 03
   Compress: make what was selected fitTrim and summarise retrieved units so the engineer spends its window acting rather than reading. Already has measured evidence waiting: the per-candidate token cap is the first lever, and lowering it improves gold-set score but is not yet shipped because presence is not the same as usefulness.
   NEXT
4. 04
   Isolate: split context across boundariesSub-agents and sandboxed tool calls, each holding only its own slice. Tool definitions and tool results are themselves context, so this comes after compression: bloating either wastes exactly the budget compression protects.
   PLANNED
5. 05
   Claude Code migrationLeader model plus sub-agents, each with live Brain access, replacing this harness’s direct calls. The end state of isolate rather than a bucket of its own.
   PLANNED

The end goal

This repository is a proving ground, not the destination. Once retrieval and compression are proven fair and effective here, the plan is to generalize the same pipeline: one Brain per project, holding that project's own conventions, ADRs, and harvested lessons, kept current by an ingestion pipeline that runs after every commit instead of by hand. Each Brain is hosted on its own: one Postgres-plus-pgvector container connected to its own repository, so a change to the code and the documented decision behind it never drift apart. Each Brain re-embeds only what changed, runs its own evaluation loop, and only writes a lesson back after it survives the same curation gates this one already enforces.

Claude Code is the intended home for the leader side of that, not a replacement for what is built here. The plan is Fable or Opus as the leader, instructing sub-agent engineer models that each have live access to their own project's Brain. The leader delegates, the sub-agent retrieves and acts, and the loop that grades and curates keeps running underneath exactly as it does today. Nothing about the harness changes; only what gives the instructions does.

Security

Whatever the Brain returns is pasted straight into the prompt of a model that can then edit the repository. Nothing reads it in between. So the question worth asking is not whether retrieval helps, but what one wrong document could do once it is in there. The honest answer has two halves: the defences that exist and are tested, and the holes that are still open.

BUILT AND TESTED

OPEN AND UNSOLVED

Where the knowledge lives

The Brain is one Postgres database with the pgvector extension, not a dedicated vector product. That is a deliberate choice. The same database also holds the exam bank, the reference commits, the cycle ledger, and the recorded links between documents, and all of those constantly have to be read together with the embeddings. Keeping them in one place means a lesson and the run that produced it are saved or discarded together, in a single step that cannot half-succeed. Split across two systems they would eventually drift apart, and nothing would say when.