This project was built fast, with heavy AI assistance. That is exactly where research quietly goes wrong, so the interesting engineering isn't only the model — it's the machinery that keeps an AI-accelerated codebase correct when nobody is watching every line.
Hallucinated code is the easy failure: it doesn't compile, and a compiler catches it in seconds. The dangerous failures are semantic — they compile, they pass the tests, and they read beautifully, while being wrong. AI is very good at producing fluent, plausible, finished-looking output, which is precisely what makes a wrong result invisible.
| Failure mode | Caught by |
|---|---|
| Hallucinated API call | the compiler |
| Wrong logic | a unit test |
| A retracted result still published as fact | nothing |
| A model scored against 45%-wrong labels | nothing |
| A novelty claim already falsified by a 2018 paper | nothing |
Each hard-won lesson is encoded across several surfaces (the canon file, agent + command definitions, review-bot rules, docs and diagrams) and backed by enforcement tiers — from an editor hint, to a pre-commit hook, to a CI gate that fails the build. A lesson learned once becomes a check that can't be un-learned.
The critical move is the last one: because the rule is mechanical, it protects work done months later by someone (or some model) who never read the original lesson.
Ordinary CI checks file shape: does it build, do tests pass, is formatting clean. None of that can see a doc presenting a withdrawn result as current, or a retired threshold living on in a docstring. Those are semantic drift, and they get their own gates:
| Gate | What it makes impossible |
|---|---|
| tombstones | Resurrecting a retired identifier or a killed metric threshold in any doc, comment, or config — CI fails. |
| retractions | Stating a withdrawn claim unless the same doc also retracts it. A result can't quietly come back. |
| doc-orphans | A spec or decision record that nothing links to — unreachable docs rot into lies, so they fail the build. |
| canon-refs | A paraphrase of the canon that drifts from its source; stale cross-references after a file move. |
the gate passes.A method is only as honest as its record of being wrong. Each gate exists because something real slipped through first — and those are kept in the record, not scrubbed:
The novelty claim assumed no one had mapped basin-wide riparian extent. A 2018 paper (CO-RIP, κ 0.80) had. Found by reading the literature, not by a gate — which is exactly why a paper-audit command now exists to try to falsify the claim on demand.
For 25 of the first 29 merged PRs, a skipped AI review still posted a green check — "no findings" was indistinguishable from "no review." Caught only when a reviewer asked a question that couldn't be answered without actually checking. Now the review, not the check, is the gate.
A tile-layer name was validated with ^[a-z_]+$. In .NET, $ also matches
before a trailing newline, so "wetlands\n" reached an interpolated SQL literal. The AI
review caught what CI, static analysis, 20 unit tests and a careful human passed. Now \A…\z.
A public review page kept presenting a retracted number as current; a retired NDVI threshold lived on in a component docstring; a decision record had nothing linking to it. All three real. All three now impossible to reintroduce without the build failing.
This isn't a toy. The same discipline runs the distributed-systems reference app and this geospatial + RAG project, with a substantial enforcement library behind it: