This project was built fast, with heavy AI assistance. That is exactly where research quietly goes wrong, so the interesting engineering isn't only the model. It's the machinery that keeps an AI-accelerated codebase correct when nobody is watching every line.
Hallucinated code is the easy failure: it doesn't compile, and a compiler catches it in seconds. The dangerous failures are semantic: they compile, they pass the tests, and they read beautifully, while being wrong. AI is very good at producing fluent, plausible, finished-looking output, which is precisely what makes a wrong result invisible.
| Failure mode | Caught by |
|---|---|
| Hallucinated API call | the compiler |
| Wrong logic | a unit test |
| A retracted result still published as fact | nothing |
| A model scored against 45%-wrong labels | nothing |
| A novelty claim already falsified by a 2018 paper | nothing |
Each hard-won lesson is encoded across several surfaces (the canon file, agent + command definitions, review-bot rules, docs and diagrams) and backed by enforcement tiers, from an editor hint, to a pre-commit hook, to a CI gate that fails the build. A lesson learned once becomes a check that can't be un-learned.
The critical move is the last one: because the rule is mechanical, it protects work done months later by someone (or some model) who never read the original lesson.
Ordinary CI checks file shape: does it build, do tests pass, is formatting clean. None of that can see a doc presenting a withdrawn result as current, or a retired threshold living on in a docstring. Those are semantic drift, and they get their own gates:
| Gate | What it makes impossible |
|---|---|
| tombstones | Resurrecting a retired identifier or a killed metric threshold in any doc, comment, or config, and CI fails. |
| retractions | Stating a withdrawn claim unless the same doc also retracts it. A result can't quietly come back. |
| doc-orphans | A spec or decision record that nothing links to. Unreachable docs rot into lies, so they fail the build. |
| canon-refs | A paraphrase of the canon that drifts from its source; stale cross-references after a file move. |
the gate passes.A method is only as honest as its record of being wrong. Each gate exists because something real slipped through first, and those are kept in the record, not scrubbed:
The novelty claim assumed no one had mapped basin-wide riparian extent. A 2018 paper (CO-RIP, κ 0.80) had. Found by reading the literature, not by a gate, which is exactly why a paper-audit command now exists to try to falsify the claim on demand.
For 25 of the first 29 merged PRs, a skipped AI review still posted a green check; "no findings" was indistinguishable from "no review." Caught only when a reviewer asked a question that couldn't be answered without actually checking. Now the review, not the check, is the gate.
A tile-layer name was validated with ^[a-z_]+$. In .NET, $ also matches
before a trailing newline, so "wetlands\n" reached an interpolated SQL literal. The AI
review caught what CI, static analysis, 20 unit tests and a careful human passed. Now \A…\z.
A public review page kept presenting a retracted number as current; a retired NDVI threshold lived on in a component docstring; a decision record had nothing linking to it. All three real. All three now impossible to reintroduce without the build failing.
This isn't a toy. The same discipline runs the distributed-systems reference app and this geospatial + RAG project, with a substantial enforcement library behind it: