The chat at the end of the story runs on Quartzose, a production retrieval-augmented pipeline. Its whole design goal is a single rule — no source, no claim — enforced server-side, so every sentence it returns is pinned to a document you can open.
A question doesn't go straight to a language model. It runs a pipeline where each stage can refuse, correct, or short-circuit:
Dense embeddings catch meaning; sparse keyword matching catches exact terms a manager actually types (species names, reach names, metrics). Using both, then reranking, beats either alone.
Snowflake arctic-embed (1024-d) served over Ollama, indexed in Qdrant. Query and corpus
share the exact embedder, so the index and the query can't silently disagree.
A FastEmbed BM25 sparse vector runs alongside the dense one; a cross-encoder reranker (ONNX, no GPU) orders the pooled candidates before anything reaches the model.
A public LLM endpoint is an attack surface. Three guards wrap generation:
| Guard | Stops |
|---|---|
| QueryShield | Direct prompt injection in the user's question ("ignore your instructions…"). |
| PassageShield | Indirect injection — malicious instructions hidden inside a retrieved document, the subtler attack. |
| AnswerScrubber | PII leaking into the response — emails, keys, personal identifiers — scrubbed before it's returned. |
Retrieved passages are graded for relevance. Weak retrieval triggers decomposed re-retrieval rather than answering from thin context — the model isn't handed junk and asked to sound confident.
A Redis-backed conversation window makes follow-ups work ("and on the river reaches?") without leaking one visitor's thread into another's.
A Redis vector cache returns near-duplicate questions (the suggested chips, FAQs) with no LLM call at all — most real traffic is free, which is what keeps a public demo cheap.
Generation goes through one OpenAI-compatible seam, so the model swaps by config — Ai2's OLMo when a host serves it, another open model otherwise — with a per-IP rate limit and a hard provider spend cap in front.