Insights

RAG Engineering Notes

Field notes on retrieval quality, evaluation practice, and AI security — written for engineers who have to make a system work on Monday. No product announcements.

All AI Engineering AI Security Evaluation Case Notes

AI Engineering Feb 2026 · 9 min Latest

Why your chatbot retrieves the right document but still answers wrong

Recall@5 of 0.92 and users still don't trust it. The gap between “the document was retrieved” and “the answer was correct” is where most RAG systems quietly fail — and it is measurable.

Read the note →

retrieved ≠ correct

Gold chunk retrieved 92%

Ranked in top 3 64%

Chunk contained full answer 51%

Answer correct & grounded 47%

Illustrative funnel · not client data

The 5 failure modes we find in almost every RAG audit

Stale duplicates, chunk boundaries through tables, no abstention, permission-blind retrieval, and untested reranking.

Draft · 12 min

AI Security

Prompt injection: the security hole most AI teams ship with

Indirect injection through retrieved documents, and why input filtering is not a control.

Draft · 11 min

Evaluation

Evaluation-first AI development: a practical guide

Building a query set that survives contact with production, and gating releases on it.

Draft · 14 min

On the editorial calendar

Why high retrieval recall can still produce bad answers Reranking vs larger context windows How to classify RAG failures

When GraphRAG is justified Testing prompt injection in enterprise RAG Building reliable evaluation datasets