Permis Bateau RAG

RAG chatbot + study site for the French permis bateau côtier & fluvial. A zero-dependency extraction pipeline, Voyage embeddings, a Mistral-powered knowledge graph, and a DuckDB-backed hybrid retriever — all built to help me pass an exam I knew nothing about.

🚀 Key Features

• 🤖 RAG Chatbot (rag.py chat): ask questions about rules of the road, feux, balisage, VHF, écluses — streamed answers with cited sources (lesson / sheet / note)
• 📝 Quiz Mode (rag.py quiz): random QCM from the corpus, filtered by topic, with explained corrections
• 📊 Benchmark (benchmark.py): measures retrieval coverage and answer relevance against 60 stratified QCMs + 10 out-of-domain traps
• 🚫 Off-topic refusal: politely refuses questions outside navigation instead of hallucinating
• 🌐 Static study site: 15 lessons (tronc commun, côtier, fluvial), 19 printable reference sheets, 2 full 40-question mock exams — pure HTML/CSS/JS, deployable anywhere

🏗️ Architecture

Five standalone Python scripts, one shared DuckDB database (rag/permis.duckdb) with 5 tables: documents, qcm, entities, relationships, document_entities.

extract → embed → graph → retrieve → rag (CLI)
HTML/MD   Voyage +  entities  hybrid   chat/quiz
→ JSONL   DuckDB    + rel.    rerank

• extract.py — zero-dependency parser (stdlib html.parser) → corpus.jsonl + qcm.json
• embed.py — Voyage AI voyage-3-lite (512 dims), 400-word chunks with 50-word overlap
• graph.py — Mistral mistral-small-latest extracts 10 entity types / 8 relation types, deduped by normalized name, checkpointed so an interruption never loses progress
• retrieve.py — classic (cosine similarity) vs graph (seed entities → 1-hop neighbors → linked chunks → rerank with entity-sharing bonus)
• rag.py — retrieval → context → Mistral with streaming

🔬 Results (benchmark, 60 QCM + 10 traps)

The honest takeaway: the knowledge graph does not beat classic retrieval on ranking — on a small, well-structured corpus, cosine similarity is hard to beat. Where the graph earns its keep is source diversity: it surfaces reference sheets (feux, signals, fluvial signage) that the classic path misses entirely.

⚖️ Trade-offs

Why DuckDB

One file, zero servers, real SQL, entirely reproducible pipeline: extract.py → embed.py → graph.py rebuilds the whole base from the corpus. I genuinely like the tech, and it earns its keep — no embedded database plumbing, no state to babysit.

Voyage over OpenAI embeddings

Free-tier-friendly pricing and genuinely good quality-per-dollar on a small corpus. voyage-3-lite gives 512 dims — plenty for 165 QCMs and a few hundred source chunks.

Knowledge graph was a bet that partially paid off

The graph path doesn’t rank better — but it does diversify sources and makes refusal behavior more reliable. I kept both paths and made them switchable (--no-graph), because “does it actually help?” is exactly what a benchmark should answer.

Static site, no framework

The study site is hand-rolled HTML/CSS/JS on purpose: the exam is in 2 months and a build step would be waste. It’s the kind of constraint that keeps the whole project honest.

🎯 The Demo

Want to see what the corpus feels like? Follow the Admis rule: 35/40 correct on the real exam (5 errors max). Try six real questions below:

🔗 Links

• Live study site — lessons, reference sheets, mock exams
• GitHub Repository — MIT code, personal corpus
• Blog post — the full story, from /teach to a hand-rolled RAG pipeline

Built to learn. Validated by benchmark. Sailing the Med by September. ⛵