Retrieval-augmented generation (RAG) has become the default pattern for grounding LLMs in private documents. But most RAG tutorials assume English-only corpora and embedding models trained predominantly on Western text. When we set out to build knowledge assistants for East and West enterprises, we quickly learned that a pipeline that works for English PDFs fails quietly on Swahili policy documents, Amharic HR handbooks, and Yoruba customer support archives.
This article walks through how the LesterTechLabs AI team designed multilingual RAG pipelines for three many languages — the mistakes we made early on, the architecture we landed on, and the evaluation framework we use before every production launch.
The problem with English-first RAG
Standard RAG stacks look simple: ingest documents, chunk text, embed chunks into a vector
store, retrieve relevant passages at query time, and pass them to an LLM with a citation
prompt. In English, off-the-shelf embedding models like text-embedding-3-small
or open-source alternatives often deliver acceptable recall on internal wikis and PDFs.
n language content breaks this assumption in several ways:
- Morphological richness — Swahili agglutination means a single root word appears in dozens of inflected forms. Fixed-size chunks split conjugations across boundaries, hurting retrieval.
- Script and tokenization — Amharic Ge'ez script requires UTF-8-safe parsers and embedding models that were actually trained on Ethiopic text, not just transliterated Latin.
- Code-switching — Support tickets and internal memos frequently mix English, Swahili, and local languages in the same document. Pure language detection per chunk loses context.
- Scarce evaluation data — There are few public benchmarks for document Q&A in many languages, so teams ship without measuring faithfulness or citation accuracy.
Our first prototype used English embeddings on translated documents. Accuracy looked fine in demos — until real users asked questions in Swahili about untranslated source PDFs. Retrieval missed the right passages 40% of the time. That failure mode is worse than no AI at all: confident wrong answers with fake citations.
Project scope and document sources
The engagement covered three document collections for a regional financial services group:
- Swahili — Product terms, branch SOPs, and customer FAQ PDFs (~2,400 pages)
- Amharic — HR policies and compliance manuals (~800 pages)
- Yoruba — Call-centre scripts and escalation playbooks (~350 pages)
Documents arrived as PDF, Word, and HTML exports from SharePoint. Access control required role-based retrieval: branch staff could not see head-office legal content. Every query and retrieved source needed audit logging for internal compliance review.
Architecture overview
We implemented a four-stage pipeline shared across languages, with language-specific tuning at ingestion and retrieval:
- Ingestion — Parse, extract text (including OCR for scanned PDFs), detect primary language per section, and attach metadata (collection, ACL group, file hash, page number).
- Chunking & embedding — Language-aware chunk sizes, multilingual embedding model, and duplicate detection via content hashing.
- Retrieval — Hybrid semantic + BM25 search, cross-encoder reranking, and ACL filters applied before results reach the LLM.
- Generation — Grounded answers with mandatory source citations, refusal when retrieval confidence is below threshold, and feedback capture for continuous tuning.
The stack ran on the client's AWS account: PostgreSQL with pgvector for embeddings, Redis for caching frequent queries, Celery workers for ingestion jobs, and FastAPI for the Q&A API. LLM calls used Claude for generation with retrieved context only — no document data in training pipelines.
Choosing embedding models
We benchmarked three approaches on a held-out set of 120 question–passage pairs per language, annotated by native-speaking reviewers:
- English-only embeddings on machine-translated chunks
- Multilingual models (
multilingual-e5-large,BGE-M3) - Language-specific fine-tuned adapters on top of multilingual base models
Translation degraded Amharic numerals and legal terminology. Multilingual E5 won on recall@5 for Swahili and Yoruba out of the box. For Amharic, a thin fine-tune on 4,000 synthetic Q&A pairs generated from the client's docs (with human review) lifted recall@5 from 71% to 89%.
Key lesson: do not assume one embedding model fits all languages in a corpus. We store the embedding model version in chunk metadata so re-indexing is triggered when models change.
Chunking for morphologically rich text
Default 512-token chunks with 50-token overlap worked poorly for Swahili policy documents where a single sentence carries legal meaning across clauses. We moved to:
- Structure-aware splitting — Respect headings, numbered clauses, and tables before token limits.
- Language-specific targets — 350 tokens for Swahili/Yoruba prose, 280 for Amharic (denser script per token).
- Parent–child indexing — Small chunks for retrieval, parent sections passed to the LLM for broader context.
Parent–child indexing was the single biggest accuracy improvement after switching embedding models. Retrieval hit precise clauses; generation received enough surrounding context to avoid truncating negations or exceptions.
Hybrid retrieval and reranking
Pure vector search missed exact product codes and Swahili legal terms that BM25 matched reliably. We combined:
- pgvector cosine similarity (top 20 candidates)
- PostgreSQL full-text search with language-specific dictionaries where available
- Cross-encoder reranking on the top 10 pairs before prompt assembly
Queries in English about Swahili documents still worked: we embed the query with the same multilingual model and apply collection-level language filters only when the user explicitly selects a locale — otherwise code-switched corpora remain searchable.
Citations, guardrails, and refusals
Every production answer includes clickable citations (filename, page, section). The system prompt requires the model to quote only from retrieved passages. If reranker scores fall below a calibrated threshold, the assistant responds with a refusal and suggests contacting support — never inventing policy text.
We run weekly faithfulness checks: 50 random production queries re-judged by reviewers against source documents. Target is ≥92% faithful answers with valid citations. Hallucination rate dropped from 18% (English-only prototype) to under 4% after multilingual tuning and refusals.
Results and what we'd do differently
After eight weeks, the client rolled out internal Q&A to 1,200 staff across and Ethiopia. Self-service policy queries reduced tier-1 HR ticket volume by an estimated 34% in the first quarter. Swahili and Amharic satisfaction scores matched English within five points on post-answer surveys.
If we started again, we would invest earlier in native-speaker evaluation sets and OCR quality for scanned Amharic PDFs — noisy ingestion caused more production issues than embedding choice. We would also automate re-indexing webhooks from SharePoint on day one instead of weekly batch jobs.
Building your own multilingual RAG pipeline
Multilingual RAG is not a translation problem bolted onto an English pipeline. It requires language-aware ingestion, multilingual embeddings, hybrid retrieval, and evaluation benchmarks built with speakers of the target languages — before users ever see the chat interface.
If you're planning document Q&A for Swahili, Amharic, French, or mixed-language corpora across , our RAG Systems team designs private pipelines from ingestion through production monitoring. Get in touch with your document types, languages, and access requirements — we respond within 48 hours.