A retrieval pipeline is a chain of small decisions — how you split the text, how you embed it, how you rank what comes back. Each one is tuned, usually implicitly, for English. Chain them together on Arabic content and the losses compound into a system that confidently returns the wrong paragraph.
Chunking on the wrong boundaries
Fixed-token chunking assumes roughly uniform information density per token. Arabic morphology packs more meaning into fewer tokens in some constructions and far more tokens into the same meaning in others, so a 512-token window covers wildly different amounts of actual content depending on the passage.
Splitting on structure instead — headings, numbered clauses, table rows — outperformed fixed windows in every corpus we have measured. Legal and policy documents in particular already carry the boundaries you want; the numbering is the outline.
Normalisation you cannot skip
Alef variants, taa marbuta versus haa, optional diacritics and tatweel elongation all produce distinct token sequences for what a reader considers the same word. Users type one form and the document contains another. Normalising both sides at index and query time is unglamorous and recovers more recall than any embedding-model upgrade we have tried.
Hybrid retrieval, not pure vectors
Dense embeddings are weak exactly where enterprise questions live: policy numbers, account identifiers, product codes and proper nouns. A BM25 lexical index run alongside the vector index, with reciprocal rank fusion over both, consistently beats either alone. It also gives you a debuggable path when a result is inexplicable.
Evaluate in the language you serve
The most common failure is not technical. It is a team that builds an Arabic assistant and evaluates it on English questions translated into Arabic — which are grammatical, formal, and nothing like what a real user types. Build the evaluation set from your own channel transcripts, keep the dialect, keep the typos, keep the code-switching.