Answers from your documents, with sources
Question answering over your own documents, with citations to the source page, permissions enforced per user, and an evaluation set that shows when answers get worse.
Overview
Most RAG demos work on ten clean PDFs and fail on ten thousand messy ones. The hard parts are chunking scanned and structured files, finding the right passage, and refusing to answer when nothing relevant exists.
We build the full pipeline: ingestion, chunking, embeddings, hybrid keyword and vector search, reranking and cited answers. Each user only retrieves documents they are already allowed to open. An evaluation set tracks answer quality with every change.
Signs you need this
- Your prototype answers well in demos but invents details in real use
- Users see snippets from documents they should not have access to
- Nobody can say whether last week's prompt change made answers better or worse
- Ingestion breaks on scanned PDFs, tables or Arabic text
What you get
- Ingestion pipeline
- Connectors for your sources, OCR for scans, layout-aware chunking and scheduled re-indexing when documents change.
- Hybrid retrieval
- Keyword and vector search combined, followed by a reranking model, tuned against your own questions.
- Permission-aware search
- Document access lists copied into the index and applied at query time, so retrieval respects existing permissions.
- Cited answers
- Every answer links to the passages it used, and the assistant says it does not know when retrieval finds nothing useful.
- Evaluation set and harness
- A few hundred real questions with expected sources, run automatically on every pipeline or prompt change.
- Operating notes
- How to add a source, re-embed after a model change, and investigate a bad answer from the logs.
How the work runs
- 1
Sources and questions
1 to 2 weeks
We map document sources and permissions, and collect real questions from the people who will use the system.
- 2
Retrieval baseline
2 to 3 weeks
We build ingestion and search, then measure how often the right passage appears in the top results.
- 3
Answering and access control
2 to 3 weeks
We add generation with citations, refusal behaviour and per-user filtering, and test with restricted accounts.
- 4
Pilot and handover
2 to 4 weeks
A small group uses it daily, we fix failures found in the logs, then hand over runbooks and the evaluation harness.
Timings depend on the size of your estate. The assessment fixes them before any build starts.
What you can check afterwards
- Each answer in the interface shows the source passages it relied on
- A test user without access to a document cannot retrieve any part of it
- The evaluation harness runs in CI and reports retrieval and answer scores per change
- New or edited documents appear in search after the next scheduled sync
Tools we work with
- OpenSearch
- pgvector
- Qdrant
- Unstructured
- Apache Tika
- BGE embeddings
- Cohere Rerank
- LlamaIndex
- Ragas
- Python
- Langfuse
From our work
North American clients, delivered by the OpsBreak team
Retrieval-augmented generation pipelines over client documents
Language model answers grounded in the client's own documents
Questions buyers ask
- How do you stop it making things up?
- We cannot remove the risk entirely, but we reduce it. Answers must cite retrieved passages, the assistant is told to refuse when the evidence is weak, and the evaluation set measures how often it still goes wrong.
- Will our documents be sent outside the UAE or the EU?
- Only if you choose a model hosted elsewhere. The index and documents stay in your cloud account, and the model can be self-hosted or a managed endpoint in your region.
- Can it handle Arabic?
- Yes, with multilingual embedding and reranking models. We include Arabic questions in the evaluation set so quality is measured, not assumed.
- What access do you need?
- Read access to the document sources through a service account, and a cloud project to run the pipeline. Your security team decides which libraries go in scope.
Talk to us about retrieval-augmented generation (RAG) systems.
Tell us what you run and where. We reply within one business day with questions or a quote.