I'm currently building an AI chatbot with an agentic architecture for querying internal documents and data. One of the main challenges I'm facing is designing a reliable ingestion and retrieval pipeline that can handle a wide variety of document types without sacrificing answer accuracy.

1. The document types I need to support

The system needs to work with a fairly broad range of content, including:

• Legal documents and regulations: Long documents with nested sections, articles, clauses, and cross-references.
• Reports and business documents: Documents containing tables, charts, figures, and mixed layouts.
• General text documents: DOCX, TXT, Markdown, and similar formats.
• Excel files: Multiple sheets, structured tables, headers, and potentially complex relationships between cells.
• PDFs: Both digitally generated PDFs and scanned PDFs that require OCR.
• Images embedded in documents: Diagrams, charts, screenshots, scanned pages, and other visual information that may contain essential context.

The challenge is that these formats have very different structures and require different extraction and preprocessing strategies.

2. Problems with my current approach

Currently, I'm experimenting with separate ingestion pipelines for different document types. Each pipeline handles extraction, preprocessing, chunking, and indexing according to the input format.

However, I'm running into several issues:

• Ingestion complexity: Each format requires different handling, and maintaining multiple pipelines introduces edge cases and inconsistent outputs.
• Structure preservation: Naive text extraction and chunking can lose relationships between legal articles, tables, spreadsheet headers and values, or text and its associated images.
• Scanned PDFs and visual content: OCR can introduce errors, while extracting text alone may miss important information in charts, diagrams, and complex layouts.
• Retrieval accuracy: A single retrieval strategy doesn't seem suitable for every type of content. For example, semantic search may work well for conceptual questions but perform poorly on exact legal clauses, numerical values, or spreadsheet lookups.
• Query routing: Since this is an agentic chatbot, I also need to determine how to route each question to the appropriate retrieval strategy or data source. Some questions may require a combination of document retrieval, structured data lookup, and visual content analysis.

My main concern is that adding more document-specific logic will make the system increasingly difficult to maintain, while using a single generic pipeline may compromise retrieval quality.

3. What I'm looking for

For those who have built production RAG systems handling heterogeneous enterprise documents:

1. Ingestion architecture: Would you recommend a unified ingestion framework with format-specific extractors, or completely separate pipelines for each document type? Is there a good way to normalize their outputs while preserving document structure?
2. Document parsing: What tools or frameworks have worked well for complex PDFs, scanned documents, Excel files, and image-heavy reports? I've been exploring tools such as Docling, Unstructured, and OCR-based solutions, but I'm still unsure how to combine them into a robust pipeline.
3. Indexing and retrieval: How do you handle different retrieval requirements across legal text, ordinary documents, tables, spreadsheets, and visual content? Do you use separate indexes, hybrid retrieval, parent-child chunking, multimodal embeddings, or a combination of these?
4. Agent and query routing: How do you decide whether a question should go through vector search, keyword search, structured table or spreadsheet lookup, OCR-derived content, or a vision-capable model? Do you use a lightweight classifier, an LLM-based router, or a more explicit query-planning approach?
5. Practical recommendations: If you were designing this system from scratch today, what architecture and open-source components would you choose? Are there any approaches that seem promising in demos but become unreliable or expensive in real-world use?

I'm particularly interested in approaches that prioritize answer correctness, reliable source citations, and maintainability, rather than simply maximizing retrieval coverage.

Any recommendations, lessons learned, or examples of production architectures would be greatly appreciated. Thanks!

Something that's been bugging me about agent memory evals. Every benchmark I see measures recall: did the agent remember the fact? But in long-lived agents, the harder problem is un-remembering. Plans change, facts get superseded, preferences flip. A memory that recalls a stale fact with high confidence is worse than one that admits it doesn't know.

I keep wondering why there's no standard metric for this. Something like: inject a fact, then supersede it, then ask. Measure whether the agent serves the new fact, the old one, or hedges. Call it retraction accuracy or memory half-life, whatever. Does this benchmark exist and I'm just not aware of it? How are you handling stale memory in production agents?

Most RAG projects begin with a simple assumption: if we can collect the documents, we can make them searchable.
In practice, the hardest work starts before the first embedding is generated.
Enterprise knowledge is usually scattered across PDFs, Word files, spreadsheets, slide decks, scanned pages, images, tables, and databases.
These sources rarely share a consistent structure.
A PDF may contain headers, footers, broken paragraphs, and tables whose meaning depends on surrounding text.
A chart may carry more information in its labels and relationships than in the caption.
A database row may be accurate but meaningless without its schema or business context.
Converting everything into plain text can remove exactly the context that retrieval needs.
Raw extraction also introduces duplicates, formatting artifacts, missing values, and partial records.
If those problems are pushed directly into the vector store, the retriever has to search through noise.
Poor answers are then often blamed on the embedding model, even though the real problem was the source data.
For this reason, building a RAG knowledge base should begin with data conversion and cleaning as a deliberate engineering stage.

A practical pipeline can be designed around several stages:

1. Ingest the source data and preserve provenance such as file name, page number, section, table, image, database, and record identifiers.
2. Convert each format into a structured representation. Document text, tables, chart descriptions, images, and database records should be represented as records with both content and metadata.
3. Clean and normalize the records by removing repeated headers, footers, invalid characters, broken fragments, empty fields, and formatting artifacts.
4. Preserve relationships and context. Table headers should stay connected to their values, chart labels should remain associated with the relevant visual content, and database rows should retain schema information.
5. Split the content into meaningful chunks based on document structure, sections, records, or semantic boundaries instead of using only a fixed character length.
6. Generate retrieval-ready data, such as question-answer pairs, summaries, explanations, or context-enriched passages.
7. Evaluate and filter the outputs for completeness, source consistency, language, length, and relevance.
8. Remove duplicate and near-duplicate records using rules, fingerprints, or semantic similarity, then export the validated data for embedding and indexing.

Intermediate outputs should be stored after each major step, so a failed extraction or overly aggressive filter can be inspected and corrected without rebuilding the entire dataset. This also makes it possible to use different processing branches for PDFs, images, tables, and structured database data while keeping a consistent downstream format.

The design above is implemented in OpenDCAI/DataFlow, and I’d be happy to discuss it further—automated data processing pipelines are an important capability for building reliable RAG systems.

Hi everyone,

I'm working on an LLM-based contract-audit pipeline that processes scanned documents and extracts important information, including cheque numbers, lot numbers, and other contractual references.

My main problem is inconsistent extraction. When I process the same scanned document multiple times, the model sometimes returns different results. For example, small digits in cheque or lot numbers can be misread, even though the input document is exactly the same.

I've already tried setting the temperature to 0 and using a fixed seed, but the problem persists.

My main requirement is to avoid using a separate OCR engine. I want to use a vision-capable LLM directly to read the scanned documents.

I'd appreciate advice on the following:

• How can I improve the accuracy of small digits and identifiers in scanned documents?
• Are image preprocessing, high-resolution crops, or processing individual fields separately effective?
• How can I reduce output variations when using the same image and model?
• Are there specific vision models or inference settings that perform particularly well for document extraction?
• How do you handle ambiguous digits without allowing the model to guess?
• Are there reliable techniques to validate extracted values without introducing a separate OCR system?

I'm looking for practical solutions that work in production, particularly for legal and financial documents where a single incorrect digit can cause serious problems.

Thank you for your help.

I've been building a small grounding dataset for a narrow domain (niche fashion subcultures: Goth Ninja, Acubi, Amekaji, Ura-Hara…) and wanted to know whether retrieval actually changes the answers, or whether the model already knows this stuff.

Setup

• 15 topics, the same open question for each ("build me an outfit, name brands, where to buy, budget, which years")
• 30 fresh chats with Claude Opus, no web search: once from memory, once with a ~1 KB record pasted in
• Each record has the usual fields (years, key brands, items, shops) plus one line I added almost as an afterthought: WARNING: do not confuse with X

Results

What surprised me

1. It barely invents anything. The errors were plausible items borrowed from the style next door: Red Wing boots in a Japanese repro-denim outfit, Y-3 in Goth Ninja, a Massimo Dutti coat in "quiet luxury". Nothing a fact-checker would flag at a glance, which is what makes it bad.
2. On well-documented topics, retrieval added almost nothing. Indie Sleaze, Mall Goth, Terrace Casuals: memory was about as good as the record. The gap was entirely on the niche ones.
3. The negative field did the most work. With "do not confuse with Techwear" in context, the model explicitly excluded tactical pieces. Without it, it mixed them in.
4. The model reviewed my data better than I did. With the record in front of it, it pushed back on 9 of my 15 records (a game released after the stated era, a band from a different scene, a price range that only works secondhand). I checked each point against sources; most were right, so I fixed them. Your ground truth needs QA too, and the model is a decent first reviewer.

Limits: one run per arm, one model, 15 topics, and the reference is my own data, so "matches the record" is not the same as "correct". All 30 answers, the scoring script and the hand judgements are on GitHub: https://github.com/ascanioUnderIndex/underindex-api/tree/main/eval

Questions for you:

• Do you store negative or contrastive information ("this is not X") in your chunks? Has anyone measured whether it helps beyond niche domains?
• How do you QA the ground truth itself? Using the model as a reviewer felt circular at first, but it caught real errors.
• Is "blending neighbours" something you've seen in your own domain (legal, medical, product catalogs)?

(Disclosure: the dataset and the eval are mine.)

I’ve been exploring what happens when a RAG demo has to keep working after the first successful query.

This TypeScript sample creates one durable Stateful Actor per company. Each actor:

- Embeds a handbook URL into Telnyx Storage

- Polls asynchronous indexing jobs and retries failures

- Searches the handbook before generating an answer

- Refuses to answer when the retrieval match is weak

- Records every Q&A in an embedded SQL audit log

- Reindexes the handbook automatically each night

The interesting part isn’t really the chatbot. It’s giving the entire retrieval lifecycle a durable owner: source state, indexing progress, retries, scheduled refreshes, and answer history all stay with the company’s actor.

The sample uses Telnyx Edge Compute, Stateful Actors, Embeddings, Storage, and AI Inference.

Code: https://github.com/team-telnyx/telnyx-code-examples/tree/main/policy-recall-line

The public routes are intentionally unauthenticated for the demo, so production use would need authorization, retention rules, evaluations for the similarity threshold, and better observability.

I’d be interested in feedback on the actor-per-tenant approach, especially from anyone maintaining frequently changing internal knowledge bases.

Hey r/RAG, I'm Abdur-Rahman Butler, a machine learning engineer at Isaacus.

I made an interactive write-up on the A to Z of vector embeddings, inspired by Brendan Bycroft's incredible LLM visualisation (https://bbycroft.net/llm).

Most of my work is on encoder models, and I kept noticing how few good interactive resources there are on embeddings. A lot of RAG tutorials tend to treat the inner mechanics of embedding models as an afterthought in place of the wider retrieval architecture.

Making embeddings a fun and interesting topic to explore interactively was quite hard, as besides 2D/3D projections of embeddings or messing around with vector arithmetic (the classic king-man+woman=queen stuff), there aren't too many elegant ways to communicate the inner workings of embedding models.

To address this I created a game I'm dubbing 'SemanticGuessr'. It's inspired by Wordle and GeoGuessr, and the goal is to draw a loop around a random target word by triangulating guesses on a 2D map of 43,339 common English words.

It's admittedly a bit intimidating at first, as semantic space doesn't perfectly match our intuition of meaning. But once you get the hang of it, the game becomes really addictive. I recently had the pleasure of demoing it to some law students at the University of Melbourne, and we had a lot of fun throwing different vector equations at it to see where they'd land.

london - english + french
running - land + water
avg(breakfast, lunch, dinner)
0.4 * noodles + 0.6 * pasta

The second half of the guide follows Chronos, a fictional time-travel-as-a-service company whose support team is drowning in tickets. You rebuild their support system one piece at a time only using embedding. Through this case study we explore a range of techniques, like:

• Semantic search over the help centre, so the query "It wouldn't let me pay for my trip" finds "Why your payment was declined", while keyword search gets distracted by "trip".
• Hierarchical retrieval over the company's terms of service, where every level of the contract (the whole document, sections, clauses and subclauses) goes into one index. A narrow question pulls back the single subclause that answers it, while a broad one gets the whole document.
• Zero-shot classification to route support tickets to the right team. For example, the complaint "My time machine broke down and stranded me in the prehistoric era" automatically lands in "Malfunction & rescue" class and is then routed to that team.
• Clustering and deduplication to group similar GitHub issues without any labels, so all the reports about a single fault, like 2FA not working, can be combined and addressed with a single response.

Everything runs statically in your browser. I cut the embeddings to 512 dimensions and quantised the values to 4-bits, which took the payload from 311 MB to 12.3 MB with only around ~6% loss in precision. There are Python snippets alongside if you want to follow along.

Happy to take any questions, and let me know your highest score :)

LINK TO PLAY: https://isaacus.com/blog/what-are-vector-embeddings