Speaker:
Asif Qamar
LinkedIn: https://www.linkedin.com/in/asifqamar/
Technology Leader | AI/Data Scientist | Computer Scientist | Educator | Theoretical Particle Physicist
For reference, the paper is available on arXiv: https://arxiv.org/abs/2501.07391v1
Key Highlights of the session :
- Baseline RAG (2023 Standard) – Uses a retriever (search engine/vector DB) and a generator (LLM) to fetch Top-K results and generate responses, aiming to reduce hallucinations, update stale information, and improve retrieval quality.
- Challenges in RAG – Issues include hallucinations, unreliable answers, poorly framed queries, ineffective chunking, and difficulties retrieving from multi-source data.
- Hallucination Mitigation – Techniques like back-referencing (e.g., Perplexity AI) force LLMs to cite sources, reducing unreliable responses.
- Query Quality & Expansion – Query refinement via LLM-assisted rewriting, knowledge graphs, and hybrid search (semantic + keyword retrieval) significantly improves search precision.
- Optimal Chunking for Retrieval – AI-driven chunking and propositional decomposition (e.g., DenX Retrieval) outperform single-document vector indexing, ensuring better information retrieval.
- Hierarchical & Multi-source Retrieval – Enterprises integrate structured databases, internal documents, and web sources to improve retrieval accuracy across multiple domains.
- Graph-RAG for Knowledge Structuring – Organizes knowledge hierarchically to improve retrieval quality but increases computational and API costs.
- Agentic RAG for Adaptive Retrieval – AI agents allocate compute dynamically based on query complexity, improving accuracy but increasing latency and cost—critical in high-stakes fields like law and medicine.
- Key Research Findings on RAG Performance – Mixed conclusions on LLM size impact, significant prompt sensitivity, optimal chunking strategies, retrieval stride importance, and multilingual retrieval challenges.
- Disagreements & Practical Insights – The group challenges research claims that chunk size, knowledge base size, and query expansion have little effect, arguing that corpus quality, diverse chunking, and human query limitations make these factors critical.