r/Rag • • Dec 05 '25 Bravo

Discussion Reasoning vs non reasoning models: Time to school you on the difference, I’ve had enough

2 Upvotes

People keep telling me reasoning models are just a regular model with a fancy marketing label, but this just isn’t the case.

I’ve worked with reasoning models such as OpenAI o1, Jamba Reasoning 3B, DeepSeek R1, Qwen2.5-Reasoner-7B. The people who tell me they’re the same have not even heard of them, let alone tested them.

So because I expect some of these noobs are browsing here, I’ve decided to break down the difference because these days people keep using Reddit before Google or common sense.

A non-reasoning model will provide quick answers based on learned data. No deep analysis. It is basic pattern recognition. 

People love it because it looks like quick answers and highly creative content, rapid ideas. It’s mimicking what’s already out there, but to the average Joe asking chatGPT to spit out an answer, they think it’s magic.

Then people try to shove the magic LLM into a RAG pipeline or use it in an AI agent and wonder why it breaks on multi-step tasks. Newsflash idiots, it’s not designed for that and you need to calm down.

AI does not = ChatGPT. There are many options out there. Yes, well done, you named Claude and Gemini. That’s not the end of the list.

Try a reasoning model if you want something aiming towards achieving your BS task you’re too lazy to do.

Reasoning models mimic human logic. I repeat, mimic. It’s not a wizard. But, it’s better than basic pattern recognition at scale.

It will break down problems into steps and look for solutions. If you want detailed strategy. Complex data reports. Work in law or the pharmaceutical industry. 

Consider a reasoning model. It’s better than your employees uploading PII to chatGPT and uploading hallucinated copy to your reports.


r/Rag • • Jan 21 '26

Discussion We almost wasted a month building RAG… then shipped it in 3 days

6 Upvotes

When we started building our RAG MVP, we almost made the classic mistake:

spending weeks only on chunking, storing, and retrieval.

But then we asked ourselves:

Why are we reinventing this for an MVP?

So instead, we did something simple.

We searched for open-source RAG products that already work well and found projects like Dify and RAGFlow.

Then we went deep into their code.

Claude helped a lot in understanding modules, data flow, and architecture.

Once we understood how Dify structures the full RAG pipeline, we implemented the same architecture in our system.

Result: end-to-end RAG working in 4 days, not 1 month.

What do you think about this approach?


r/Rag • • Sep 02 '25

Tools & Resources Introducing Papr: Everyone's engineering context. We're predicting it.

0 Upvotes

Everyone's Engineering Context. We're Predicting It.

We started our journey when ChatGPT 3.5 launched, we built a memory GPT / plugin on ChatGPT to make ChatGPT remember. We started simple by building a vector database using pinecone. Things were working well until we realized once you add a lot of data the more data you add the worse retrieval gets.

We then added a knowledge graph, to our vector database using Neo4j and used LLM's like ChatGPT to dynamically take un-structured data like meetings, slack messages, documents and build a knowledge graph using a fixed ontology. That worked really well, and we saw significant improvements in retrieval accuracy for multi-hop queries.

For example, we can take customer zoom meetings, slack threads, docs and add it to Papr memory. Then our memory graph (hybrid vector + knoweldge graph) included both the unstructured data but also built a web of connected memories that includes tasks (action items), people, companies (i.e. customers), projects, insights, opportunities, code snippets and built relationships between them. I can easily ask Papr to search for top problems in my customer discovery calls and it will find core problems, insights and tasks extracted from unstructured data in meetings, email and slack.

This then started to work well, but again as soon as I hit scale it become harder to find relevant answers to my real-world queries. We clearly needed a way to measure retreival accuracy and improve it. So we created a new metric that we are calling retrieval-loss.

We created the retrieval-loss formula to establish scaling laws for memory systems, similar to how Kaplan's 2020 paper revealed scaling laws for language models. Traditional retrieval systems were evaluated using disparate metrics that couldn't capture the full picture of real-world performance. We needed a single metric that jointly penalizes poor accuracy, high latency, and excessive cost—the three factors that determine whether a memory system is production-ready. This unified approach allows us to compare different architectures (vector databases, graph databases, memory frameworks) on equal footing and prove that the right architecture gets better as it scales, not worse.

Whe then discovered something fascinating, if we treat memory as a prediction problem, actually with more data our prediction models improve and thus retreival gets better with more data. We built initial prediction memory layer on top of our hybrid memory graph architecture that started to demonstrate solid results even with scale!

Today I personally have more than 22k memories which is ~20 million tokens and I personally use papr.ai to find relevant context daily and it simply works!

The Formula:

Retrieval-Loss = −log₁₀(Hit@K) + λL·(Latency_p95/100ms) + λC·(Token_count/1000)

Where:

  • Hit@K = probability that the correct memory is in the top-K returned set

  • Latency_p95 = tail latency in milliseconds

  • λL = weight that says "every 100 ms of extra wait feels as bad as dropping Hit@5 by one decade

  • λC = weight for cost

  • Token_count = total number of prompt tokens attributable to retrieval

Traditional RAG (vector search): More data → volatile performance → agent death Our approach: More data → stable performance → agents that actually scale

The key insight? Memory is a prediction problem, not a search problem.

Instead of searching through everything, we predict the 0.1% of facts your agent needs and surface them instantly. Our predictive memory graph achieves:

We turned the scaling problem upside down. More memories now make your agents smarter, not slower.

Ready to give Papr a try?

👉 Read the full story: https://open.substack.com/pub/paprai/p/introducing-papr-predictive-memory?utm_campaign=post&utm_medium=web

👉 Start building: platform.papr.ai

👉 Join our community: https://discord.gg/J9UjV23M

Built with: MongoDB, Neo4j, Qdrant, #builtwithmongo


r/Rag • • Jan 04 '25

Discussion PSA Announcement: You Probably Don't Need to DIY

6 Upvotes

Lately, there seem to be so many posts that indicate people are choosing a DIY route when it comes to building RAG pipelines. As I've even said in comments recently, I'm a bit baffled by how many people are choosing to build given how many solutions are available. And no, I'm not talking about Langchain, there are so many products, services, and open source projects that solve problems well, but it seems like people can't find them.

I went back to the podcast episode I did with Kirk Marple from Graphlit, and we talked about this very issue. Before you DIY, take a little time and look at available solutions. There are LOTS! And guess what, you might need to pay for some of them. Why? Well, for starters, cloud compute and storage isn't free. Sure, you can put together a demo for free, but if you want to scale up for your business, the reality is you're gonna have to leave Collab Notebooks behind. There's no need to reinvent the wheel.

https://youtu.be/EZ5pLtQVljE


r/Rag • • Dec 23 '25

Discussion GPT 5.2 vs. Gemini 3: The "Internal Code Red" at OpenAI and the Shocking Truth Behind the New Models

1 Upvotes

We just witnessed one of the wildest weeks in AI history. After Google dropped Gemini 3 and sent OpenAI into an internal "Code Red" (ChatGPT reportedly lost 6% of traffic almost in week!), Sam Altman and team fired back on December 11th with GPT 5.2.

I just watched a great breakdown from SKD Neuron that separates the marketing hype from the actual technical reality of this release. If you’re a developer or just an AI enthusiast, there are some massive shifts here you should know about.

The Highlights:

  • The Three-Tier Attack from OpenAI moving away from "one-size-fits-all" [01:32].

  • Massive Context Window: of 400,000 token [03:09].

  • Beating Professionals OpenAI’s internal "GDP Val" benchmark

  • While Plus/Pro subscriptions stay the same, the API cost is skyrocketing. [02:29]

  • They’ve achieved 30% fewer hallucinations compared to 5.1, making it a serious tool for enterprise reliability [06:48].

The Catch: It’s not all perfect. The video covers how the Thinking model is "fragile" on simple tasks (like the infamous garlic/hours question), the tone is more "rigid/robotic," and the response times can be painfully slow for the Pro tier [04:23], [07:31].

Is this a "panic release" to stop users from fleeing to Google, or has OpenAI actually secured the lead toward AGI?

Check out the full deep dive here for the benchmarks and breakdown: The Shocking TRUTH About OpenAI GPT 5.2

What do you guys think—is the Pro model worth the massive price jump for developers, or is Gemini 3 still the better daily driver?


r/Rag • • Jan 28 '25

Discussion Deepseek and RAG - is RAG dead?

6 Upvotes

from reading several things on the Deepseek method of LLM training with low cost and low compute, is it feasible to consider that we can now train our own SLM on company data with desktop compute power? Would this make the SLM more accurate than RAG and not require as much if any pre-data prep?

I throw this idea out for people to discuss. I think it's an interesting concept and would love to hear all your great minds chime in with your thoughts


r/Rag • • Jul 16 '26

Discussion With 1M+ token context windows becoming standard, is traditional RAG becoming obsolete for most use cases?

1 Upvotes

Genuinely curious where this community lands on this. With Gemini, Claude, and others pushing context windows past 1M tokens, I keep seeing takes that RAG is just a workaround for models that can't hold enough context, and that it'll fade out as context windows grow and get cheaper.

But cost, latency, and retrieval precision still seem like real advantages for RAG even with huge context windows, especially at scale or when you need up-to-date/private data.

For people running RAG in production: has long-context made you rethink your architecture, or is it solving a completely different problem than what RAG solves? Where do you still think RAG clearly wins?


r/Rag • • Dec 21 '25

Discussion Google's NEW Gemini 3 Flash Is INSANE Game-Changer | Deep Dive & Benchmarks 🚀

0 Upvotes

Just watched an incredible breakdown from SKD Neuron on Google's latest AI model, Gemini 3 Flash. If you've been following the AI space, you know speed often came with a compromise on intelligence – but this model might just end that.

This isn't just another incremental update. We're talking about pro-level reasoning at mind-bending speeds, all while supporting a MASSIVE 1 million token context window. Imagine analyzing 50,000 lines of code in a single prompt. This video dives deep into how that actually works and what it means for developers and everyday users.

Here are some highlights from the video that really stood out:

  • Multimodal Magic: Handles text, images, code, PDFs, and long audio/video seamlessly.

  • Insane Context: 1M tokens means it can process 8.4 hours of audio one go.

  • "Thinking Labels": A new API control for developers

  • Benchmarking Blowout: It actually OUTPERFORMED Gemini 3.0 Pro

  • Cost-Effective: It's a fraction of the cost of the Pro model

Watch the full deep dive here: Google's Gemini 3 Flash Just Broke the Internet

This model is already powering the free Gemini app and AI features in Google Search. The potential for building smarter agents, coding assistants, and tackling enterprise-level data analysis is immense.

If you're interested in the future of AI and what Google's bringing to the table, definitely give this video a watch. It's concise, informative, and really highlights the strengths (and limitations) of Flash.

Let me know your thoughts!


r/Rag • • Feb 05 '25

Simple Guide on how to build a RAG system

0 Upvotes

r/Rag • • May 05 '26

Discussion Why your Enterprise AI has Goldfish Memory (and why RAG isn't fixing it)

0 Upvotes

I’ve spent the last few months talking to Ops leaders who are frustrated with the one-step-forward, two-steps-back nature of their AI implementations.

The story is always the same: They build a custom GPT or a standard RAG (Retrieval-Augmented Generation) system, feed it their SharePoint/confluence, and it works great for about a week. Then, the hallucinations start. The AI forgets a policy update from last Tuesday or mixes up a 2019 contract with a 2024 renewal.

The problem isn't the LLM. It’s that your AI doesn't have a Context Graph.

Most people assume a Knowledge Graph is enough. A Knowledge Graph is great at saying "Entity A is related to Entity B" (e.g., Paris is the capital of France). It’s a static map.

But in a high-stakes business, facts aren't static. They have temporal traces and causal edges.

A Context Graph (what we’re seeing firms like 60x and others move toward) doesn't just record what is connected; it records the how, when, and why.

  • Temporal Context: It knows that a 30% discount approved by a VP on Friday was a one-off exception for a specific client, not a new company-wide policy.

  • Decision Traces: It maps the lineage of a decision. When the AI gives an answer, it’s not just pulling a text chunk; it’s traversing a graph of past approvals and meeting outcomes.

If you’re building AI for a business that relies on institutional memory, you have to stop treating your data like a giant library and start treating it like a living network of decisions.


r/Rag • • Jul 31 '25

Tutorial Why pgvector Is a Game-Changer for AI-Driven Applications

Thumbnail
reddit.com
0 Upvotes

r/Rag • • Nov 11 '25

Tools & Resources I think ChatRAG has proven that there is a market for RAG boilerplates! 🚀

2 Upvotes

Hi!

Carlos here! I'm the creator of ChatRAG, a Next.js boilerplate for launching cloud-based RAG-powered AI chatbots in minutes, instead of weeks or even months.

I launched ChatRAG 8 days ago, and this was one of the first places I posted about it. Since then, we've made $2.7k in revenue. And I think this is proof that there is a real demand for RAG tools and boilerplates that make the implementation of this technology much faster and easier.

I'm writing this to encourage others in this community to think about building other tools and/or boilerplates related to RAG since I think it's an underserved market, that is very willing to invest in new tools if they prove to make the DX quicker or easier. I don't think indie developers should leave this economic opportunity only to big companies with VC funding. There's real potential here for solo builders and small teams to create valuable solutions and capture market share.

Want to end this post by saying thank you to all of the members of this community that upvoted and/or commented on my original ChatRAG post from 8 days ago. Eternally grateful to you all. I hope to see more people building to make RAG more accessible for more people.

All the best,

Carlos


r/Rag • • Dec 01 '25

Discussion Embedding Drift: The Silent RAG Breaker Nobody Talks About

11 Upvotes

One of the strangest failure modes in RAG systems isn’t chunking, or embeddings, or rerankers.
It’s something quieter: embedding drift.

Your system still runs.
Your vector DB still returns results.
Nothing throws an error.

But retrieval quality slowly falls apart without telling you why.

Why embedding drift actually happens

1. Model updates shift the meaning-space

Even small updates to an embedding model can change how:

  • concepts cluster

  • relationships are encoded

  • similarity is interpreted

Your queries are now searching in a slightly different universe than the one your documents were embedded in.

2. Old documents + new embeddings

A common pattern:
Teams re-embed new docs, but keep old vectors untouched.

Now your vector DB is half “old space,” half “new space.”
Queries behave unpredictably across the two.

3. Changes in how structure is encoded

Newer embedding versions sometimes:

  • weight punctuation differently

  • handle tables/code blocks differently

  • encode metadata cues differently

Your retrieval stops matching the intuitions you built earlier.

How drift breaks retrieval without breaking anything

The system doesn’t fail loudly.
It fails silently.

Symptoms show up like:

  • top-k results that feel “off”

  • drops in recall that are hard to measure

  • unrelated chunks creeping into the context

  • rerankers fixing less than expected

  • parent-child mapping becoming unstable

It looks like the LLM is suddenly “worse,”
when the real issue is that query vectors and stored vectors no longer agree on meaning.

Why drift becomes a real problem in long-lived RAG systems

Any system with:

  • regular data updates

  • model upgrades

  • new chunking strategies

  • new retrieval setups

will eventually hit embedding drift.

The longer your index lives, the more the vector space drifts.
RAG pipelines depend on semantic consistency once that’s gone, everything feels fragile.

Practical defenses teams use

Not universal rules just common patterns that help:

  • re-embed the whole corpus together, not piecemeal

  • version every embedding alongside the chunk

  • A/B test retrieval before and after model upgrades

  • rebuild the index periodically instead of patching

  • use sparse search or hybrids as stabilizers when drift happens

Most issues people blame on the LLM are actually drift problems hiding upstream.

Embedding drift doesn’t look like a failure.
It looks like “RAG being weird today.”

But behind the scenes, the meaning-space changed and the system didn’t know it.

Keeping embeddings consistent over time matters far more than people think.

What have you seen in real setups?
Do you version embeddings, rebuild regularly, or rely on hybrids to stay stable?


r/Rag • • Jun 11 '25

Buy vs. Build: The RAG Solution Dilemma for CTOs

1 Upvotes

Retrieval-augmented generation (RAG) has emerged as a powerful approach for enhancing large language models with up-to-date, accurate information from proprietary data sources. Companies looking to leverage RAG make a critical decision: Should they build in-house custom solutions or purchase existing platforms? This choice carries significant implications for resource allocation, long-term maintenance, and ultimate success.

Buy vs. Build: The RAG Solution Dilemma for CTOs https://medium.com/@tselvaraj/buy-vs-build-the-rag-solution-dilemma-for-ctos-fed59543e159


r/Rag • • Mar 14 '25

Technically, is RAG the same thing as lossy compression?

0 Upvotes

I'm trying to wrap my head around RAG in general. If the goal is to take a large set of data and remove the irrelevant portions to make it fit into a context window while maintaining relevance, does this count as a type of lossy compression? Are there any lessons/ideas/optimizations from lossy compression algorithms that apply to the same space?

Conclusion:

  • Short answer: No

  • Long answer: Maybe a little at a higher level

  • Personally: Still helpful for me to think about, but probably shouldn't try and use this to "helpfully" explain RAG to anyone else.

To count as compression, a better description would be something like "query-specific semantic compression", because it does use lossy semantic compression (embeddings) to create do searches. It does dynamically determine relevance when figuring out which parts to use. And it does balance information density with information precision, similar to audio codecs balancing file size with sound quality. But it isn't trying to produce a compressed "copy" of the source.

So, ultimately, there may be some common information theory and signal processing ideas like frequency analysis since both are fundamentally about preserving the most important information while dealing with constraints. Not all thing fit nicely though. I try and look at a specific signaling concept like Fast Fourier Transforms which tries to decomposes signals into simpler component parts and find patterns not obvious in the original representation, FFT doesn't really fit at any lower level beyond what I just said.


r/Rag • • Sep 01 '25

Just learned how AI Agents actually work (and why they’re different from LLM + Tools )"

0 Upvotes

Been working with LLMs and kept building "agents" that were actually just chatbots with APIs attached. Some things that really clicked for me: Why tool-augmented systems ≠ true agents and How the ReAct framework changes the game with the role of memory, APIs, and multi-agent collaboration.

Turns out there's a fundamental difference I was completely missing. There are actually 7 core components that make something truly "agentic" - and most tutorials completely skip 3 of them. TL'DR Full breakdown here: AI AGENTS Explained - in 30 mins

  • Environment

  • Sensors

  • Actuators

  • Tool Usage, API Integration & Knowledge Base

  • Memory

  • Learning/Self-Refining

  • Collaborating (Multi-Agent System)

It explains why so many AI projects fail when deployed.

The breakthrough: It's not about HAVING tools - it's about WHO decides the workflow. Most tutorials show you how to connect APIs to LLMs and call it an "agent." But that's just a tool-augmented system where YOU design the chain of actions.

A real AI agent? It designs its own workflow autonomously with real-world use cases like Talent Acquisition, Travel Planning, Customer Support, and Code Agents

Question for the community: Has anyone here successfully built autonomous agents that actually work in production? What was your biggest challenge - the planning phase or the execution phase?

  • Also curious about your experience with ReAct framework vs other agentic architectures.


r/Rag • • Sep 20 '24

Tools & Resources Comparison of the Top RAG Frameworks

9 Upvotes

We’ve just released our 2024 guide on the top RAG frameworks. Based on our RAG deployment experience, here are some key factors to consider when picking a framework:

Key Factors for Selecting a RAG Framework:

  1. Deployment Flexibility: Does it support both local and cloud deployments? How easily can it scale across different environments?

  2. Data Sources and Connectors: What kind of data sources can it integrate with? Are there built-in connectors?

  3. RAG Features: What retrieval methods and indexing capabilities does it offer? Does it support advanced querying techniques?

  4. Advanced Prompting and Evaluation: How does it handle prompt optimization and output evaluation?

Comparison page: https://pathway.com/rag-frameworks

It includes a detailed tabular comparison of several frameworks, such as Pathway (our framework with 8k+ GitHub stars), Cohere, LlamaIndex, LangChain, Haystack, and the Assistants API.


r/Rag • • May 29 '26

Showcase The model was never the bottleneck. Got local RAG working on 8GB RAM with full citation accuracy! Here's what actually changed

10 Upvotes

Been building a local RAG stack for technical documents , started with aviation manuals (the kind you legally can't upload to ChatGPT), now getting requests from nursing students who need their assigned textbooks to answer questions with citations, not the internet's best guess. The wall I hit: my demo rig has 32GB RAM. A nursing student's laptop doesn't. Same retrieval quality was non-negotiable — these are students who need to cite their actual assigned textbook, not a hallucinated approximation of it.

Here's what actually mattered when designing for constrained hardware: Swapped down to gemma4:e2b, 2 billion parameters, and citation accuracy didn't move. Ran a crosswind limitation query against a 400-page aviation AFM. Four page citations came back. Page 57, 357, 376, 459. All correct. The retrieval is doing the work, not the inference.

Chunk boundaries matter more than chunk size. At 1800 characters with section-aware splits, a small model reads a clean retrieved chunk and formats a precise answer. Give it a badly split chunk and even a large model struggles. Spent more time on boundary logic than on anything else in the stack.

Protected blocks can't be split. For safety-critical content; WARNING, CAUTION, DANGER blocks in aviation manuals, critical drug interaction notes in nursing textbooks, a chunk that splits mid-warning is worse than no answer. Made these atomic. Non-negotiable. Cloud toggle for students who need it. For machines that truly can't run local inference, routing to Claude Sonnet via API key gives the same retrieval quality with zero local compute. The vectors stay local. Only the question travels.

Stack: pdfplumber + ChromaDB + Ollama on the local side, Anthropic API for cloud toggle. Runs clean on an M2 MacBook 8GB.

Happy to go deeper on the boundary-aware chunking logic or the cloud/local toggle architecture if anyone's building something similar.


r/Rag • • Dec 24 '25

Discussion Vibe coded a RAG, pass or trash?

0 Upvotes

Note for the anti-vibe-coding community; don't bother roasting, I am okay with it's consequences.

Hello everyone, I've been vibe-coding a SaaS that I see fit in my region and is mainly reliant on RAG as a service, but due to lack of such advanced tech skills.. I got no one but my LLMs to review my implementations.. so I decided to post it here appreciating surely if anyone could review/help;

The below was LLM generated based on my codebase[still under dev];

## High-level architecture


### Ingestion (offline/async)
1) Preflight scan (format + size + table limits + warnings)
2) Parse + normalize content (documents + spreadsheets)
3) Chunk text and generate embeddings
4) Persist chunks and metadata for search
5) For large tables: store in dataset mode (compressed) + build fast identifier-routing indexes


### Chat runtime (online)
1) User message enters a tool-based orchestration loop (LLM tool/function calling)
2) Search tool runs hybrid retrieval and returns ranked snippets + diagnostics
3) If needed, a read tool fetches precise evidence (text excerpt, table preview, or dataset query)
4) LLM produces final response grounded in the evidence (no extra narration between tool calls)

## RAG stack

### Core platform
- Backend: Python + Django
- Cache: Redis
- DB: Postgres 15


### Vector + lexical retrieval
- Vector store: pgvector in Postgres (per-chunk embeddings)
- Vector search: cosine distance ANN (with tunable probes)
- Lexical search: Postgres full-text search (FTS) with trigram fallback
- Hybrid merge: alias/identifier hits + vector hits + lexical hits


### Embeddings
- Default embeddings: local CPU embeddings via FastEmbed (multilingual MiniLM; 384-d by default)
- Optional embeddings: OpenAI embeddings (switchable via env/config)


### Ranking / selection
- Weighted reranking using multiple signals (vector similarity, lexical overlap, alias confidence, entity bonus, recency)
- Optional cross-encoder reranker (sentence-transformers CrossEncoder) supported but off by default
- Diversity selection: MMR-style selection to avoid redundant chunks


### Tabular knowledge handling
Two paths depending on table size:
- “Preview tables”: small/medium tables can be previewed/filtered directly (row/column selection, exact matches)
- “Dataset mode” for large spreadsheets/CSVs:
  - store as compressed CSV (csv.gz) + schema/metadata
  - query engine: DuckDB (in-memory) when available, with a Python fallback
  - supports filters, exact matches, sorting, pagination, and basic aggregates (count/sum/min/max/group-by)


### Identifier routing (to make ID lookups fast + safer)
- During ingestion, we extract/normalize identifier-like values (“aliases”) and attach them to chunks
- For dataset-mode tables, we also generate Bloom-filter indexes per dataset column to quickly route an identifier query to the right dataset(s)


### Observability / evaluation
- Structured logging for search/read/tool loop (timings and diagnostics)
- OpenTelemetry tracing around retrieval stages (vector/lexical/rerank and per-turn orchestration)
- Evaluation + load testing scripts (golden sets + thresholds; search and search+read modes)
------------------------------------------------------------------------

My questions here;
  
Should I stop? Should I keep going? the SaaS is working and I have tested on few large complex documents, it does read and output is perfect. I just fear whatever is waiting for me on production, what do you think?

If you're willing to help, feel free to ask for more evidence and I'll let my LLM look it up on the codebase.

r/Rag • • Jul 23 '26

Discussion 111 agents burned an entire Claude Max limit in 30 minutes—and produced no report

0 Upvotes

We created a research process that started 111 agents put 123 claims in a queue, for checking and used up the entire Claude Max limit in around 30 minutes.

The unexpected thing was that the issue wasn't the model. The issue was spread of tasks and checking claims too late.

The biggest change came from one rule:

The agent that finds a claim cannot check that claim.

We also made sure every accepted claim had a primary-source URL and a supporting quote.

After making that change the same subscription lasted 10 times longer.

How are you stopping verification spread from getting out of control in -agent RAG systems?


r/Rag • • Mar 22 '25

Chatbot using RAG Flask and React.js

0 Upvotes

I want the steps to build a chatbot using rag, flask, and react.js and Ollama, Qdrant, and Minio to help HRs filter CVs


r/Rag • • Aug 16 '26

Discussion I wanna build an ai startup like these below ( Knowledge Graph/RAG/Ontology) ⬇️ and Whats ur thought? I need help please.

0 Upvotes

Startup should be like an AI-driven tech company. Knowledge graph/Ontology to build and scale the knowledge graph layer underpinning a nee generation of intelligent enterprise product. Designing and shipping kg, not just conceptual ontology work.

Shaping a graph and ontology platform that power:

• AI retrieval and RAG workflows
• Entity linkage and reasoning systems
• Cross-domain and temporal knowledge modeling
• Regulatory and compliance intelligence products
• Agentic AI applications

Working closely with AI/ML to turn complex, unstructured information into structured, queryable intelligence that directly feeds live AI systems.

Tech stack like these:

• Extensive use of Neo4j and Cypher in live (production) environments
• Comprehensive ontology/taxonomy modeling
• Python engineering skills
• Knowledge graph integration with LLMs, RAG, or vector search systems
• Experience balancing formal semantics with practical application requirements
• Ability to provide technical leadership while remaining actively involved in hands-on application development
• RDF/OWL, inference engines, entity resolution, legal/regulatory data, ESG, healthcare, or bthe pharmaceutical industry would be highly valuable.

The challenge of building the intelligence layer behind complex AI products at scale.

As for my questions:

  1. I plan to launch this venture/business as a solo founder. Do you think this makes sense?

  2. How do you envision this company operating exactly?

  3. If I establish the company, how should I explain my business model to the companies I intend to serve in the real world? (I am genuinely apprehensive about this.) After all, things don't always go as expected in this field. Considering that many companies don't even know what artificial intelligence is, how will they react to this type of business? In other words, will they truly understand what I do?

  4. Which types of companies do you think would benefit most from this business? What problems would it solve most effectively?

  5. What are the real-world problems and complaints companies have regarding this area?


r/Rag • • Sep 09 '25

The Agentic RAG Playbook

0 Upvotes

Me & my friends dropped this playbook on Agentic RAG - hard focus on reliable deployment.

P.S. The playbook calls out the "validation engine" as a core piece - for true verification, not just retrieval.

Playbook - https://futureagi.com/mastering-agentic-rag?utm_source={{ebookmark0809}}&utm_medium={{organic}}&utm_campaign={{content_marketing}}


r/Rag • • Apr 06 '25

Will RAG method become obsolete?

0 Upvotes

https://ai.meta.com/blog/llama-4-multimodal-intelligence/

10M tokens!

So we don't need RAG anymore? and next so what 100M Token?


r/Rag • • Jan 11 '26

Discussion Could RAG as a service become a mainstream thing?

0 Upvotes

Now I know what I'm about to say is technical and will fly off the head of a lot of people who lurk here and I'd like this thread to be approachable to those people also I'd like to give them some context. I would post this on other dev focused forums but I dont have enough clout there so this is what I had in mind. Dnt worry I wont do deep dive on the math or the specifics. Even if you are non tech person. I feel you will still find this interesting as I broke down very simply and you'll gain a greater understanding of LLMs as whole compared to everyone

Traditionally we all been building the same stack since 2021 for chabots and RAG based LLMs. PDF to LangChain to Chunking to Embeddings to Pinecone to Retrieval.

If this seems Greek to you I’ll explain how a typical agent specific chatbot or RAG powered LLM actually works.You upload a PDF then LangChain splits it into chunks each chunk gets converted into a dense vector using an embedding model like those words get tokenized and then given a positional ID so for example 'John owns this site' can be converted into ["John": 1.3, "owns": 2.0, "site" : 3.2...] with text-embedding-ada-002 or all-MiniLM(name of the model that does this). These vectors live in a high dimensional semantic space usually 384 to 1536 dimensions. Each vector represents the meaning of the text, and are converted into vectors yes like you learned in high school geometry vectors that have direction and magnitude.

When a user asks a question, the query is also turned into a vector. like 'who owns this site' becomes [1.1,2.0,3.2....] which is similar to the chunk that existed earlier We then use cosine similarity or sometimes dot product

Linking an article that goes into greater depth

https://spencerporter2.medium.com/understanding-cosine-similarity-and-word-embeddings-dbf19362a3c

we use those o find the chunks whose vectors are most similar to the query vector. Those relevant chunks are pulled from the vector database (Pinecone, Weaviate, Chroma, etc.) and stuffed into the LLM’s prompt this way the entire context need not be fed to the LLM for output just the part that is relevant which results in millions of tokens being queried in milli seconds

The LLM then processes this prompt through dozens of layers. The lower layers mostly handle syntax, token relationships, and grammar and higher layers build abstract concepts, topics, and reasoning. The final output is generated based on that context.

This is how it fundamentally works it is not magic just advanced math and heavy computation. This method id powerful because this is basically allows you to use something calling grounding which is another concept used in machine learnings for your LLM in your own data and query millions of tokens in milliseconds.

But it’s not bulletproof and here is where LangChain which is a Python framework comes in with orchestration by adding prompt engineering, chain of thought, agents, memory to reduce hallucinations and make the system more reliable.

https://docs.langchain.com/

All that is good but here’s what I’ve been thinking lately and the industry also seems to be moving in the same direction

Instead of this explicit LLM + LangChain + Pinecone setup why can’t we abstract the entire retrieval part into a simple inference based grounded search like what Google’s NotebookLM does internally. In NotebookLM, you just upload your sources (PDFs, notes, etc.) like here if I uploaded a research paper and I can immediately start chatting.

There’s no manual chunking, no embedding model choice, no vector DB management, no cosine similarity tuning. Google’s system handles all of that behind the scenes. We don't exactly know how it happens because that is gatekept but it uses something called In model RAG. The retriever is most probably co-trained or tightly coupled with the LLM itself instead of being an external Pinecone call. Google has published research papers in this area

https://levelup.gitconnected.com/googles-realm-a-knowledge-base-augmented-language-model-bc1a9c9b3d09

and NotebookLLM probably uses a more advanced version of that, it is much simpler, easier and faster to implement and very less likely to hallucinate. This is especially beneficial for low-scale, personal, or prototyping stuffbecause there is zero infrastructure to manage and no vector DB costs. it is just upload and as

Google has actually released a NotebookLM API for enterprise customers which is what inspired me to make this thread

https://docs.cloud.google.com/gemini/enterprise/notebooklm-enterprise/docs/api-notebooks#:~:text=NotebookLM%20Enterprise%20is%20a%20powerful,following%20notebook%20management%20tasks%20programmatically:

The only roadblock is that NotebookLLM rn only allows for 1 million tokens or around 50 books or for me an enterprise customer around 300 books which for the projects that I worked on is enough so if they remove that limit. Google could indeed make the traditional stack obsoleteand charge a heafy sum for a RAG as a service of sorts which already exist and with NotebookLLM API, Vertex API we may be moving towrads ot sppn but google might take the cake with this one in the future I'd be interested in talking about this someone familiar with RAG retrieval pipelines and from Seniors working in this space. Are you still building custom pipelines, or are you moving to managed retrieval APls?