Anyone here tried the "compile instead of RAG" approach?
Posted by riddlemewhat2@reddit | LocalLLaMA | View on Reddit | 8 comments
Been seeing this idea where instead of doing the usual RAG loop, you compile all your sources into a markdown wiki first, then query that directly. The interesting part is that saved answers become part of the wiki too. The more you use it, the richer the context gets.
Came across this repo the other day while going through Karpathy's post: https://github.com/atomicmemory/llm-wiki-compiler
Not sure how it holds up at scale, but the idea of building a persistent corpus instead of re-fetching context every time feels like a meaningfully different approach. Curious if anyone's actually run this in production and what the tradeoffs looked like
robotrossart@reddit
We implemented this for our robotic demonstrator. The wiki was created from the source code and we then created a separate MD file from the log files where the machine produces during operation. We want to run this using a local model ( for data sovereignty and model stability ). We had to use a RAG to feed the local model for performance reasons. So, the answer to your question is actually both! The beauty of the wiki is that it’s human readable. You can see it here: https://api.robotross.art/atf/index.html
Our repo is here: https://github.com/UrsushoribilisMusic/agentic-fleet-hub
MihaiBuilds@reddit
the "compile into a wiki" idea is cool but I think it hits a wall pretty fast. once you have a few thousand entries the markdown wiki itself becomes the problem. the sweet spot imo is hybrid, keep the persistence and accumulation benefits but put a real search layer on top (vector + full text). that way the corpus grows without turning into a mess
crantob@reddit
how? why?
CreativeKeane@reddit
What is a everyone's RAG or Compile stack and progress?
How are u guys converting documents (word, pdf, etc) into texts, chunking, and labeling them for Rag or Compile?
I'm super curious.
No-Refrigerator-1672@reddit
RagFlow - it's an all-in-one solution: it does document processing, OCR, chunking, keywording, different RAG tecniques (i.e. RAPTOR, Knowledge Graph extraction), then exposes the results either as MCP server, as rag agent that can be queired as OpenAI API compatible LLM, or in web interface.
Embarrassed_Art_6966@reddit
i just set up a local thing that does it all automatically. it handles the parsing and chunking, then gives you a few ways to actually use the data.
honestly its the only way ive found that doesnt make me want to pull my hair out trying to glue five different tools together.
CreativeKeane@reddit
Good to know. Noted. I'll look more into it. I appreciate you pointing me in the right direction.
I have dabbled with Kernel Memory / Azure AI Search and it KM has its own ingestion process but I found it to be bad, and curious what other tools and alternatives other are using out there.
No-Refrigerator-1672@reddit
RAG Flow can be good or bad; as it assumes you'll finetune ingestion, chunking and retrieval parameters yourself, on a knowledge base by base basis. If you're willing to spend time doing that, then it's pretty good; 77k start on GitHub speak for themself.