AutoRAG Huggingface Space Release : Optimize RAG & evaluation dataset generation without installation.

[-]

3oclockam@reddit

Looks good. Will this handle tables in pdfs?

[-]

jeffrey-0711@reddit (OP)

If you select OCR model like LlamaParse, it supports tables! Just select LlamaParse from here and insert llama cloud key. Or you can use upstage document parser, which supports pdf tables also.

We support table hybrid parser also in original AutoRAG.

[-]

3oclockam@reddit

Thanks for suggesting OCR. I need a local solution so will look at Tesseract

[-]

jeffrey-0711@reddit (OP)

Huggingface Space List

AutoRAG Github => https://github.com/Marker-Inc-Korea/AutoRAG

[-]

jeffrey-0711@reddit (OP)

Optimize document-based LLM Chatbots (RAG system) without installation! AutoRAG Hugging Face Space Now Open.

For those who have developed RAG systems, you are always think about, "How can I improve performance?" It’s one of the most frequent thoughts when working on RAG development.

However, there is no "ultimate way" to boost RAG performance, and you must study and do experiment with various RAG techniques.

This is where AutoRAG comes in to solve these challenges.

Now, you can use AutoRAG directly on Hugging Face Spaces without installation.

With this update, anyone even non-developers, can easily do the following tasks:

Create an LLM chatbot (RAG) based on their own PDF documents
Optimize the created RAG pipeline and compare it to an unoptimized version
Directly use the optimized RAG pipeline
Automatically generate datasets for optimization

In short, anyone can now easily create optimized document-based LLM chatbots with just a few clicks, without installation.

[-]

wikarina@reddit

Thanks a lot was looking exactly for that!

[-]

_underlines_@reddit

Some inputs:

RAG is not only QA, it can be instructions, summarization, multi step problem solving
The search space or hyperparameters for RAG Optimization is not just chunk strategy, length and overlap, it's also: embedding model, DOC format, prompt and hundrets of RAG techniques like HyDE, RAPTOR, Reranking, Hybrid Search etc.
A human expert generated eval set is worth gold: Using synthetic eval sets only leads to a bias. I once got a 100% perfect score from 300 synthetic data points in RAGAS :)

Question:

Does AutoRAG work with a certain library (haystack, llamaindex, etc.)?

[-]

Hello! Totally agree about all three points that you mentioned. - You can test instructions or summarization task using AutoRAG also. (But not on the huggingface unfortunately). - Yes, the search space is huge. We supports query expansion like hyde, query decompose. Also lexical retrieval and hybrid retreival (not just rrf, cc with 4 normalization method + hyperparameter optimization). Many passage rerankers, passage filters and augmenter. And prompt technique + various embedding & LLM models. It is already 50+ module in it and you can use it just editing config YAML file. (You can select pre-made one) - Yes human-in-the-loop super super super important. That is why we want to build a data generation tool for human-in-the-loop, designed only for RAG evaluation & optmization. We are gathering waitlist for that here.

And here is the answer to your question.

AutoRAG builts on the LlamaIndex and have few Langchain modules in it. It optimizes & build RAG pipeline for you. I want to know the reason or use-case to use AutoRAG & library at the same time! If we think it is great as feature, we will add it.