Any kind of LLM for OCR?

Posted by noneabove1182@reddit | LocalLLaMA | View on Reddit | 7 comments

Having a lot of trouble searching for this info (related, where do people find their LLM news besides here?) Trying to figure out if I can implement an OCR with a LLM to improve the output, but finding 0 info.. any hints from anyone?

Reply to Post

7 Comments

[-]

soSpursy7@reddit

I would recommend using something like tesseract OCR model first to read in images into text and then use that text as you normally would with a LLM. LangChain looks like it has support for reading in images and pdfs: [https://python.langchain.com/en/latest/modules/indexes/document\_loaders/examples/image.html](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/image.html) [https://python.langchain.com/en/latest/ecosystem/unstructured.html](https://python.langchain.com/en/latest/ecosystem/unstructured.html)

[-]

noneabove1182@reddit (OP)

yeah I'm currently using tesseract but running into a few issues, one being that there's graphics in the screenshot that I want it to ignore, and the other that there's large spaces between some numbers that it's ignoring and concatenating the numbers instead of recognizing them as distinct entities, so thought maybe some kind of AI approach would work better i'll take a look at langchain's image recognition thanks !

[-]

Electrical-Young-360@reddit

Hey u/noneabove1182 1 year has passed :) have you found a good way to do that besides tesseract? currently struggling on the same point

[-]

noneabove1182@reddit (OP)

> 1 year has passed 💀 There is a better OCR that I found yes, Surya: https://github.com/VikParuchuri/surya

[-]

Electrical-Young-360@reddit

ah yeah ive heard a lot of good things about Surya... But sadly this doesnt work that good for my business documents...

[-]

dxvdxkxtn@reddit

I have a project I'm working on right now, I'm using tesseract OCR to convert the image to text. I then pass it through the openai API with a prompt that explicitly asks it to look at certain details of what I need to get the context and fix OCR transcription mistakes. So far it's working like a hot damn.

[-]

vedangs@reddit

Looking for something like this do you have something that you can share