Handling a large amount of files

Posted by Euchale@reddit | LocalLLaMA | View on Reddit | 9 comments

Hi all, what is currently the best way to handle a large amount of files? Its around 2000files at 500mb total of small txt and config files. I have a software here that is lacking a certain feature, and I already got it 90% there and would like AI to scan the files to see if I missed any mention of a specific thing, or anything coding related. I´ve already had some success with Gemma 4 (it suggested a python search first, and then I uploaded the summary), but I was wondering if there is an even better approach.

Reply to Post

9 Comments

[-]

arnauddsj@reddit

you can download riflet.com, it's 100% desktop based, you can concatenate all these texte files into one.

[-]

substandard-tech@reddit

For mechanical scans the AI makes the tool which does the scan.

[-]

false79@reddit

Set up RAG. This is what it's meant for.

[-]

bluelobsterai@reddit

If you have $200 a month to blow, install Claude Code. You can thank me later.

[-]

bluelobsterai@reddit

If you have $200 a month to blow, install Claude Code. You can thank me later.

[-]

bluelobsterai@reddit

If you have $200 a month to blow, install Claude Code. You can thank me later.

[-]

bluelobsterai@reddit

If you have $200 a month to blow, install Claude Code. You can thank me later.

[-]

bluelobsterai@reddit

You’re going to want to look at a coding harness like opencode or aider. Claude code works with local models. You’re going to need a quality local model to handle all of the tool calling. You have a bit of a learning curve, but I definitely would not use python for your project. If I had the budget I would use Claude code and I would use Scripps to update the files. Grep and search, make a plan. Then let the coding agent do what it needs to do. The coding agent can plan and execute this entire task for you. The smarter and more intelligent models like opus can do this with a low effort prompt. Using local models you’ll have to be more of a pilot and tell it what to do.

[-]

Euchale@reddit (OP)

Thanks, I will look into what you said. Honestly for what I am doing Python search was already good enough, just a question if it found all the hidden settings as well. Basically my machine supports different sizes of a thing and automatically detects them by itself and does a self-setup. I got it where it detects the thing, and where I can give it the target values for the self-setup. With the python searches/AI I found the part of the code that allows the software to send the "I have reached the target values, go to next step"-signal. Its shitty and poorly programmed science software, not a well programmed software. Like one config file with the exact same content exists 6 times spread among several folders. Absolute mess.