Is there something wrong with Local LLM ability to read file?

Posted by Umr_at_Tawil@reddit | LocalLLaMA | View on Reddit | 19 comments

So I've been feeding the sub file of anime episodes into Claude/ChatGPT/Deepseek and ask them to find all full name of Japanese character in it and put it into a python array so I can run a script to flip the name back to the original Japanese order (personally I hate hearing one thing and read another thing in sub), and they have been very reliable with this task.

I thought that this would be one thing that LocalLLM could easily do, so I downloaded LMStudio, and so far, every model I have tried, Qwen3.5/3.6-9B/27B, Gemma4 of similar size...etc... all failed to find all the fulll names in subtitle file that I gave them, not a single success so far. I have tried increasing context size and everything.

Does this mean that whatever LocalLLM use to read file is really behind Cloud LLM right now?

[-]

ttkciar@reddit

On one hand, yes, 27B local models are lagging behind multi-trillion-parameter cloud models. That should not be too surprising.

On the other hand, this should be within the capabilities of a local model with particularly good long-context competence, like K2-V2-Instruct. One of my uses for it is to give it very long IRC chat logs and ask it to identify and describe every participant in the chat, which doesn't seem too different from your task.

You might want to give K2-V2-Instruct a try. Be warned that it is very slow, though, since it is a 72B dense model.

[-]

abhuva79@reddit

Why the hell are you trying to do a completely deterministic task with an LLM?
I mean honestly, searching a string in a text-body and changing its order can be considered one of the beginner tasks in programming.

[-]

Umr_at_Tawil@reddit (OP)

It seem like you misread or misunderstand something.

tell me, given a subtitle file, how do you know all the full name of character in it? especially for Japanese anime character whose name can be very uncommon and doesn't exist in any name dictionary?

and not like you can just look up the name of the characters on an anime database site either, some minor character don't get listed there at all.

[-]

Particular-Award118@reddit

It's literally find and replace. The word doesn't need to be in a dictionary for you to search the string and find it. 400 lines is absolutely nothing for a script to parse. Since you asked so nicely: import sys

# Define name mappings: "Given Family" -> "Family Given"
# Add entries as needed
NAME_MAP = {
"Tanjiro Kamado": "Kamado Tanjiro",
"Nezuko Kamado": "Kamado Nezuko",
# ...
}

def flip_names(text, name_map):
for western, eastern in name_map.items():
text = text.replace(western, eastern)
return text

if __name__ == "__main__":
if len(sys.argv) < 2:
print(f"Usage: {sys.argv[0]} [output_file]")
sys.exit(1)

infile = sys.argv[1]
outfile = sys.argv[2] if len(sys.argv) > 2 else None

with open(infile, "r", encoding="utf-8") as f:
text = f.read()

result = flip_names(text, NAME_MAP)

if outfile:
with open(outfile, "w", encoding="utf-8") as f:
f.write(result)
print(f"Done. Written to {outfile}")
else:
print(result)

[-]

Umr_at_Tawil@reddit (OP)

this script assume you already know all the character name to put it into a NAME_MAP there, now tell me how you get that NAME_MAP in the first place.

[-]

Particular-Award118@reddit

Oh I now understand what you're trying to do. I'm this case it would probably be better to get the model just to extract the names, essentially making the name map, then you do the rest deterministically with the script.

[-]

Umr_at_Tawil@reddit (OP)

yeah that is what I'm trying to do with Local models and they have been failing while Cloud models been able to do it very reliably.

[-]

121POINT5@reddit

If the name isn’t in a name dictionary it’s probably not in a dictionary. So subtract words that are in a dictionary to get 90%+ of the way there.

[-]

Umr_at_Tawil@reddit (OP)

that wouldn't work too since many anime invent words for in-universe stuff all the time that doesn't exist in dictionary, and not to mention some name do exist in name dictionary.

basically sound like it would create too much noise compared to just dragging the sub file into Claude/ChatGPT/Deepseek and get reliable result like 99% of the time in my experience so far.

[-]

abhuva79@reddit

yeah, seems the wording you used was a bit misleading to me. So you dont know the names before hand. Thats a different thing then. The flipping is rather easy to do.
If you tried with full files and it works on bigger (paid) models, but not on local ones - try lower context amount first (split the file in several parts). If its not working at all, time to check your prompts - you may need to be way more specific, have few-shot examples in there and so on.
Did you already tried that?

[-]

Mayion@reddit

And why does that bother you? People can do whatever they want. That's not what the topic is about.

[-]

Awwtifishal@reddit

Part of the problem is having all the content in the context. If you configure enough context for the whole file, that's one problem solved. Another issue is that the LLM may not be able to find all instances reliably. One solution for this is having the input be sectioned in parts and you query each part instead of the whole thing at once. Which could also solve the first problem if you don't have enough context.

[-]

UnlikelyPotato@reddit

It means you need to setup a tool or use a friendly easy to use local environment. Not every local llm should have access to local files.

[-]

Umr_at_Tawil@reddit (OP)

I dragged the file into the chat interface like with a cloud LLM, it read the file and give me some names but always misses some.

[-]

Particular-Award118@reddit

Perhaps your harness restricts how many lines it can parse?

[-]

Umr_at_Tawil@reddit (OP)

I have tried pasting the whole content of the sub file into the chat and still doesn't work :/

[-]