Contex length

Posted by Lost-Health-8675@reddit | LocalLLaMA | View on Reddit | 3 comments

I have been laying in bed last night and though a bit about contex length and how can I - let's say take it to next level.

Looking at memory palace - it's ok, but it wasn't what I was looking for.

And then it hit me.

What I tried first is looking for something similar online - there was nothing similar - nothing that would pull data out of contex file that is over 100k tokens big in milliseconds, without loosing contex, without mistakes, without extra fuss

Then I fired up my maschine and talked to qwen3 6 27b

then with gemma4 31b

then again qwen... and that lasted for hours

Guys I think I'm on to something.

Now is time to stop my all ongoing work and focus on this. I hope in fee weeks I will have something for community to use (going for open source)

Lets see where will this take us :)