1Soundwave3

250x lossless compression of K/V cache memory: 95% in RAM usage reduction. 205gb -> 10gb with No Performance Downgrade

Posted by 1Soundwave3@reddit | LocalLLaMA | View on Reddit | 4 comments