Ollama now official supports llama 3.2 vision

[-]

Arkonias@reddit

Msty is a llama.cpp wrapper, so doubt 3.2 vision support will land anywhere that uses that in a while. Ollama supports it due to their own custom go stuff.

[-]

Msty's actually Ollama based under the hood and you can upgrade the Ollama instance manually without needing to wait for Msrty to update. I've been using Llama 3.2 vision with Msty for the past two weeks or so with the preview release of Ollama.

[-]

AnticitizenPrime@reddit

Msty is Ollama based, works already.

[-]

Sudden-Lingonberry-8@reddit

now they need to support python 3.12

[-]

mpasila@reddit

When llama.cpp support? (so it will support all platforms and not just one)

[-]

JakoDel@reddit

llama.cpp has to be one of the worst popular projects out there. they are ignoring too many things. there is a commit that doubles the performance for mi100 gpus just sitting there because one of the developers of the project wanted to do it in another way that never came.

[-]

Few_Painter_5588@reddit

They mentioned that they want new devs to join the project before they implement this.

[-]

robberviet@reddit

Unlikely sadly.

[-]

Lossu@reddit

Sadly llama.cpp is allergic to vision models

[-]

Red_Redditor_Reddit@reddit

I want to know too.

[-]

Melbar666@reddit

Open WebUI / ollama 0.4.0 / llama3.2-vision:11b

If I post a follow up picture, it does not see the picture, it only pretends and hallucinates the content. Is this a known bug?

[-]

JasonP27@reddit

Hmm... how much VRAM do you need for this? I have a 10GB 3080 and 64GB RAM

[-]

Intelligent_Jello344@reddit

Llama 3.2 Vision 11B requires least 8GB of VRAM, and the 90B model requires at least 64 GB of VRAM.

[-]

JasonP27@reddit

Sweet, guess I know what I'm installing later today

[-]

earslap@reddit

more VRAM?

[-]

JasonP27@reddit

Lol I wish. I bought this PC last year before I knew anything about running local AI models. I would have gone for the 3090 instead. Now I'm stuck with this for a while.

[-]