Running 8B models on new M5 iPad?

Posted by PhaseExtra1132@reddit | LocalLLaMA | View on Reddit | 11 comments

I’m getting the new iPad since mine died and wondering if anyone tested running the 8B models on the iPad yet.

Reply to Post

11 Comments

[-]

National_Meeting_749@reddit

"Memory Bandwidth 153GB/s" 😬😬😬 I'm Jealous of some apple hardware. Not the ipad hardware though. Could you? Probably. Quickly? No.

[-]

The only reason I’m not getting like an old m chip iPad is because it seemed very interesting they bumped the ram. Thinking it would be fun to have another device that can do quick local Ai stuff on those vacations trips with just the iPad is very intriguing to me.

[-]

National_Meeting_749@reddit

just tailscale into your local machine on whatever form factor you like best, no matter the ram

[-]

PhaseExtra1132@reddit (OP)

Then I have to rely on the WiFi. I have camping trips where I go fishing in the mountains and sometimes the data is non existent there. I can maybe get a starlink and set up a connection. I have my laptop but the iPad form factor is kinda unbeatable when hiking or fishing.

[-]

National_Meeting_749@reddit

Then go enjoy your camping or hiking brother. I think we've lost the plot if you're trying to talk to your LLM on a hike or a fishing trip. lol. I'm not gonna help you invade the most disconnected parts of our lives with tech. lmao. Have a good one man.

[-]

PhaseExtra1132@reddit (OP)

Thanks no problems bro. My camping is closer to glamping so it’s a mix of fishing and watching anime downloaded on my phone when I can’t sleep lol. One of these days I’ll reduce the tech bit by bit. Until then I’ll try to have ChatGPT tell me the best camping spots and what to bring. New to the whole thing lol

[-]

jarec707@reddit

Sure, using this on MyDeviceAI, m5 11” iPad Pro https://preview.redd.it/0n78g6se2wwf1.jpeg?width=2420&format=pjpg&auto=webp&s=f495b07be706b27d3d2e388cf25e544baff6c133

[-]

PhaseExtra1132@reddit (OP)

How is it? How many tokens/sec are you getting?

[-]

jarec707@reddit

I just posted this image in another thread, and it shows about 23 tokens per second with an MLX model. https://preview.redd.it/ig2hi0bl5wwf1.jpeg?width=2420&format=pjpg&auto=webp&s=a9af9046a9a3b4ba10d20ef0485f3664dec8f11c

[-]

PhaseExtra1132@reddit (OP)

I’ll just ask you questions on that thread

[-]

PhaseExtra1132@reddit (OP)

Source for image: https://www.reddit.com/r/ipad/comments/1o7bpfy/ipads_pro_m5_have_12gb_ram_and_16gb_ram/ And also I checked and apples site also: https://support.apple.com/en-us/125407