Dev on mac, seems promising but what will I miss?

Posted by No_Afternoon_4260@reddit | LocalLLaMA | View on Reddit | 26 comments

I've been a linux boy since 10 years, before I used to have macs. I've great memories of mac, the fact that's based on unix will be helping I'm sure. I've been really happy with cuda, linux+cuda makes me really feel I can do anything. I know on mac I'll be relying on mlx implementation and whatnot. From my understanding llm inference is acquired. Training is a no go (probably not a software support problem? Nvidia cards just have the proper accelerator?). From my really quick research I found a macos desktop version for comfyui so I guess they can run diffusion models. I found that transformers from huggingface should run fine, it relies on pytorch which I'm sure is correctly implemented on macos. Am I wrong? What would I be missing? Have you found any library that's badly supported or things that are just impossible because it rely on a x86 implementation?

Reply to Post

26 Comments

[-]

InspirationSrc@reddit

I'm also trying to choose between Mac and PC. And unifying memory looks great, everyone talking about how you can run big models. But it's going to run them far too slow to be useful and it requires special format. And even smaller models like qwen3 30b are in questionable territory. Plus you can't extend Mac, it has thunderbolt but I can't find working solution for attaching egpu. But... PC so big, ugly, harder to maintain, and less resell value.

[-]

No_Afternoon_4260@reddit (OP)

For now I can resell my cuda cards more expensive than when I bought them.. same thing for my ram.. for now

[-]

InspirationSrc@reddit

Oh, forgot to mention, everyone is saying that tuning is no go on macs.

[-]

bunny_go@reddit

Everything is a bit more fuxkery. No apt install. No cuda. Different docker. Not x86. You can make everything work but if you heavily really on these the switch will be painful. For webdev you wouldn't notice much difference though.

[-]

chisleu@reddit

>No apt install https://brew.sh/ >No cuda https://github.com/ml-explore/mlx >Different docker different backend... >Not x86 Fucking good. X86 is a an ancient monster. Long overdue for slaying. I get your point. Mac is different for sure. But Mac currently has first tier support for language model development. It's an entirely different platform from what is being used in commercial applications though. That is where I would draw the important distinction. If you are trying to accomplish something specific like tiny tuning a smaller, open weight, dense model then you could choose Mac as the platform to do that work. It would be cheaper than owning equivalent hardware. However, if your goals are to scale the process up, Mac's have a ceiling of like 512GB of RAM and the compute is SLOW compared to equivalent PC hardware. If you are just doing personal inference, Macs are a no brainer. I say this sitting in a room with 4 blackwells humming as I type this.

[-]

bunny_go@reddit

Sometimes people really struggle to see how silly their comments are. I say no apt install, you say there is brew. Is brew apt? Is brew as capable as apt? No. What's your funking point then? I say no cuda, you say mlx. Is mlx cuda? Is mlx *remotely* as capable as cuda? No. What's your funking point then? I say it's not x86. You say that's actually an advantage. As if building on a different platform than the server environment is running on is somehow "Fucking good". No, it's fucking terrible. And then, like a teenager who achieved nothing but has rich parents, you try to flex with, allegedly, "4 blackwells humming" in the room. First, no one cares, even if it's true. Second, if you worked in any professional environment for actual money, you wouldn't be sitting in a room with "4 blackwells" because then you'd need it's just dumb.

[-]

my_name_isnt_clever@reddit

Have you actually tried to use an Apple Silicon mac for development work, or did you do some basic research and conclude you hate it on paper?

[-]

bunny_go@reddit

I did. Some companies I worked with insisted that we all need to be cool kids and work on notebooks, namely Mac's, because then you can work from anywhere. As if I ever need to run docker in a coffee shop. I guess they didn't get the memo about remoting into dev environments but whatever. Whenever we had different dev / build / server architectures (and OSes) we ran into problem that we shoulnd't have. Yes, we solved all of them. Yes, it cost actual time and therefore money to solve all of them. Unless your whole stack is apple silicon, like iOS development, then I can't see how it's the better choice to mix architectures. If you use it as a terminal to access your actual dev environment, it's fine, but then you don't need an overspecced Mac to do that.

[-]

chisleu@reddit

you mad bro.

[-]

No_Afternoon_4260@reddit (OP)

>I say this sitting in a room with 4 blackwells humming as I type this. Feels more sexy ig lol

[-]

No_Afternoon_4260@reddit (OP)

What do you mean by different docker?

[-]

bunny_go@reddit

Could you try typing your question into your favourite local or hosted LLM and see what comes up?

[-]

StardockEngineer@reddit

Why choose? I use my Mac as a front end to my Linux box.

[-]

No_Afternoon_4260@reddit (OP)

Haha I completely agree! I'll be working in a company the guy convinced me that I'll be happy with my mac while the server is busy. I told him that if I have access to the server a chromebook could suffice but I'm afraid he didn't want to tell me that time on the server will be really sparse. To me it seems completely counter intuitive to dev on mac and run on cuda.

[-]

Accomplished_Ad9530@reddit

MLX works with CUDA (in addition to Metal), so you can deploy on Linux servers

[-]

No_Afternoon_4260@reddit (OP)

Care to elaborate?

[-]

Accomplished_Ad9530@reddit

MLX has three backends: CPU, Metal (GPU, Mac/iOS only), and CUDA (GPU, Linux only). So you write your MLX code once and you can run it with any of the backends. To quote MLX dev u/awnihannun “It goes in both directions. You can develop locally on you Mac and run at scale on Nvidia GPUs in the cloud. And then you and others use the same model you train in the cloud and use it on your Mac and potentially export / deploy it on any Apple platform that supports Metal.“ The MLX documentation (https://ml-explore.github.io/mlx/build/html/index.html) is pretty good, and the devs are really responsive and nice.

[-]

StardockEngineer@reddit

Right. I just dev on the Linux box remotely with VSCode or Cursor over SSH. Works great! And I love starting jobs over mux and disconnecting completely. And keeping stuff like browsers and UI off the Linux box’s GPU gives me just a bit more VRAM to use.

[-]

No_Afternoon_4260@reddit (OP)

>I just dev on the Linux box remotely with VSCode or Cursor over SSH. I know right? >I love starting jobs over mux and disconnecting completely. What's mux?

[-]

StardockEngineer@reddit

Like tmux

[-]

drwebb@reddit

If you haven't heard, Apple silicon perf is really good for LLM inference. Training LLMs is hard on any consumer hardware, but personally I'd pick more VRAM than compute. You can always wait longer

[-]

BidWestern1056@reddit

i dev abt 50/50 on mac/linux and build npcsh and npc studio for bringing together all the disparate ai tools like text gen/diffusion and set things up to automatically incorporate context where appropriate https://github.com/npc-worldwide/npcsh https://github.com/npc-worldwide/npc-studio these can run diffusion/text gen etc. you can even fine tune diff models from within the ui and then use them

[-]

BidWestern1056@reddit

the main issue w macos tho ive had lately is some transformers lock that messes things up depending on exact versions for torch/transformers/diffusers

[-]

Bastian00100@reddit

Money

[-]

Accomplished_Ad9530@reddit

MLX is great for fine tuning. Also, x86 vs arm isn’t really relevant (and Rosetta works nicely for those rare circumstances) for ML since it’s important to run on accelerators, i.e. CUDA/Metal/CoreML. Most major model architectures are supported on M series Macs, and, since MLX is well designed, new models are usually supported really quickly (faster than llama.cpp, for instance).

[-]

jackshec@reddit

cuda is great, but mac is also great for prototyping