He wants to run large models on the Mac Studios (and get the benefit of the Studio's combined memory capacity & high memory bandwidth) without the downside of the slow prefill in the current Mac Studios.

Other people have had this idea, too. The Macs run the show with this, but will send your context to the blackwell card for prefill/prompt processing. Then the blackwell will send the processed result back to the Macs, and the Macs will spit out inference tokens using their higher memory bandwidth.

It's a great idea in concept, but as shown in the video I linked above, even with 50Gbps networking between the Macs and the blackwell, the overhead of those transfers eats up the time saved by having the much more powerful blackwell do the prompt processing.

And to answer your other question, no, the model isn't restricted in size by the vram on the blackwell card.

I really suggest you watch the video I linked above. He talks in quite a bit of detail about this, including showing performance comparisons and explaining why it's a good idea in concept (and does work!) but it's let down by "only" having 50Gbps between the machines. Even at 100Gbps it is unlikely to be significantly faster than just using the Macs alone.