Group Buys for Shared Compute or Model Hosting? Is this a thing?

Posted by JustinPooDough@reddit | LocalLLaMA | View on Reddit | 4 comments

I've been using GLM 5.1 a lot lately, and I love this model. However I don't love sending all my requests to China. I'm not freaking out about it, but it's not ideal. I don't want to send my data to any provider ideally.

With the cost and availability of Cloud compute, it looks to me like someone could theoretically orchestrate a "Group Buy" to rent something like a cluster of 8xH100s - maybe 16x. Unless Gemini has failed me, this would be enough to host GLM 5.1 at FP8.

My questions are:

Is anyone doing this - or has anyone tried to do this?
If you wanted to bring costs down to say 50 bucks a month per user, how many users would you need?
Would the hardware support this at a reasonable t/s?

Genuinely curious. I would be interested in such a deal personally. I would imagine you would want to auto-ban open-claw users or people clearly abusing the API - or at least segregate non-coding use cases to a separate group and separate hardware... thoughts?

[-]

HVACcontrolsGuru@reddit

I honestly use Modal and just scale across the GPU size I need. B200 for $8/hr I can run 15-20 Gemma 4 31B agents at about 1k tokens/s Just can’t recall if that is per an agent or total throughout. Using teams of them to build parts of my fine tuning corpus for SFT

Sufficient_Prune3897@reddit

How is this different than a provider? Except here you will have to front load the money yourself

Heavy-Focus-1964@reddit

I just came up with a crazy scheme that might just work.

You get 10 friends. One of you learns how to bake bread. One of you learns how to shoe a horse. One of you learns how to grow crops.

Once you have each of the tasks necessary to sustain life, boom — never have to pay for anything again. Unlimited money glitch.

SomeOrdinaryKangaroo@reddit

Me and my 3 friends are talking about buying one h100 to share since neither of us can afford it ourselves. Still figuring out details and how we are gonna share it. I absolutely think its worth it.