I'm actually kind of curious about this myself, so I did the math. Here's a breakdown of why it could make sense for someone to do this. It makes a bunch of completely baseless assumptions that probably don't all hold true for OP.

He probably spent ~$75K USD on this before tax ($4,700 MSRP × 16 = $75,200). Given the size of the investment, I'm just gonna go ahead and assume that someone making this kind of purchase has a business and will be able to write this off as a business expense (or more likely, write off its depreciation over the next few years). Assuming they expense any depreciation and then recuperate the residual value in a few years (let's assume for ~$3000 USD in 3 years), these could easily have a true/effective cost closer to $4,700 - $3,000 = $1,700, $1,700 × (1 - 0.30) = $1,190 per unit (this baselessly assumes that it would be offsetting income that would otherwise taxed at 30%) or closer to $1,190 × 16 = $19,040 total. So in this hypothetical the cluster would have a ~$19K effective/net cost over 3 years (or ~$6.35K per year).

Now let's see how much API usage it takes to hit ~$6.35K per year. For Kimi K2.6, it's $0.95/1M input and $4/1M output (edit: I made a mistake here, see my note at the end). Baselessly ssuming a ~3:1 input to output token ratio (this varies a lot by use case), that's about $6.85/4M tokens total, or about $1.71/1M on average (note however that there seem to be K2.5 providers that offer ~half this cost). At that price, they'd need to process ~3.7B tokens (at that same 3:1 ratio) per year to reach the same cost. If this cluster is running 365 days/year, that's ~10.15M tokens per day, or 423K tk/hr, or 7,050 tk/min, or 117 tk/sec. Considering this is for combined input and output, that feels very feasible to surpass with such a big node, but it also hinges on a 24/7/365 usage assumption which is likely unrealistic. There's one big caveat though... I didn't factor in electricity at all, and frankly I don't feel like it.

Anyway, with enough usage, the right tax/cost recuperation factors in place, and relatively affordable electricity, it's very possible for this to be comparable to cloud models in term of economics, at least for a business.

There are also other factors though. Off the top of my head, I can think of: - Privacy re: valuable business information - Privacy re: client or employee information (incl. possible contractual obligations/restrictions & legal requirements) - Cost stability/predictability - Different accounting treatment for investments vs operating expenses (varies greatly depending on where he's located) - Response latency - Independence / self-reliance - Stability / predictability (quality won't suddenly change out of the blue, and they won't be forced off of one soon-to-be-discontinued model at an inconvenient time to optimize all their work around some new model) - Better looking balance sheet with these assets on hand could feel more comfortable for investors or debtors - More end-to-end control could mean better optimizations around caching, which could help reduce costs

Conclusion: the margins are pretty tight, but with enough utilization/uptime, this could achieve significant non-monetary benefits at a reasonably low relative cost increase, or potentially even a cost reduction compared to using an API. But this requires HEAVY utilization and reasonable electrical costs.

Wait a minute... I forgot to adjust the API cost for the ability to write it off as business expenses at a similar rate as the depreciation. I don't feel like adjusting the math on that, but it definitely does make it harder to achieve a similar cost. Not impossible though.