Build a local AI server with backup
Posted by carcaliguy@reddit | LocalLLaMA | View on Reddit | 2 comments
A company that I support wants a local backup SQL server. I have everything in the cloud for 5 years no issues.
I told them it's a waste but if you let me build a local AI server I will feed it company data and we can use it to develop documents, proposals, review docs, etc. we also can make it a backup server for the cloud systems.
I have a budget of 10-25k. I am new to this so I would like something modular. I'm thinking AMD 9 whatever, 96 more of ram and 2 GeForce RTX 4090, nvme drives.
The SQL is no issue for me the cloud is only 32gb of ram and SSD. We will probably never use the local backup.
I want something I can expand upon, add more GPU etc. Any ideas, pointers, books, recommendations. I have unlimited electricity available also at this location. I should also build a mining rig or wonder if I can use this also when not being used or scale as needed.
This is exciting and seems like a really fun project. I'm open to opinions, recommend parts, etc.
Also what model would you run local for re-creating proposals. I want to feed it all the prior docs and then have it spit out something good enough for a manager to review/edit and send.
Interviews2go@reddit
Do the backup server, it’s more valuable for disaster recovery. As far as the ai box goes, get a dgx spark or Mac Studio for the experiments and see if the company will budget 4k for one of those for your experiments with AI.
MelodicRecognition7@reddit
lol, that's a very rare occasion when management is smarter than IT