Build a local AI server with backup

Posted by carcaliguy@reddit | LocalLLaMA | View on Reddit | 2 comments

A company that I support wants a local backup SQL server. I have everything in the cloud for 5 years no issues.

I told them it's a waste but if you let me build a local AI server I will feed it company data and we can use it to develop documents, proposals, review docs, etc. we also can make it a backup server for the cloud systems.

I have a budget of 10-25k. I am new to this so I would like something modular. I'm thinking AMD 9 whatever, 96 more of ram and 2 GeForce RTX 4090, nvme drives.

The SQL is no issue for me the cloud is only 32gb of ram and SSD. We will probably never use the local backup.

I want something I can expand upon, add more GPU etc. Any ideas, pointers, books, recommendations. I have unlimited electricity available also at this location. I should also build a mining rig or wonder if I can use this also when not being used or scale as needed.

This is exciting and seems like a really fun project. I'm open to opinions, recommend parts, etc.

Also what model would you run local for re-creating proposals. I want to feed it all the prior docs and then have it spit out something good enough for a manager to review/edit and send.