[D] What's the one thing you wish you'd known before putting an LLM app in production?
Posted by Bbamf10@reddit | LocalLLaMA | View on Reddit | 4 comments
We're about to launch our first AI-powered feature (been in beta for a few weeks) and I have that feeling like I'm missing something important.
Everyone talks about prompt engineering and model selection, but what about Cost monitoring? Handling rate limits?
What breaks first when you go from 10 users to 10,000?
Would love to hear lessons learned from people who've been through this.
AffectSouthern9894@reddit
Being the only competent AI engineer on my team, sometimes I feel like Saruman going mad trying to manage orc logistics and tribal politics. Hoping that one of them doesn’t get the fire too close to the spiky bombs. I think I need to email the devops engineer and setup a 10am meeting tomorrow..
No_Afternoon_4260@reddit
Exactly x)
Are you still trying to explain what you do? I feel getting trust from your team isn't easy today, that shit feels magic and pointing to limits are hard or time consuming
MelodicRecognition7@reddit
prepare a plan B in case Cloudflare goes down again
AffectSouthern9894@reddit
Being the only competent AI engineer on my team, sometimes I feel like Saruman going mad trying to manage orc logistics and tribal politics. Hoping that one of them doesn’t get the fire too close to the spiky bombs. I think I need to email the devops engineer and setup a 10am meeting tomorrow..