How We Serve Millions of Requests on a Single VM
Posted by gametorch@reddit | programming | View on Reddit | 10 comments
Posted by gametorch@reddit | programming | View on Reddit | 10 comments
gabeech@reddit
Without knowing what types of requests are being serviced, what resources are allocated to the VM, what your response budgets are, and on and on. This doesn’t really explain anything.
Why does it take 50-500 (?!!!!) ms to start rendering a sprite sheet? How long is your average response time to that render request? How many of those millions of requests are just analytics pings?
gametorch@reddit (OP)
The average time to write the response request is <5 ms.
If you read the article, that 500 ms is the time between the user clicks a button and sees a response. That is completely dependent on their internet connection beyond the <5 ms consumed to authenticate them and push their job to the queue.
gabeech@reddit
Millions of requests with … 234 sign ups? I did read the article it says that 500ms is … the time it takes to start processing.
The worst case you’ll see from the internet is around 300ms round trip. Outside of really crappy cell signal or some sort of hardware in the way.
Responding ‘I got it, it’s in a queue to eventually get you what you asked for’ is not your actual response time the user cares about. Your response time is ‘here is what you asked for’ is.
This sounds like someone solving problems they don’t have, then writing a vague article tooting their own horn.
gametorch@reddit (OP)
don't care about you buddy. keep being miserable while I actually try to build something people are willing to pay for and share solid engineering advice for free.
gabeech@reddit
Then… share advice? There is no real advice in there. you’d need actual scale to experience the really interesting pain points when you get to 50,100,1000+ rps.
I’m not miserable at all.
I’d focus more on building a responsive product and less writing ‘I/O is slow, slap a queue on it’ articles.
gametorch@reddit (OP)
It is absolutely good advice and you know it lmao. have a good one man 🫡
tagattack@reddit
There are 2.5 million seconds in a month.
There are at least a dozen cores in the average mid-sized, compute optimized instance.
This isn't even a daily average of 1qps?
gametorch@reddit (OP)
I trying to puff up my feathers and pump my chest, just being honest about our requests per month here. I hope the advice is still useful. We're sitting at <3% cpu usage 99.999% of the time with this setup.
CanvasFanatic@reddit
No disrespect intended, but you could serve “millions of requests per month” from a raspberry pi running a Wordpress blog connected to home Internet.
gametorch@reddit (OP)
absolutely