What is the best way to host and store videos?
Posted by The-amazing-man@reddit | learnprogramming | View on Reddit | 13 comments
I'm currently working on an application for online courses to an online school, I mainly care about integrity, security, and the experience of the student while watching the course. What is the best way to deal with the videos?
I have read about Cloud Flare Stream and it provides the exact service I want but it's very expensive, I don't have that enterprise budget scope. I also read about Bunny*net, it has way more better prices but the delivery (when the video is streamed to the student) fees still so expensive.
I though about using Youtube with unlisted videos links with tunnels to secure the link a little bit, but it will drop the performance noticeably.
Is there any better ways to do this? I would appreciate an advice.
NongDan43@reddit
loadvid.com - very stable
teraflop@reddit
Have you considered just hosting WEBM files on a plain old HTTP server?
The-amazing-man@reddit (OP)
I don't know how that works. I would really appreciate an explaination.
teraflop@reddit
Google "how to convert video to WEBM" and "how to set up a web server", and then feel free to follow up if you have more specific questions.
Or if you prefer, just do the first part, and upload your WEBM files to S3 and use S3 as your web server.
HolyPommeDeTerre@reddit
Playing video is a storage and network heavy. It can also be CPU intensive if you start doing conversion of formats.
You are working on a hobby project (for now) IMO. Streaming video can be very very hard when you try to achieve best performances all around the world. For a lot of reason, scaling video solutions is very expensive.
Now, what is streaming. In CS, everything is a stream. Unix represent everything as a stream. A stream is a flow of data. I want to read a file that is 1MB. I open a stream, read the first 1KB, then the next 1KB etc... Until I reach the end of the stream. You consume 1MB of RAM. You read the files at your pace.
Network exchanges are streams. We send packets over the network and the reader is reading them one by one. hTTP, being a network protocol, relies on streams (1.1 in a basic version, 2.0 with control flow, irrc).
This help the server serve files that are GBs in size without ever reading the whole file. You push 1KB (arbitrary) chunks in the stream. The reader will get them.
What is it for a user to be streaming a video over the internet (YouTube or cloud flare). They open a stream to a server. The server will send chunks to the user. The user's browser will get the encoder/decoders for this kind of file then it'll process the frames.
HTML can handle natively videos. HTTP can handle native stream. Your server can handle native streams. You're not gonna require much scaling right now, build a POC with simple technology first.
As mentioned by another commenter, you can server webm (web media) files. They are videos optimized for sharing (I don't really know the diff). So if your server starts streaming a webm video (or a MP4 one), the user's browser will be able to play it.
Your last problem is storing the video files. For now, a big enough storage that the server has access to will be the easiest. But it brings other questions (latency, backup...)
The-amazing-man@reddit (OP)
Thanks! That is so informing.
But I have a question though, will this work for a high scale?
There will be some stress on the server from day 1, I am not sure that hosting it on my web server is a good idea.
I have 200+ GB of videos and 1000+ users. I'm not sure about concurrent users but I would say in hundreds.
But again, I'm not even sure if this scale is considered big.
HolyPommeDeTerre@reddit
Scaling is way harder. This starts to be a design system question and not really video streaming anymore.
200GB video and 1000+ users is pretty ok. Most servers can be sized easily to handle streaming that amount. You may need 2 servers and some load balancing at some point to help you. Think about server state (stateless) in the first place.
I would put some monitoring in the load of the server. Size it a bit bigger to see, launch in prod. Look, analyze and adjust.
But it depends if your users are all in the same place, close to the server or all far away in distance: some one moon, some in china, some in Europe.... In the first case, it's easy, people should have a good bandwidth with very little latency. In the second case, latency increases and bandwidth drops drastically. That's why you start to replicate your data to different world regions. And that's why it starts to be very very expensive as video is heavy on network and storage.
backfire10z@reddit
What’s wrong with YouTube? How would YouTube “drop performance” (I’m not entirely sure what that means in this context)?
The-amazing-man@reddit (OP)
Well, I want to secure the URL of the videos, so I need to make some kind of tunnel in between Youtube and my backend in order to encrypt the link to prevent intruders from intercepting the request and revealing the URL as much as possible.
That's why the performance will drop at some point.
backfire10z@reddit
Ahh, I misunderstood your post. You’re selling these videos. YouTube would not be good (and likely against ToS).
Conscious-Pace7669@reddit
dmazzoni@reddit
It's impossible without knowing your scale. How many users? How many videos? How large are the videos? Where in the world are your users located?
In general my advice would be to start simple - just host the video on your web server and use a HTML5 video player. If your site gets so large and busy that you need something better, upgrade when you need it.
The-amazing-man@reddit (OP)
Well the scale is medium to high from day 1, I am not sure that hosting it on my web server is a good idea.
I have 200+ GB of videos and 1000+ users. I'm not sure about concurrent users but I would say in hundreds.
But again, I'm not sure if this scale is considered big.