Video Processing
Netflix onboards new content by breaking each video into chunks and encoding every chunk in every format and resolution.
A codec is a way to compress video, and lossy compression loses data to keep the file smaller. Formats differ by internet connection speed, and resolutions differ by device. The number of formats into the number of resolutions is the number of videos processed.
One computer takes time and has a chance of failing, so the original video is broken into chunks. One resolution, one format, one chunk is one task.
- Chunks were first three minutes each. A chunk that ends in the middle of a car scene causes a lag.
- Chunks are now based on scenes. A shot is four seconds, and shots are collated into a scene that is fetched together.
For a sparse movie, where viewers jump between points, Netflix sends only the data asked for. For a dense movie, watched continuously, it proactively fetches the future parts.
The video is stored in Amazon S3, a cheap store for static data.
ISP Synergy
Open Connect is a cache that Netflix places with internet service providers, so that a request for a movie is answered near the user.
An internet service provider (ISP) maps an address to an IP address, a computer somewhere on the internet. The Netflix servers are usually in the US. From India, a signal takes a lot of time to go there, and video is a lot of data.
Netflix extended caching to ISPs:
- A request from India for a Bollywood movie does not hit the Netflix US servers. The ISP asks the Open Connect box, which holds many movies.
- If the movie is there, it is returned quickly. Bandwidth and time are saved, for Netflix and for the ISP.
- The box is localized: different movies for India, Britain and the US.
Around 90% of Netflix traffic is taken care of by these boxes.
New content is copied in around 4 a.m., when the load on the boxes is minimum. Netflix processes a movie into chunks and populates the box with them.
Virtual Advertising for live sports events
Virtual advertising replaces parts of a live sports video with an ad chosen for the user who is watching.
A recommendation system groups each user, ranks the ads and takes the top one. The options for rendering:
- Render on the client's GPU. Cell phones have poor GPUs, and the ad stream has to be sent as a second video.
- Green screen the sidelines. People in the stadium then see no advertisements, and that revenue is lost.
- Use the fixed cameras. A football match has 20 cameras, each fixed at a position and angle, so the section of video that is the side stand is known and replaced with an ad. Players in front of the stand are the challenge.
- Key overlays. The colors of the stadium ad are replaced and the jersey color is kept. It is fast, and it fails on a jersey of the same color and on skin colors.
- AI. The players, the football and logos are identified as objects and kept, and everything else is replaced.