Virtual Advertising for live sports events
What this lesson covers
This lesson extends the live streaming design to a specific problem: showing contextual ads inside the video of a live sports event, so different viewers see different ads in the same part of the frame. The first step is segmenting users, for example by age or interests, using a recommendation system treated here as a black box that assigns each user to a group so ads can be ranked and the top one shown. The lesson then focuses on rendering. Rendering ads on the client's GPU is a poor fit because many phones have weak GPUs, it drains battery, and a separate ad stream has to be sent alongside the match. Green screening the sidelines makes replacement easy but removes ads for people in the stadium, which is a business trade-off. To keep stadium ads and still replace them for viewers, the system uses the fact that broadcast cameras are fixed, so for each camera the region covered by the advertising boards is known. The challenge is players moving in front of those boards. Colour keying, which replaces only the known colours of the current board ad while keeping jersey and skin colours, is fast but breaks down when colours overlap and needs careful tracking of which ad is showing. AI-based object detection that keeps players and the ball while replacing everything else is more general but more compute-intensive. Targeted ads let broadcasters charge advertisers a premium.
- Users are segmented by a recommendation system so the highest-ranked ad can be chosen for each group
- Client-side GPU rendering is a poor fit for phones and needs a second video stream
- Green screens make replacement easy but remove ads for spectators in the stadium
- Fixed camera positions make the advertising board region in each frame predictable
- Colour keying is fast but brittle, while AI object detection is more general but costs more compute