Cyber Monday is a big day for online businesses. Two days after Black Friday, online stores sell products with massive discounts.
The idea is to end the year profitably.
Two years ago on this very day, Shopify transferred 43 Petabytes of egress traffic across the infrastructure.
That's ~30TB of traffic per minute!
To understand the scale, imagine thousands of vendors with millions of customers viewing, purchasing, and reviewing products.
Everyone fetching descriptions, product images, reviews, prices, offers, delivery times, inventory, and recommendations in real time.
That's INSANE. And Amazon's scale is likely even bigger 🤯
"Amazon. com Recommendations: Item-to-Item Collaborative Filtering" explains how collaborative filtering helped them generate real-time personalized product recommendations.
1. Collaborative Filtering
- This simple recommendation technique notes user preferences to suggest items you might like (based on other users with similar preferences).
2. Item-to-Item Collaborative Filtering
-
A more efficient approach compared to simple collaborative filtering. This involves recommending items based on what other users have shown interest in, purchased, or rated highly.
-
So, while viewing "USB cable" items, the algorithm recommends items bought by other users who previously purchased a cable. The item similarity is determined using various metrics, such as time, location, ratings, etc….
3. Data Representation
- User-item interactions are stored as a matrix, where rows correspond to users, columns correspond to items, and each entry represents the user's interaction with the item (e.g., purchase, rating). This matrix serves as the foundation for identifying item similarities.
4. Similarity Calculation
-
Amazon employs techniques like cosine similarity to measure the similarity between items.
-
The basic idea involves taking each feature of the item, like price and rating, and mapping them into an n-dimensional space. You can then find the distance between two points through techniques like Euclidean Distance, etc…
-
The goal is to identify items frequently co-purchased or interacted with by users.
5. Scalability and Performance:
- Caching item similarities and horizontal scaling reduce latency. Practical Algorithms enable quick and efficient retrieval of item recommendations based on user interactions.
So that's what powers super-fast recommendations at Amazon!
At InterviewReady, we discuss research papers in the live classes (free for everyone with the system design course).
