Upload in chunks to a resumable session so a dropped connection resends only the missing parts. Split the video into segments, transcode them in parallel into a ladder of resolutions and bitrates, and publish as soon as the first rendition is ready. Players fetch short segments from a CDN and switch bitrate as bandwidth changes.
hardPattern: CachingFrom Devana's Google question set
Problem
Design the path from a creator pressing upload to a viewer on another continent pressing play.
Cover ingesting a very large file over an unreliable connection, transcoding it into the formats every device needs, storage, and global delivery.
POST /uploads {size, filename} -> {upload_id, chunk_size}
PUT /uploads/{upload_id}/chunks/{n} <bytes, checksum> -> {ok}
GET /uploads/{upload_id} -> {missing_chunks}
POST /uploads/{upload_id}/complete -> {video_id}
GET /videos/{video_id}/manifest -> renditions and their segment URLsThis is one of Devana's own questions, written in the style of Google's interviews. It is not a question leaked from Google, and Devana is not affiliated with Google.
Requirements
- Uploads of up to 100 GB, over connections that drop.
- A video should be watchable within minutes of upload, not hours.
- Viewers on connections from 200 kbps to 1 Gbps.
- Playback starts in under 2 seconds at the 95th percentile.
- Worth asking first: is live streaming in scope? Assume not: uploaded videos only.
Scenarios
Scenario 1
A creator's connection drops 60 GB into a 100 GB upload. Expected: The upload resumes from where it stopped; the 60 GB already sent is kept.
The client asks which chunks the server has and sends only the rest. Each chunk counts as received only once its checksum matches, so a chunk cut off mid-transfer is simply sent again.
Scenario 2
A viewer on a 300 kbps mobile connection presses play. Expected: Playback starts quickly at a low rendition and steps up if the connection improves.
The player picks a rendition from the manifest by its measured bandwidth, fetches a few seconds of video at a time from a nearby CDN edge, and can switch rendition at every segment boundary.
Hints
Open one at a time, and try the problem again after each.
Hint 1
A 100 GB upload will fail partway at least once. What lets it continue without starting again?
Hint 2
Transcoding a long video in one piece takes hours. What can you split it into, and run at the same time?
Hint 3
Viewers range from 200 kbps to 1 Gbps. What does the player need so that both get smooth playback?
Hint 4
Which bytes do most views touch first, and where should those bytes live?
A worked answer
Requirements and estimates
Assume 10^6 uploads a day averaging 1 GB: 10^15 bytes, a petabyte a day of source video, before the renditions add more. Playback dominates: assume 10^9 views a day of 5 minutes at an average of 2 Mbps. That is 75 MB a view, 7.5 × 10^16 bytes a day, on the order of 7 terabits a second on average.
No set of origin servers serves that. Delivery has to come from caches near viewers, and the origin should only see misses. Every later choice follows from that one number.
Resumable upload
The client opens an upload session and sends the file in fixed-size chunks; with 64 MB chunks, 100 GB is about 1,600 chunks. Each chunk carries a checksum, and the server records it as received only when the checksum matches. After a drop, the client asks which chunks are missing and sends just those, several at a time.
The bytes go straight to object storage through signed URLs, so no application server carries 100 GB. The upload service only keeps the session: which chunks have arrived. When every chunk is in, it starts processing.
Transcoding in parallel
Split the source at keyframes into segments a few seconds long, and put one job per segment and rendition on a queue. A one-hour video in 4-second segments is 900 segments; with six renditions from 240p to 4K, that is 5,400 jobs that can run at the same time, so the wall-clock time is minutes rather than hours.
Do the lowest renditions first and publish as soon as one rendition is complete. The manifest, the list of renditions and their segments in a standard format such as HLS or DASH, gains the higher renditions as they finish.
Storage
Keep the source in object storage on a cheaper tier, so the video can be transcoded again when a new format arrives. Renditions and their segments live in object storage too. A relational table holds each video's metadata and which renditions are ready, which is what the playback API reads to build the manifest.
Global delivery
Viewers fetch segments from CDN edges near them. The player measures its throughput and picks a rendition for each segment, starting conservatively so the first frame arrives fast; the first seconds of each rendition are the hottest bytes, so cache those widely to meet the 2-second start.
Place videos by predicted demand: a creator's subscribers by region, early views, what is trending locally. Predicted videos are pushed to edges ahead of time; the long tail stays in regional caches and the origin and is fetched on a miss.
The whole design at a glance
How to say it in the interview
Split the answer into three pipelines, upload, processing and playback, and name the constraint each one answers: resumable chunks for the dropped connections, parallel segment transcoding for minutes rather than hours, and a CDN with adaptive bitrate for the 2-second start across 200 kbps to 1 Gbps. Put a number on the egress early; it is the reason the CDN carries the design.
The core in code
The upload session's bookkeeping, which is what lets a 100 GB upload survive a dropped connection: a chunk counts only when its length and checksum match, the client asks what is missing after a reconnect, and the upload completes only when every chunk has arrived.
Python
import hashlib
class UploadSession:
"""A resumable upload: the file in fixed-size chunks, each one
recorded only after its length and checksum match."""
def __init__(self, size, chunk_size):
self.size = size
self.chunk_size = chunk_size
self.count = -(-size // chunk_size) # ceiling division
self.received = {} # chunk number -> checksum
def expected_length(self, n):
if n == self.count - 1:
return self.size - n * self.chunk_size # the last chunk may be short
return self.chunk_size
def put(self, n, data, checksum):
if not 0 <= n < self.count or len(data) != self.expected_length(n):
return False
if hashlib.sha256(data).hexdigest() != checksum:
return False # damaged in transit: send it again
self.received[n] = checksum # a repeated chunk is harmless
return True
def missing(self):
return [n for n in range(self.count) if n not in self.received]
def complete(self):
return not self.missing()JavaScript (Node.js)
const crypto = require("node:crypto");
class UploadSession {
// A resumable upload: the file in fixed-size chunks, each one
// recorded only after its length and checksum match.
constructor(size, chunkSize) {
this.size = size;
this.chunkSize = chunkSize;
this.count = Math.ceil(size / chunkSize);
this.received = new Map(); // chunk number -> checksum
}
expectedLength(n) {
// The last chunk may be short.
return n === this.count - 1 ? this.size - n * this.chunkSize : this.chunkSize;
}
put(n, data, checksum) {
if (!Number.isInteger(n) || n < 0 || n >= this.count) return false;
if (data.length !== this.expectedLength(n)) return false;
const actual = crypto.createHash("sha256").update(data).digest("hex");
if (actual !== checksum) return false; // damaged in transit: send it again
this.received.set(n, checksum); // a repeated chunk is harmless
return true;
}
missing() {
const out = [];
for (let n = 0; n < this.count; n++) if (!this.received.has(n)) out.push(n);
return out;
}
complete() {
return this.missing().length === 0;
}
}Both versions are tested against the same cases, including every example on this page.
- A chunk is recorded only after the length and checksum check, so a chunk cut off by a dropped connection never counts, and the client sends it again.
- Putting the same chunk twice is harmless, which matters because a client that lost the reply cannot know whether its last chunk arrived.
- In production the session lives in a database, the bytes go to object storage, and
completetriggers the processing pipeline.
Follow-up questions
How do you make a 100 GB upload resumable after a dropped connection?
Split the file into fixed-size chunks under an upload session. The server records each chunk once its checksum matches; after a drop, the client asks which chunks are missing and sends only those, several at a time. Chunks go straight to object storage through signed URLs, so no application server carries the bytes.
Do you transcode the whole video before publishing, or can you publish sooner?
Publish sooner. Transcode segments in parallel, lowest renditions first, and mark the video watchable once one complete rendition exists; the higher ones join the manifest as they finish. Viewers in the first minutes may only get the lower qualities.
How do you decide which videos to push to which CDN edges?
By predicted demand: the creator's subscribers by region, early view velocity and what is trending locally. Predicted videos are pushed to edges in those regions ahead of time, and the long tail is fetched from regional caches or the origin on a miss.
A video unexpectedly goes viral in one country. What happens?
Edges there miss at first, so regional caches and request coalescing at the edge make sure each segment is fetched once rather than by every viewer at the same moment. As views climb, the demand signal places the video on more edges in that country, lowest renditions first.
Common mistakes
- Uploading the file in one request, so any drop restarts all 100 GB.
- Transcoding the whole video as one job and publishing only when every rendition is done.
- One bitrate for everyone, which stalls slow connections and wastes fast ones.
- Serving video from origin servers, with no plan for the egress the numbers show.
- Sending upload bytes through application servers instead of straight to storage.
Practice it in a mock interview
Reading a solution and explaining one under questioning are different skills. The button below starts this problem as a voice mock interview with Josh, Devana's AI interviewer, with a whiteboard he can read.
Related problems
- Design a News Feed: Push to most followers, pull from the few huge accounts, and rank the merged candidates.
- Design a URL Shortener at Scale: Key generation, a cache-first redirect path, and analytics kept off the hot path.
- Caching: the pattern guide, with more practice questions.