Introduction to media streaming: Part 2

Bitrate vs Resolution

Bitrate and resolution correlate to the amount of data in a media file. It's easy to mix these two up, so let's clarify how they differ.

Resolution measures the detail in a single video frame by counting the pixels across its width and height. For instance, a 1920 by 1080 display is made up of 1,080 stacked rows, with each row stretching 1,920 pixels wide. More pixels means more detail, and then a bigger picture.

Bitrate - think of it as the amount of digital information your audio or video is allowed to use per second. Giving it more information to work with creates a crisper, more detailed media experience, usually measured in kbps or Mbps. This is where the actual compression trade-off lives.

The thing is, you can't judge video quality by resolution by itself. A 1080p file can look terrible - blurry, pixelated, and washed out - if it's encoded at a very low bitrate. On the flip side, a lower 720p resolution with a healthy bitrate can produce a much cleaner image than a poorly compressed 1080p video.

There's another piece of the puzzle too: the codec being used. Remember from Part 1 that codecs are responsible for compressing and decompressing video. Newer codecs are simply better at squeezing the same visual quality into fewer bits. For example, a video encoded with H.265 (HEVC) or AV1 can often look just as good as an older H.264 (AVC) stream while using significantly less bandwidth. In other words, two videos with exactly the same resolution and bitrate don't necessarily look identical - the codec plays a huge role in how efficiently those bits are used.

Now you must be thinking: why not just use the highest bitrate for everything? Well, because bitrate is directly tied to file size and bandwidth. A higher bitrate means more data has to be transmitted every second, which affects latency, and if the viewer's connection can't keep up, you get the thing nobody wants: buffering. Just like the issue with downloading, adaptive bitrate streaming (ABR) was invented as the direct solution to finding the perfect balance between great video quality and a viewer's internet speed.

Adaptive Bitrate Streaming (ABR)

Remember the manifest files we talked about in Part 1 - .m3u8 for HLS, .mpd for DASH? This is where they truly prove their value.

Preparing video for streaming involves generating multiple encodings instead of a single output file. This strategy is vital because it allows the stream to work seamlessly on the massive, ever-increasing variety of screens, devices, and browsers available today. The media is processed into various resolution, bitrate, and container combinations called renditions or variants, typically structured into a ladder such as:

• 240p @ 400 kbps
• 480p @ 1000 kbps
• 720p @ 2500 kbps
• 1080p @ 5000 kbps

All of these get chunked up, and the manifest lists every rendition as an option. The player is the one that decides, chunk by chunk, which rendition to pull next based on the device's current network speed, buffer health, and sometimes device screen size.

Have you ever noticed a video go blurry when your network drops, only to clear up a moment later? This isn't a bug in the player. To prevent the video from freezing up, the player immediately requests the next segment at a lower quality. As you watch, it measures your real-time download speed and automatically steps up to a clearer picture as soon as it's confident your connection can handle it.

Let's make that a little more concrete. Imagine you're watching a football match in 1080p over your home Wi-Fi. Everything looks sharp because the player has determined your connection can comfortably sustain that bitrate. Halfway through the match, someone in your house starts downloading a large game update. Suddenly, your available bandwidth drops. The player notices that the next few video segments are taking longer to download than before. Rather than waiting until playback stalls, it proactively requests the next segment at 720p, or even 480p if necessary. The picture becomes a little softer, but the video keeps playing without interruption. A few minutes later, the download finishes. Segment downloads become faster again, the player's confidence in the available bandwidth increases, and it gradually moves back up through higher quality renditions until you're watching in 1080p again - all without restarting the stream or you having to do anything manually.

The paragraph above overly simplifies what happens under the hood with ABR. What is the player actually measuring, and how does it decide? We need to go a level deeper.

How ABR algorithms actually decide

Broadly, these algorithms fall into a few categories.

Throughput-based - the simplest approach: you measure how fast recent segments downloaded, estimate the device's available bandwidth from that, and then pick the highest rendition whose bitrate comfortably fits under that estimate. The catch is that throughput is rather unstable. An increase in latency on a single segment doesn't necessarily mean your connection became poor - it might just be a momentary dip, or an issue with the CDN serving that segment. Switching too aggressively on every fluctuation and you get quality flapping, where the video visibly jumps between resolutions every few seconds, which is arguably worse for viewer experience than just staying at a slightly lower quality. Instead of panicking and reacting to a single slow download, most players use a moving average to smoothly track network speed. Once that average confirms your connection is stable, the player then steps back up to a clearer quality.

Buffer-based - instead of just staring at download speeds, this approach looks at the actual runway the player has left: the amount of segments currently sitting in the buffer. The logic is simple: if the buffer is healthy and growing, you can afford to take a risk and bump up the quality. But if the buffer starts shrinking toward empty, then you need to hit the brakes and drop to a lower resolution immediately, even if your download speed looks fine. A viewer can tolerate a blurry video for a few seconds, but they absolutely hate it when the video freezes completely to reload. Because a frozen screen is the ultimate mood killer, real-world video players are built to keep the video moving, even if it means dropping the quality for a bit.

Hybrid approaches - in practice, most modern players (ExoPlayer, AVFoundation, Shaka Player) combine both signals along with a few extras: device screen size, past switching history to avoid flapping, and sometimes prediction models trying to anticipate network drops before they happen. In practice, most modern players build on some combination of throughput and buffer health.

So what is the player actually trying to optimise? Every ABR algorithm is ultimately balancing three competing goals:

• Quality - give the viewer the best resolution their connection can handle
• Stability - don't switch renditions so often that it's distracting
• Rebuffer risk - never let the buffer run dry

The player can optimise for any one of these, but trade off against the other two. Tuning this balance well is most of what makes one player's ABR implementation "feel" better than another's, even when they're pulling from the same manifest.

Next, let's look at how to protect these files when transmitting them over the internet.

Protecting Premium Content: DRM

ABR solves the problem of keeping video playing smoothly. But it doesn't solve a completely different problem that studios and streaming platforms care about a lot: how to stop people from just copying and stealing the video the second it reaches their device. That's where Digital Rights Management (DRM) comes in.

At a high level, DRM encrypts the media so that even though the chunks are sitting on the device, or passing through the network, they're useless without a decryption key. That key is only handed over by a license server after checking that the request is legitimate - the user has a valid subscription, the device is authorized, that kind of thing.

The three major DRM systems you'll run into are:

• Widevine - Google, used across Android, Chrome, and most smart TVs
• FairPlay - Apple, used across iOS, Safari, and Apple devices
• PlayReady - Developed by Microsoft, used across Windows and some smart TVs/consoles

Because no single DRM system covers every platform, most large streaming services encrypt their content with multiple DRM systems at once, and the player picks whichever one matches the device it's running on. This is usually paired with Common Encryption (CENC), a standard that lets the same encrypted media file be decrypted by different DRM systems, so you're not re-encoding your entire catalog three times over.

It's worth noting that encoding/decoding a video is totally different from encrypting it with DRM. Encoding is just a way of compressing a video file so it's small enough to travel smoothly over the internet. Encryption, on the other hand, is all about locking the file down to protect it from being stolen.

What actually happens when a license gets requested

"Gate the key behind a license server" is easy to say. Here's what that exchange actually looks like step by step, using Widevine as the running example (FairPlay and PlayReady follow the same general shape, with system-specific details):

1. The player requests encrypted content. The video chunks arrive encrypted - the player can download them fine, but can't decode them yet.

2. The player extracts protection info. Encrypted content includes metadata (in Widevine's case, called PSSH data) that tells the player's Content Decryption Module (CDM) which DRM system is in use and which key is needed.

3. The CDM generates a license request. This is a device-specific, cryptographically signed request - it proves the request is in fact coming from a legitimate, unmodified player/device combination, not something spoofing it.

4. The request goes to the license server. This is a server the streaming platform controls, separate from the server hosting the video segments. It checks things like if the user's subscription is active, if the device is authorised, if the account has rights to the particular title, and if the region is licensed.

5. The license server responds with an encrypted license. If everything checks out, the server sends back a license containing the decryption key, but the key itself is encrypted so that only the specific CDM that made the request can unlock it.

6. The CDM decrypts content locally. From here, decryption happens inside a protected, isolated part of the device - ideally hardware backed, so the key never becomes accessible to the OS, the app, or anything that could extract it. Then the decoded frames get handed to the video player for playback.

This is way more complicated than just checking a password. With normal auth, the app only needs to ask: "Is this the right user?" But with video security, the question is: "Can we let this exact video play on this exact device without letting the user, or any unauthorised software on their machine, ever touch the actual video file or the secret key?"

A note on security levels

Not all decryption happens with the same level of protection. DRM systems support different tiers of security. Widevine, for example, ranges from L1 - which is hardware backed, keys and decryption happen in a trusted execution environment - down to L3, which is software only with no TEE backup. Premium content (think 4K, HDR) is often gated to only play on devices that support the higher security level, because software-only decryption is more exposed to extraction attacks. This is part of why a 4K stream might quietly downgrade to a lower resolution on an older or rooted device - because the device doesn't meet the security requirement for that content tier, so the platform serves a rendition it's comfortable protecting less strictly.

Tools like Shaka Packager (mentioned back in Part 1 for encoding) and FFmpeg also handle the encryption step, tagging the encrypted chunks with the right metadata so the player knows which license server to talk to and which key to ask for.

Wrapping Up

Quick recap:

• Resolution determines how many pixels are displayed, bitrate determines how much data is available each second, and the codec determines how efficiently those bits are used.
• ABR works by encoding multiple quality renditions and letting the player switch between them, usually driven by a combination of throughput and buffer level signals, balanced against quality, stability, and rebuffer risk.
• DRM protects content by encrypting it and gating decryption keys behind a license server via a signed request/response handshake, with Widevine, FairPlay, and PlayReady covering the major platforms, and security levels (like Widevine's L1/L3) determining what quality tiers a device is even allowed to play.

That covers the client-side fundamentals I set out to write about - from the basic "why streaming exists" question all the way down to what happens in a DRM handshake. If you're building or debugging a media player and want to go even further, the ExoPlayer and Shaka Player source/docs are genuinely excellent references - a lot of what's above is drawn from spending time reading and understanding them, and on OTTVerse.

This is Part 2 of a multi-part series on media streaming internals. Catch Part 1 here.