Intelligent Chroma Subsampling

In this post we will talk about chroma subsampling and how it can be used to make your images load faster if done right. Chroma subsampling is a pretty old technique and has been used in numerous contexts like television colour systems like NTSC, etc. but in this post we are only going to be focussing on how it applies to images on the web.

Chroma subsampling is used in lossy formats like JPEG and WebP to reduce image sizes by discarding some of the colour information in the images. If done right, the user should not see any difference in the quality of the image after chroma subsampling is done.

Why does it work

The main reason why chroma subsampling works is because the human eye is more sensitive to brightness than colours. Our eyes have two kinds of photoreceptors - rods and cones. Rods are more receptive to brightness and cones are important in detecting colours and we have 120 million rod cells and 6 million cone cells. It’s no wonder we are more sensitive to changes in brightness than changes in colour.

To exploit this fact formats like the JPEG use the YCbCr colour scheme. Instead of representing colours using R, G and B, they convert it to Y (luminance / brightness), Cb (blue) and Cr (red). Each colour can be converted from the RGB colour space to the YCbCr colour space with a simple mathematical formula. The advantage of separating out the brightness and colours into separate channels is that they can be compressed in different ways. As we just saw, the human eye is more sensitive to variations in brightness than colour. Chroma subsampling works by keeping the brightness channel intact but by discarding some information in the chroma (colour) channel.

For example, here is an image and the individual Y, Cb and Cr components below it. As you can notice, the maximum amount of detail is in the image with the brightness channel. The Cb and Cr channel add more details to the overall image, but do not contribute as much as the Y channel. This is the fact that chroma subsampling exploits, leading to smaller images. You can try this out for your images using this script.

There are different modes of Chroma subsampling depending on how much information is discarded. The most commonly used setting is 4:2:0 which results is approximately 17% smaller image sizes. Images with the subsampling setting as 4:4:4 have essentially disabled chroma subsampling.

“Be right back.. I will subsample all my images to 4:2:0” Hey hang on! Even though 4:2:0 is a good starting point, subsampling fails in certain kinds of images. Chroma subsampling is great for photos, where there are usually no sharp edges and there is usually a smoother graduation in colour. However certain kinds of images like images which have text in them, screenshots, drawings, medical images, etc. do poorly when chroma subsampling is enabled. These kind of images are better off when encoded in a lossless format like PNGs or if they are in a lossy format like JPEG, make sure subsampling is disabled for them. Some formats like WebP do not have a configurable chroma subsampling level. For WebP, it is always 4:2:0, so there is no way to disable chroma subsampling when encoding in that format.

## Dexecure’s Intelligent Chroma Subsampling Dexecure uses machine learning algorithms to automatically figure if the image belongs to a category which is not well suited to chroma subsampling. In those cases, subsampling is automatically disabled for lossy formats, automatic format conversion to formats like WebP is disabled and a different set of optimisations are applied to these images. If you are optimising your images with Dexecure, this is something that you need not worry about at all. Images are automatically categorised behind the scenes and the right quality, size, subsampling levels, formats, etc is chosen. And the best part is all of this happens by default, no messy URLs!