Operators who wish to provide 4K content must have a minimum of 2 compressed versions of it. One in 4K and the other in 1K or full high definition. Users having 4K resolution displays in addition to sufficient network bandwidth will receive the larger 4K file. Users having displays with poorer resolutions or bandwidth restrictions will receive the HD or FHD file. The system chooses appropriate segments from the 4K or 1K files at the core datacenter to satisfy adaptive bitrate control requirements. For viewers with 4K displays, this switching can cause annoying changes to the image quality.

Besides, 4K files can be quite large and duplicating program content will only result in excess remote cache space and storage being used up. Streaming 4K files also take up a considerable amount of bandwidth. This directly points at the fact that there is a limit on how many programs providers can offer in 4K at one time.

This picture is completely altered by AI-enabled super resolution. It makes it possible for operators to deliver a HD/FHD or 4K experience to viewers using a small program file of only 1K. 

Scaling with CNN models— 

It has been found with research that CNNs can reduce file size significantly by scaling each frame individually. This results in little to no loss of image quality on the end user’s device. Usually, two CNN models are created by content providers for all 4K contents that they wish to deliver. Each of these will be used as an input data for the CNN models. While one model is used to downscale each frame from UHD to FHD, the other to upscale the FHD frames into UHD. This rigorous training gives rise to a CNN model that has gained expertise in upscaling, and can restore textures, edge sharpness as well as minute details absent in the input 1K content.

Input specific training is imparted to the two models simultaneously. This means that the model assigned with the task of downscaling is now able to remove details in such a manner that the model meant for upscaling will be able to restore it correctly. 

Let’s understand with an example. If you trace a picture and ask a skilled artist to make a real art out of the traced outlines, will they be able to do it? Well, without necessary instructions, the artist might fail. But by telling them about how you want the lines to be—jagged or smooth, by letting them know that the shape in the water is that of a dolphin, or giving out other relevant details, you will effectively “train” the artist, who will now be able to accurately fill in all the details that are not in the outline.

Machine-learning algorithms train in a similar fashion and scale visual content with great finesse. They give higher-definition outputs than what is achievable using compression methods that are strictly mathematical. Besides, ML scaling shows compatibility with standard video codecs.

The training technique used for the CNN models we discussed above is long and complex, just like any deep-learning network training, and is usually undertaken at a data center. But once trained, the models tasked with downscaling and upscaling jobs can show incredible performance speed. 

The market is now home to powerful chip-level AI systems that can scale content on a real-time basis. In fact, the upscaling model can be small enough to operate even in a smart streaming device.