Recently, we added support for High Dynamic Range (HDR) output in our decoders. But this work was a substantial endeavour to implement owing to the wide variety of output pipelines we support. Especially compared to other vendors with much simpler output pipelines. This blog post will explain why.
We’re often asked “Surely it’s easy to add support for X SDI card or X video output pipeline (e.g PC HDMI output)?” – the answer is no and this blog post helps explain why. Every new output blows up the number of permutations we have to QA.
Providing a consistent user experience across a variety of output pipelines is a tricky problem. We are rolling out AI alongside our existing human testing, especially in areas of the product with large numbers of permutations like HDR. As explained in this blog, just for HLG alone, there were ten separate permutations that needed to be handled!
AI is able to verify tedious structures and implementation details.
Background
Unfortunately many aspects of HDR testing output could not be easily automated, such as making sure a television correctly showed the HDR icon. This meant a lot of this work required human testing to make sure behaviour was as expected.
In addition, this human testing required changing back and forth from SDR to HDR. The number of permutations continued to increase. There were ten different permutations in total that needed to be tested. In addition there were complexities such as the “Alternative Transfer SEI” message produced by certain content providers. Let’s look at these challenges in order of difficulty.
OBE SDI Card and ST 2022-6
First of all let’s look at the SDI-based pipelines that we control fully. These were the easiest to implement in these pipelines because we control the creation of the entire SDI frame in software. In the case of HDR we are able to write the Video Payload ID (VPID) to signal HDR and we are able to capture this using our own hardware and confirm the HDR flag was present.
There’s a reason we built these output modules ourselves – it’s so we could have the utmost control when outputting video.
Alternative Transfer SEI
Some content providers send what is known as an “Alternative Transfer SEI” message in their compressed bitstream. This means they could flag the video as SDR/709 in the video headers so that older decoders that predate HDR wouldn’t choke at seeing a video format “from the future” that they wouldn’t understand. Subsequently they would send an Alternative Transfer, that covers the actual video transfer characteristics.
We had to implement and test this capability.
Blackmagic Decklink Products
The Blackmagic Decklink series of cards proved to be challenging as we had to perform a big SDK upgrade (the first in over a decade) to a version that supported HDR. In addition we discovered (along the way) three separate subtypes of card that needed testing, with the HDMI one being particularly difficult as it required a physical verification on a TV to show HDR and SDR.
Blackmagic HDR
After a time consuming SDK upgrade, making sure we didn’t break any existing behaviour, we could see that the VPID was changing successfully to HDR. Easy we thought? Not quite.
Blackmagic non-HDR (NUC)
During this process, we found that cards which didn’t support HDR, such as the SDI Micro, as used in our small form-factor NUC decoders, produced no output. I.e the act of signalling HDR in a device that didn’t support HDR caused no output to be produced at all! So we had to detect these devices and not signal HDR in their output so that when they decoded HDR content, we could still output a picture.
Blackmagic HDMI
In a previous blog post, we wrote about how we now support HDMI output in our low-latency decoders, mainly to aid in-home monitoring applications. HDMI proved challenging as we had no existing HDMI measurement infrastructure – instead we had to output to various TVs and be sure that the HDR flag appeared, and disappeared when SDR content was sent.
HDMI output required physical verification
ST-2110
ST-2110 also proved time consuming to qualify, as different vendors managed SDR/HDR flagging in a different way, so we had to implement both. Some vendors use a 2110-40 Video Payload ID. We had never output payload ID before, as there was no reason to implement this arguably legacy data in the ST-2110 domain.
ST-2110-40 Video Payload ID (VPID)
We had to reimplement the VPID output discussed above in our 2110 output stack. Furthermore, we weren’t sure what third-party products were actually going to use, VPID or SDP. So we developed internal tooling to read the VPID in the 2110-40 domain and make sure the values were correct. This is always a tricky problem, as we may reimplement the same mistakes as when we create the VPID.
We used AI tooling to attempt to independently verify that we were correct in addition to standard human verification.
ST-2110 SDP
Other devices use the SDP text file format (as part of NMOS) to signal HDR. We also had to gather various third-party device SDPs to make sure we output the most interoperable SDP output – trying to be too clever for example and signalling BT.2100 instead of HLG + BT.2020 might break third-party receivers.
So after collecting a swathe of HDR SDP examples, we implemented the most common type.
Conclusion
We now have HDR output in our low-latency decoders. But it was a time-consuming (but not necessarily difficult) process. We have started to use AI to verify some elements of this process (with human supervision). But as a result we have the most flexible HDR decoding platform on the market.
