HEVC Support in SRT Provider and LLHLS Publisher

[getroot]

I'm happy to announce that SRT Provider and LLHLS Publisher finally support HEVC. This is currently available for testing with the master branch and will be included in the next release (0.15.12) once fully tested.

The test procedure is as follows.

1. Set as follows in OBS. (version 29.1.1 in my environment)

Select HEVC codec

Stream over SRT

2. Bypass Video Encodes in OME.

<Encodes>
  <Video>
    <Name>bypass_video</Name>
    <Bypass>true</Bypass>
  </Video>
  ...

3. Play with LLHLS.

[bchah]

🤩 Great work! I look forward to testing this very soon as I have a Haivision Makito X4 encoder in the lab. And of course will try the many macOS hardware encode options too 👍🏻

[saltarob]

Indeed, great work 👍🏻🌟 Congratulations. May I ask if it is then possible to push an HEVC HLS stream to YouTube as well? Thank you.

[Morpheus235]

So I gave this a Test and testing OBS with your settings worked flawlessly but streaming from one of our Haivision X1 HEVC Hardware Encoder in any setting resulted in no Video chunks being served by the llhls endpoint and timing out. Audio Chunks and Playlists were still being served fine. Are there specific requirements for the HEVC Transmuxing to llhls?

[bchah]

Anecdotally, the only reliable HEVC encode pipeline for OME I have found so far is NVENC via OBS. I tested with a Haivision Makito X4 the other day and had similar issues to what you describe with HEVC. Using the Apple HEVC encoders yields the same results. Unfortunately I'm not seeing any errors in the log file so haven't been able to gather enough evidence to submit an issue about this.

[getroot]

Since I don't have a Haivision device, I'll test with Apple HEVC. Most likely an HEVC compatibility issue.

[getroot]

@bchah

Is it correct to reproduce Apple HEVC like this? Or is it a different product?
To reproduce this problem, I tested with the latest master branch with the same settings as above, and it is working normally.
I'm not sure if the NAL Parser I recently patched fixed this as well. Could you please confirm that your problem is reproducible with the latest master branch?

[bchah]

Thanks @getroot for testing! Are you testing from a pre-2019 Mac Pro? Recent Macs use a dedicated “Media Engine” that presents as such: The very latest ones even have multiple media engines. I tried your latest build and the three HEVC options, in this order: - HEVC Hardware Encoder - HEVC T2 Hardware Encoder - HEVC Software Encoder The software encoder is the only one that would display an image on HLS playback. However it is very limited and not appropriate for use in OBS (I think it is very optimized for things like FaceTime) as it overloads easily and the encoding falls apart. However I can see some issues in the attached log for the other encoders - mainly the HEVC Hardware Encoder which represents the new “Media Engine” that all Macs with Apple Silicon use going forward. Please have a look through the attached log and let me know if it provides any clues. I can send you some test streams too if that helps - you know how to reach me :)

[getroot]

Come to think of it, even if I don't have those encoders, if you send me the files recorded with those encoders, I can analyze and solve the problem with ffmpeg codec copy. Please share the sample file (mp4).

[bchah]

Ahh, my email reply did not include any attachments! Here is the missing screenshot and log file, and I will get you those recordings ASAP 👍

engine (2).log

[bchah]

@getroot here are the files!

HEVC Hardware Encoder (T2 Chip on all 2017+ Intel Macs):
https://github.com/AirenSoft/OvenMediaEngine/assets/33604674/2573543a-fb2f-4bf0-801e-6cdd7f804c51

HEVC Hardware Encoder (Media Engine on Intel Mac Pro 2019):
https://github.com/AirenSoft/OvenMediaEngine/assets/33604674/d8948ea9-264f-48b1-84e1-ab7c14283a15

HEVC Hardware Encoder (On all Apple Silicon Macs):
https://github.com/AirenSoft/OvenMediaEngine/assets/33604674/7e65d288-9edd-4fa3-b320-228750b21a11

[Morpheus235]

@getroot I just tested this using a Haivision/Makito Device and it still seems to be stuck generating Video Chunks. Unfortunately the log does not provide any useful info on why it fails. Other than the failing http requests for Video Chunks everything else seems to Work. When only sending a Video Stream it also fails retrieving the Manifest.

Like @bchah already offered I could set up some teststreams from one of these Devices facing a Server of your choice.

[getroot]

Come to think of it, even if I don't have those encoders, if you send me the files recorded with those encoders, I can analyze and solve the problem with ffmpeg codec copy. Please share the sample file (mp4).

[Morpheus235]

Sure, attached is a link to google drive containing a sample ts and mp4 file directly recorded from the encoder using ffmpeg and -c copy.
google drive url

For reference the settings Used inside the Encoder:

[getroot]

@Morpheus235 @bchah
Thank you for supporting the sample files. I reproduced the problem with the sample files you provided and solved it.
Please test with the latest master branch.

[getroot]

@bchah I want to know if your problem is solved too!

[bchah]

Oops, sorry I forgot to reply here @getroot! Yes, the Apple HEVC streams are working with SRT + LLHLS. The decoding is a bit glitchy, with the usual 'green lines' SRT playback shows when the signal is having trouble keeping up - even when running everything on one local machine. But I don't have any evidence to suggest this is an OME-related problem, so I will continue testing.

In my application, I have already set it up to call the Engine API and depending on the incoming codec (H264 or H265), it will choose the WebRTC or LLHLS player accordingly. This is a streaming dream come true 😀

[Morpheus235]

@getroot
Can confirm this works with Main now.
Sending 8bit with Main10 Profile made Chrome crash my Driver and subsequently disable the H265 Decoder. After I reset my Flags in chrome it worked again.

Main10 4:2:0 10bit works.
Main10 4:2:2 8/10bit does not work (Probably Browser Support)
{code: 302, message: 'Some of the media could not be downloaded due to a network error.', reason: 'Error occurred when downloading.', error: 'manifestIncompatibleCodecsError'}

Also there seems to be an issue parsing SPS Messages from the Encoder Frames:
[2023-06-19 12:00:01.015] E [AW-LLDASH0:377811] LLDASH | cmaf_packetizer.cpp:277 | Could not parse SPS (SPS: -1--1, PPS: 114-130921) from LLDASH frame for [#default#bypass/6214da945c3c9e0019f19bcb]

[getroot]

Thank you for your confirmation.

The log is from the deprecated LLDASH module. DASH Publisher and HLS Publisher (Legacy) were deprecated a long time ago and are no longer being updated. It is recommended to remove HLS, DASH and LLDASH from Publishers