Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffers in pool may not always point at the video compositors output #39

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Developer-Ecosystem-Engineering

We received a report of black frames in an applications video content once it was encoded with NextLevelSessionExporter.

Investigation revealed a similar issue as #38.

It appears NextLevelSessionExporter is pulling buffers from the pool that was created, and sending those buffers back to the API. The assumption a buffer pulled from that pool will contain the output of the Metal video compositor is not always correct.

This change will pass the pixelBuffer directly, or use the toBuffer if a renderHandler exists. It resolves black frames in video content in our testing, @GrandPoohBear you may want to give it a try.

…utput

We received a report of black frames in an applications video content once it was encoded with NextLevel.

Investigation revealed a similar issue as NextLevel#38.

It appears NextLevelSessionExporter is pulling buffers from the pool that was created, and sending those buffers back to the API. The  assumption a buffer pulled from that pool will contain the output of the Metal video compositor is not always correct.

This change will pass the pixelBuffer directly, or use the toBuffer if a renderHandler exists.
@piemonte
Copy link
Contributor

piemonte commented Jun 3, 2022

@Developer-Ecosystem-Engineering thanks for the PR! could you explain how these code paths are different? when an optional statement is called, nothing happens, just as your code does. did you observe different results?

@Developer-Ecosystem-Engineering
Copy link
Author

Hi @piemonte!

@Developer-Ecosystem-Engineering thanks for the PR! could you explain how these code paths are different? when an optional statement is called, nothing happens, just as your code does. did you observe different results?

The difference is which buffer we are passing to pixelBufferAdaptor.append.

If the renderHandler is nil, the existing pixelBuffer should be passed to pixelBufferAdaptor.append.
If the renderHandler is not nil, then the renderHandler needs to copy the contents of the pixelBuffer to the toBuffer.

Without this change, there is no default renderHandler, so the toBuffer is assumed to automatically contain the contents of the pixelBuffer, but that is not always the case. This usually works fine, except in some error cases where it doesn't!

The observable difference is sometimes video files re-encoded by NLSE had black frames prior to the change and now those same video files re-encode no longer black. Perhaps @GrandPoohBear could share one of their sample videos, we are unable to provide the reference video in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants