Add TonyMcMapface as a tonemapping mode (or allow using custom LUTs for tonemapping)

[Calinou]

• Related to Implement tonemapping curves #4376,
  Implement further color correction post processing effects #4731,
  Make Filmic tonemapper desaturate bright colors (like ACES) #6096
  and Implement AgX as a tonemapper #7545.

Describe the project you are working on

The Godot editor 🙂

Describe the problem or limitation you are having in your project

Godot's current tonemappers have various limitations, with none of them "ticking all the boxes" so to speak:

• Linear: Highlights are blown out, bright lights are too saturated.
• Reinhardt: Scene loses saturation and contrast, bright lights are too saturated.
• Filmic: A decent neutral tonemapper that slightly darkens the scene, but bright lights are too saturated.
• ACES: Bright lights desaturate nicely, but bright blues steer towards purple hues, which is not always desired. Additionally, ACES darkens scenes a lot. This often requires lighting adjustments if you've designed the scene without ACES in mind. This often happens in Godot as ACES is not the default tonemapping method. Godot's built-in adjustments are also too limited to brighten dark parts of the scene without affecting bright parts of the scene, and without reducing contrast too much.

Note that the above descriptions assume you're using a whitepoint of 6.0, which is common in scenes with bright highlights.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

TonyMcMapface describes itself as:

Tony is a display transform intended for real-time applications such as games. It is intentionally boring, does not increase contrast or saturation, and stays close to the input stimulus where compression isn't necessary.

It ships as a LUT along with a bit of shader code for sampling. There's no tweakables; if you need a different look, do it before and/or after the display transform.

Please note that the shader output is linear, so if "gamma space" sRGB is desired, the sRGB OETF must be used afterwards.

Bevy, which has recently adopted TonyMcMapface as its default tonemapper, has a slideshow example here comparing TonyMcMapface to Reinhard tonemapping:

https://bevyengine.org/news/bevy-0-11/#new-default-tonemapping-method

Reinhard	TonyMcMapface

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

As TonyMcMapface was generated using expensive offline shader derivation, the tonemapper comes as a DDS texture which is a 3D look-up-table. There are also HDR/EXR formats that could be created if needed (see h3r2tic/tony-mc-mapface#4 (comment)). This means the LUT needs to be embedded in the Godot binary, both in the editor and export templates.

This tonemapper could be used by default in new projects using the approach proposed in #4834. Existing projects wouldn't be affected.

The main challenge of this approach is that the DDS file is 432 KB (306 KB when zipped), which is non-negligible as it'll increase Godot's binary size by this much.
However, given tonemapping is not available when using the Compatibility rendering method, we could strip this data from web exports which only support the Compatibility rendering method. This would prevent increasing web exports' base size (and therefore loading times).

This binary size concern means that it may be better to allow using a custom LUT as a tonemapper, separately from the color correction step. As a result, we won't need to bundle the texture in all projects (even those that don't use TonyMcMapface tonemapping) and users will be able to use their own tonemappers by providing a 3D LUT. The downside of this approach is that we won't be able to default to TonyMcMapface in new projects, and discoverability will suffer to an extent.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as this can't be implemented by a script or shader code without modifying the engine. See below.

Is there a reason why this should be core and not an add-on in the asset library?

You can see my existing findings with trying to use the existing TonyMcMapface 3D LUT as a color correction effect in h3r2tic/tony-mc-mapface#4.

Unfortuntaely, it doesn't work as expected as color correction in Godot is performed after tonemapping. This is generally desired behavior, but not when you're trying to use color correction as a tonemapper 🙂

If there was a way to change the existing Tonemapping → Color correction → Adjustments order, then this could be implemented by an add-on with no script or shader code needed. While this approach is the easiest to implement, it would also prevent using color correction at the same time as a custom tonemapper. This can be limiting when trying to achieve a certain scene mood, as TonyMcMapface itself does not provide any adjustments by design.

[allenwp]

If HDR LUT is implemented for tone mapping and HDR color grading, this is an example of how the workflow could be used.

To further optimize things, linear to gamma space conversion could possibly happen in this LUT, but I believe that would make it problematic for other effects like the glow effect.

Edit: My mistake. The TonyMcMapface tonemapper is actually best described as an SDR color correction LUT applied on a basic Reinhard tonemapping. So my initial thoughts I wrote above about an HDR tonemapping LUT do not apply to this proposal.

[jirisvd]

The TonyMcMapface tonemapper is actually best described as an SDR color correction LUT applied on a basic Reinhard tonemapping.

It seems that if Tony McMapface isn't going to be built-in, the only way to support it would be to allow users to write custom tonemapping shaders, just like they can write e.g. custom sky shaders. Then they could copy and paste the appropriate Tony McMapface sampling shader code (translated to .gdshader) and provide a copy of the 3d LUT texture as a uniform themselves.

I guess it would be good to decide which solution (hard coding support for Tony McMapface into the engine or implementing custom tonemapping shader support) is preferred for Godot, because both options have their pros and cons.

Just to summarize what was already mentioned:

Adding Tony McMapface directly into Godot:

• [+] Very easy for users to enable, good discoverability, could be used as a default if blessed by maintainers
• [-] LUT texture size bloat

Adding support for custom tonemapping shaders:

• [+] Makes it possible to use Tony McMapface as well as any other LUT for tonemapping, very flexibile
• [-] Bad discoverability, users have to copy and paste shader code and download the LUT themselves if they want to use Tony McMapface
• [-] More complex to implement

[jirisvd]

Having thought about this for a bit, directly adding Tony McMapface to the engine seems to me like the better option. The problem outlined in the proposal revolves around the limitations of existing tonemappers and a lack of a balanced neutral option that would look good out of the box without the need for extensive tweaking.

Supporting Tony McMapface natively would be a targeted solution to this problem that's very easy to use and well worth the ~300 KB of binary size considering how common the problem of "I can't get the colors in my scene to look right" is.

[jirisvd]

Opened a PR: godotengine/godot#97095