[BUG] Bad legibility of small text when using HDR

[jbaiter]

Describe the bug
When using HDR, small text with thin letters is hard to read compared to SDR. This was already reported shortly after the HDR release to me by a Reddit user, but I didn't really notice it myself, since I was mostly playing action games at the time. But playing Yakuza now in HDR, it's really noticeable and detracts from the experience.

Screenshots

• Image Comparison 1 (pay attention to the HUD text)
• Image Comparison 2 (dialog box and esp. speaker name)

SDR:

HDR:

HDR screenshots are tricky since the brightness gets lost (which is why the HDR screenshots are darker), but the text legibility issue really is that bad on my Steam Deck OLED display.

To Reproduce

• Play a game with small text, or simply use the Playstation menus
• Start a session with HDR, take a screenshot
• Exit session, start another with SDR, take a screenshot
• Compare screenshots, SDR text should be much more readable.

I will try with the official Remote Play app tonight to see if this is an issue specific to Chiaki or if it's a general problem with the way the PS5 encodes content in HDR mode.

[weakvar]

+1 I noticed that too. For now I decided to play without HDR.

[jbaiter]

So I just tried to reproduce this with the official Windows app on my HDR TV, but unfortunately I cannot get HDR to work with the Remote Play app, all I'm getting when toggling the HDR box is a black screen :-/

What I did manage to do, however, was testing it with my Pixel 7a Android phone, and the text is very much readable on it, even though the display is much much smaller. The text displays none of the fuzziness I'm seeing on the Deck. (Can't take a screenshot unfortunately, since HDR screenshots don't seem to work on my phone).

So I'm currently inclined to suspect our HDR rendering code as the part where the problem is introduced.

Another observation: Small white text gets a readability boost by setting the rendering preset to "Fast", but it still looks significantly worse than without HDR.

[jbaiter]

I'm beginning to suspect the downscaling algorithm used, I'm streaming at 1080p, and when I render to an external 1080p display, the text is perfectly legible and there's no discernible difference to the non-HDR picture.

And here's where it get's interesting: When I force 1080p as the resolution for both external and internal display (i.e. the image is downscaled by SteamOS/gamescope and not libplacebo), the legibility issue disappears.

However, you'd think that simply setting the resolution to 720p should then solve this issue (since placebo doesn't downscale then either), but unfortunately that does not seem to solve the issue, since the PS5's downscaler (that does the scaling to 720p for us) exhibits the same issue!

So the workaround for now:

• Set streaming resolution to 1080p
• In the SteamOS properties of Chiaki, in the "Shortcut" section, hardcode the game resolution to 1920x1080
• Enable the "Set resolution for internal and external display" option

@weakvar Can you see if that helps in your case as well?

I'll play around with the various downscalers available in libplacebo tomorrow and see if any of them help alleviate the issue without OS-level workarounds.

Update: Using lanczos instead of the default hermite helps. While still not as good as non-HDR, legibility of small text is definitely improved, I'll do a comparison with no dedicated downscaler (i.e. "just let the GPU handle it", used with the "fast" preset) tomorrow night.

[jbaiter]

Here's a comparison between different downscaling methods and the SDR image:
https://imgsli.com/MjU4ODEz/0/1

My takeaways:

• Brightness definitely plays a role, the SDR image is much brighter, this might help with text clarity when downscaling
• hermite (default and high quality presets) has the worst rendering of small text
• none (fast preset) is a bit better, but still bad
• no big difference between lanczos and ewa_lanczos4sharpest, overall best placebo downscalers from the ones I tried
• the 1080p image downscaled by SteamOS looks by far the best, still worse than SDR, but definitely a huge improvement

SDR

HDR with Hermite Downsampling

HDR with Lanczos Downsampling

HDR with SteamOS Downsampling

[eworthing]

+1 Experiencing the same issues.

@jbaiter nice job with the testing. I couldn't quite put my finger on why I was having such a hard time seeing things in HDR on my OLED Deck and had switched back to SDR.

[xingfanxia]

Experiencing the same

[streetpea]

Possibly related haasn/libplacebo#281

[jbaiter]

It might be worth a shot to re-enable linearization in a Flatpak build and check if that helps:
haasn/libplacebo@030b59d

The mentioned artifacts can be avoided by simply not using a Lanczos variant for scaling (which we don't, High Quality preset uses Hermite, Fast uses no dedicated scaling algorithm).

---

Tried it out, better, but still a far cry from the legibility on SDR.

https://imgsli.com/Mjg0Nzcw

Made a mistake with the build, re-building...

---

No discernible difference, even with the linearization enabled :-/
Maybe I'm doing something wrong? Here's the patch to libplacebo:

From 82fa1af8488c8ffa75343b90091251300e0fc9c6 Mon Sep 17 00:00:00 2001
From: Johannes Baiter <[email protected]>
Date: Mon, 5 Aug 2024 17:22:34 +0200
Subject: [PATCH] Re-enable linear downscaling on HDR sources

---
 src/renderer.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/renderer.c b/src/renderer.c
index 802480d7..05764eed 100644
--- a/src/renderer.c
+++ b/src/renderer.c
@@ -1973,10 +1973,10 @@ static bool pass_scale_main(struct pass_state *pass)
     if (params->disable_linear_scaling || fbofmt->component_depth[0] < 16)
         use_sigmoid = use_linear = false;

-    // Avoid sigmoidization for HDR content because it clips to [0,1], and
-    // linearization because it causes very nasty ringing artefacts.
-    if (pl_color_space_is_hdr(&img->color))
-        use_sigmoid = use_linear = false;
+    // Avoid sigmoidization for HDR content because it clips to [0,1]
+    if (pl_color_space_is_hdr(&img->color)) {
+        use_sigmoid = false;
+    }

     if (!(use_linear || use_sigmoid) && img->color.transfer == PL_COLOR_TRC_LINEAR) {
         img->color.transfer = image->color.transfer;
--
2.45.2

[jan2000]

Look at the square symbol next to "Grab" in "Image comparison 1". In the HDR screenshot the top and right side of the square symbol seem less bright than the left and bottom, while in the SDR version all the sides of the square seem equally bright. Zoomed in, we can see that for both the left and bottom side a row of pixels lies within the white part of the symbol in the image that was resampled. But, for the top and right all the pixel lie on an edge, in between the dark and white parts of the symbol resulting in grey pixels. Now comparing the HDR version to the SDR, we see that the pixels that lie on the edge are all too dark.

The same thing can be observed in the screenshots of the word "hotel". (Ignoring the weird upsampling in the zoomed-in versions, that make the pixels have inconsistent sizes.) For the letter L the pixels (mostly) lie on the white part, while for the letter H the pixels lie on the edge. In the non-SteamOS HDR screenshots the H is too dark in comparison to the L.

This is a tell-tale of resampling done in non-linear color space.

[jbaiter]

@jan2000 in the libplacebo thread you mentioned that you had a patched version that re-enabled scaling in linear color space in libplacebo. Could you share your patch so I can try it with Chiaki?

[jan2000]

I patched a .dylib that was bundled with a mpv binary, so I do not think that would help you much. But the patch did basically the same as: if (0 && pl_color_space_is_hdr(&img->color)) use_sigmoid = use_linear = false;.