Basically using automatic resolution with the PS5 connected to the HDMI2.1 port, when you turn on the TV the colors are all washed out. It's not a matter of calibration, the colors are simply incorrect, it's like watching a HDR signal in SDR to give you an idea.
In the video output properties of the PS5 it says the signal is RGB HDR (I thought it shouldn't be possible atm since that should be equivalent of YUV444 in terms of bandwidth?).
To fix the situation I had to switch to -2 in the 4K transfer rate option but that is a YUV420 HDR signal so with the worst possible color compression (although quite frankly in normal applications it's absolutely not noticeable).
I can also turn the setting back to automatic AFTER having switched to -2 and it goes back to RGB HDR but this time the colors are now correct (but I don't trust at this point that it's a RGB HDR signal as it claims...). The issue is that as soon as you turn off and on the TV you have to do the same thing again and again.
So basically you either use YUV420 or you're stucked doing the trick of Automatic-> -2 ->Automatic.
Since I was not happy with both solutions I simply put the PS5 on an other HDMI port and it works perfectly at YUV422 with no issues...it's not HDMI2.1 but it's enough for YUV422 HDR at 60Hz with game mode on. But if we want to play with 120fps and in the future with VRR this is going to be a headache.
There is definetly some bug somewhere,I don't know if the blame is on the PS5 or the TV or both. I hope it gest noticed and fixed, the console should be able to output RGB/YUV444 at 120Hz with VRR eventually but right now it's kinda barebone and buggy.