OK so not 100% sure This will solve the issue you guys are having with HDR and game mode but I am having the same issue with watching Blu-Ray disks on my xbox. The screen would automatically dim after about a minute of resetting the brightness to a higher level. The fix that I found is to adjust the Eco solution setting which to save energy automatically dims the screen when ambient light in the Room is detected below a certain level.
Go to: menu/settings/general/Eco solution and you should be able to adjust those settings accordingly.
This has worked for me but I haven't tested while actually playing a game.
... View more