Quiet room makes no difference, neither does the accent. If I hold the Bixby button it understands exactly what I'm saying 100% of the time. OK Google also works 100% of the time. Trying to set up Hi Bixby always gets the response that Bixby can't hear me. I think they have a long way to go.
I have the same issue. However I did go into a Samsung service centre & the salesman tried to set the wake up bixby with his voice & it was recognising his voice (it did still take 5 or 6 goes). When I got home it still would not work with my voice.
I will go back soon to try to set up my voice on one of there phones.