I disabled the voice activation and set it to x2 button press short cut, I've had no disconnection issues and since disabling voice trigger I've had minimal battery impact, still getting 2 to 3 days (although I charge every other day at most)
Edit: I use the watch 4 classic bluetooth 46mm and s21 fe 5g
(Just thought I'd add this as it seems relevant)
Hope you get to the bottom of it