It shouldn't matter as you manually select the BT source, but maybe unpair one set of buds and see, or before that try again and avoid any interaction with the remote, maybe that triggers a switch.
Odd I don't use mine often but, not had such an issue.
REGARDS, OBSYDIAN - Car & Tech Evangelist / 2 decades of Samsung Ownership
DAILY: ZFold4, Watch6 Classic, Buds Pro2, 20,000 Power bank, Smart Tags
OFFICE: Book2 Pro i7 [Laptop], M70A 32” [Monitor], Samsung Level [Speaker]
GAMING: M70A 43” [Monitor], Book2 Pro i5, Dell Inspiron G5 [Laptops]
FAMILY: ZFlip4, ZFlip5, Watch6 Classic, Tab S8 Ultra, Tab S7 Plus, Buds Pro2
SPARE: Book2 Pro i5 5G [Laptop], ZFlip3
AV: QN95A [TV], Q900A / SWA900S [Soundbar, Rears],
KITCHEN: MC32K7055 [Microwave], S65DG54M3B1 [Fridge] Jet 75 (Vacuum)
NON-SAMSUNG: iPad Air 2, Air tags, iPad Mini 5, iPad Pro 12.9 GEN2, Phone 12
