Which music app are you using? I use the Samsung music app on my s21 and found if you go into the music app tap on the 3 dots at the top right and go to settings you find an option to enable or disable lockscreen controls disabling that along with the music widget found in the lockscreen settings, that worked for me on current software versions. With that said I believe also that if you have always on display enabled you may have to go into that and uncheck show audio information in there too to disable it though I haven't had an issue with that enabled so far as it is only supposed to affect the AOD and simply show track info on the AOD not the lockscreen but some people claimed it was causing the problem for them.
These are the settings I was referring to in my last post one is the settings in Samsung music app and the other is the lockscreen settings. As you can see in lockscreen settings there is a widgets entry just tap that and then look for music control or Spotify and disable it on lockscreen.