I have the same issue, new phone 20 days old. It's most visible on telephoto due the focal length, but if You look closely then You will find that it's from time to time affecting the main camera as well. I tried to get help through the telephone support. They advise me to stop a camera app and clear it's data, do a cache partition wipe. Both didn't helped. Then They advise me to do a hard reset, but this also failed to fix the problem. I will be trying to return the phone to the seller. Don't want to use a resealed phone after the service shop interference.
This is a hardware problem. I have send my phone to the Samsung service under warranty. First time they updated the software, which didn’t resolve problem. Second time they replaced the main board and cameras so basically the whole phones internals, now it's steady as it should be from the beginning. I also noticed that the autofocus is faster and more precise that it was before.
Soooo, the problem came back. I have a theory that if You are exposing the phone for vibration, like keeping it in the pocket while running or in the handlebar bag while biking it's simply loses calibration. Did anyone experienced this problem but not exposing their phone for any kind of strong vibration?