Object Eraser AI Video

[xmrdevi1ishx]

Hello,

I am interested in learning more about the possibility of using AI to remove objects from videos, similar to how it can be done with photos. I recall seeing a demonstration of this capability in a Snapdragon 8 Gen 3 trailer, but I have been unable to locate any further information on the subject.

I would appreciate any insights or resources you may have on this topic.

Thank you for your assistance.

[Star_girl]

Following.

[Babishko]

This is an interesting query @xmrdevi1ishx. AI video boost and object removal was being rumoured to feature on the S24 series as far back as December 2023. The 8Gen3 chipset will easily be capable of such actions, so I see no reason why Video boost and Object removal won't come to the S24 series at some point over the next year. It is unlikely we will have to wait for the S25 series for such video abilities.

[Reylob]

Object eraser AI on a video is a very different proposition. There some video editing programs but they are more PC than phone orientated.

Object eraser on an image is tricky still and the final results depends very much on the particular case, some of them very far from perfect. In a video, that has to be done seamless for each frame. I do not know if the phone will have or be able to sustain enough computing power for this task. This would be very compute intense, heat generation and dissipation could be an issue and battery life will suffer terribly.

[Ms_D]

It's something that the new Google handset does (Pixel 8), but Samsung, at the moment, doesn't, as far as I know, do this for videos.

[Reylob]

I really hope you are right, but I do not share your optimism. Other than the commercial material from Qualcomm, I have not seen the Video Object eraser in action. I did this post about object removal on a picture.

https://eu.community.samsung.com/t5/galaxy-s24-series/object-eraser-how-to/m-p/9264572

The final result is not bad but very far from perfect. Using the AI feature was even less than desirable. The object was replaced not erased.

On a video, we have fixed object and moving objects. Imagine that I would like to remove from a video a person walking, that person must be tracked in all frames, erased, and generated accordingly. Depending on length of the video and fps, it could take a fair bit of time. Will the phone battery be enough? Removing small static objects from a scene, like a glass might be easier to do, but it would be quite resource intensive still.

Other AI features like live translation is completely unusable after two firmware updates. Like I said at the start, I really hope that you are right.

[Reylob]

Hi @Ms_D 

Do you have any link to a video object eraser video on YT?

Video boost in done on Google's data centres not on the phone.

I have tried to find a video showing the video object eraser and cannot find it.

[Ms_D]

This is one of many I saw on yt. https://youtube.com/watch?v=s89YemDxxhI&si=tcynR7vVmsUMkJRs

[Reylob]

That is object remover on a picture not a on video.

[Babishko]

I definitely agree with you @Reylob when you explain what a resource intensive feature a video erasure tool could be. There would have to be limitations on the amount of time that such a feature could be used for within the video itself at first. To track and erase an object for say 5 full minutes in a 4K 60 video would be neigh on impossible as you rightly say. If this feature had a time limit of only ten seconds per video then that may only be of limited use anyway?. And it may not look that impressive when done. Then again it MAY look very impressive?. The fact though that we are actively talking about the introduction of such features to our devices just shows what an exciting time it is, with AI enhancing our user experiences and offering almost boundless possibilities. If not now this instant, then in the near future.