So I found the solution. I made a routine where I was turning on the flash camera for specific contacts but then later realized that once it turns on it never switches off back. Resulting in flash notifications for all the contacts. I checked if I can put a switch-off action in the routine but it is not allowed which is kind of weird. So it might be a bug or something. Deleted the routine for now and it fixed it.
