It may confuse them for a bit (a really good way to try and combat aggression), but usually doesn’t cause any harm unless you kick up an anoxic zone. I’ve rearranged tanks countless times with no fish showing any sign of distress (other than wondering why there is a hand in the tank and why it looks different than before).