Up until recently I was in the No category as well. However I lost a young Clownfish to mysterious circumstances that point to possible toxins from a leather coral, that was stressed due to the fish hosing it. This is purely anecdotal I realize. I have looked at 2 studies now of Sarcophyton (leather corals) and they have stinging cells and produce not only proteins that affect corals but hemolytic compounds that affect red blood cells. With that said, I believe it to be rare and the risks mitigated by good husbandry.
For my situation, I believe the toadstool was irritated by hermit crabs, the clownfish and water parameters, causing its mucus to be more potent. And my little Clown bore the brunt of it.