Just don't touch the thing with your bare hands, or burn/boil/etc it. It won't ooze toxin into the water or anything. Goggles are a good idea if you plan to remove it from the water and actively work with it, they can (though somewhat rarely do) squirt water when exposed to air. You don't want paly water in your eye.
You should always wear gloves when handling anything skeletal or rocky in your reef tank. There can be hidden sharp edges, and the bacteria potentially on those edges are far more threatening than palytoxin.
That said, brown palys are often invasive, and can be one of the really nasty-toxic ones. If you want to remove it, see if you can use a pair of bone cutters to cut off whatever it's attached to. If that won't work, take the whole pipe organ out of the water, wait for the paly to fully retract, and put liquid superglue over it. It'll be trapped and die. Works on aiptasia, too, and superglue won't hurt any other corals it happens to touch.
I don't think pipe organ has much of a sting to it, and palys are more likely to overgrow other corals than be overgrown. I wouldn't expect it to get crowded out.