I used tweezers. Pinch the stem as close to the rock as possible and gently peel them off the rock. It's time consuming but it worked well for me. This allows for the one at a time removal, sometimes getting more. Make sure to turn off flow & wear gloves. The Paly will release toxin. Run carbon after your done.
I've got some in my frag tank that I stupidly put in years ago. They are growing under the rack and will sting or release toxin or whatever that kills the frags within an inch or so. Every month I spend a few minutes with tweezers pulling them out. This keeps them "at bay" more or less but they always come back (kinda like kudzu). I had heard a Foxface would eat them so I put one in a couple of years ago but he didn't seem to show any interest in them.
I know some of the palys are very beautiful but they are invasive. I don't ever want another one in my tanks.
Something you may want to try is kalkwasser paste like we use for aptasia. Mix it to the consistancy of toothpase and very carefully (with all the pumps off) slowly lower a spoon of it on the palys and "wipe" it across them. Have a turkey baster at hand to blow it off any corals that some may come in contact with. Let it soak for a few minutes and turn the pumps back on. Watch to make sure none of it drifts on your corals. Good luck in removing them.
I second the kalk paste idea. I use it for any pest with a similar protocol to Harry's. I do use a plastic syringe instead of a spoon though. And I also generally siphon it off before turning powerheads back on (leave it on for 30m-1h). Turkey baster is key in case you get some on a coral.
Last night I removed some rock I could and used the exacto knife. I rinsed those well in tank water before returning them to the tank. Then I removed a dozen or so from within the tank with my tweezers.
In a day or two I am going to remove more and attempt to kill some with kalk.