The good news is, you have many options.
Obviously, you could remove the rock....but I'm guess you already know that...
If they are easily accessible, most Palys are pretty easy to peel off the rock work. The key is to carefully lift up part of the mat with a fingernail or razor blade, and then very carefully separate the mat from the rock with a razor or credit card. This is sometimes easier said than done, but worth a shot. It's essentially free.
You can also chemically kill them. Large dosers of Aiptasia X have killed some polyps of mine in the past, but you'd need to suffocate he entire colony. You could also make homemade kalk paste out of pickling lime. THIS WILL RAISE YOUR ALKILINITYY AND CHANGE YOUR pH, SO JUST BE AWARE!!
You can also coat them in superglue. Might take a lot of glue, but it works.
Boiling water in a syringe or turkey baster may also do the trick, as I know it works on aiptasia, but I've never personally tried it to eradicated Palys.
Also, you can just put another coral over the top of them so at they are completely shaded. If it's dark enough, they'll starve and die.
So yeah, you have options. Good luck!