Many LPS will do well in the sand bed and your hermits should not cause any issues - mine certainly don’t. I keep Trachyphillia, Cynarina, Hammer, Bubble, Torch and Fungias in my sand bed. All doing great. And large Zooanthids (possibly Palys, not 100% sure, not suggesting you keep Palys, just full disclosure) on a rock at sand bed level - they happily grow up and down the rock from sand level up. I’ve even got Xenia that have formed small colonies attaching to a bit of rubble in the sand.