I think you’ll be fine. I’d actually recommend a larger group. I’d also recommend starting with all females. I have 5 lyretails & 5 Bartletts, along with 9 chromis and a couple other small fish in my 160 display. They’ve been in there for over a year. The distraction of the other fish keeps the social aggression diminished, but it’s always there. Feeding them often works well, but it’s a lot of nutrient export.
The bartlett male rules with an iron fin and keeps the other four separated in different places. Two of them are sub males.
The lyretail male speakers softer but carries a bigger stick. The lyretail male is stunning, orange and purple. He’s huge compared to the females. He was the only one of the lyretails to turn male.
They do not bother with other fish for the most part.
Good luck, they’re a blast to watch.