I wouldn't feed them until they are well established inside the tank and thriving. You do not have to feed them at all ever, it just makes them grow a little faster.
And if you do feed, make it very small amounts. Don't give it large chunks of something even after it starts thriving.
Make sure you acclimate them well. Then put them in the tank and then leave them alone.
If that's not working for you, then it's likely your water, lights, or flow.
Also, people mention tank age, but I think that's a lousy way of going about it. I have a RBTA that is about 12 inches across when it stretches during the day. I put him in a 29g tank that was only 2 weeks old with dry rock. It did fine and has never once showed any issues over the past 11 months. Because it's not about tank age, it's about stability.
Meanwhile I have a 25g cube and reef that is nearly 2 years old. I would NOT put the RBTA in that tank because even though it is older, I wouldn't exactly call it stable. I keep it well stocked and I feed it well. Sometimes too well. So I from time to time get small bouts of cyano etc. The cyano typically goes away on it's own, it's very mild, but it shows that the tank isn't exactly stable even though it's much much older.
And yet, I've never once had a single bit of the "uglies" in my 29g anemone tank. No cyano ever. Because it's a tank that is dedicated to the anemone and the 2 clowns in it. It's very lightly feed, thus is has remained stable the entire time. Very little bioload.
Currently I'm in the process of building a 180g tank. Would I put my anemone in that tank right off? No. Because I plan on having many fish, many other coral etc. So this tank is going to take some time to stablize it's biologicals. If I was only going to do the anemone with a way low bioload, then I might consider it(except I have no plans on ever putting it or any other anemone in there).