as Rusty stated, new tank syndrome is something everyone goes through so don't stress. the system just needs to develop a sufficient amount of beneficial bacteria and microfauna naturally (this will take time-months). A sufficient clean up crew is also essential. Hermits, mexican turbos, cerith snails, astrea snails, and nassarius snails are a good start. I like to have nassarius snails in my mix because they go into the sand bed and eat detritus. Also it's cool to watch them come out of the sand bed when feeding and move quick to find any uneaten food. of all the snails i mentioned i think ceriths are the best all around eaters. Conch snails are great too but they aren't easy to find in LFS around me and the two i've had were bulldozers and would move many of my sand dwelling corals.
if your LFS doesn't have a decent selection of cleaners then you may want to consider a package from a company like reefcleaners
https://www.reefcleaners.org/aquarium-store/cleaner-packages-with-free-shipping
what kind of clean up crew have you employed so far?
what are your water parameters?