I pretty much did that for many years is how I know.
The fact of the matter is water changes alone aren't going to replace everything they need. You are replacing the existing water with 100% salt water. Meaning, if you do a 20% water change, then you only replenished 20% of the things your coral use.
If they happen to use less than 20% of those elements, you won't notice much. Otherwise, what will happen is you will have periods of good growth(thriving) and periods of no growth(surviving).
In my tank I rarely do water changes in, it's really noticeable with coraline algae. After I do a water change, I'll have a short period of time where it grows in the tank. Eventually it uses things up, stops growing and my urchins eat it up.