That is not too high. I keep my phosphates around that and my montipora are growing insanely fast. If they are dying, it can be any combination of flow, chemistry, or lighting.
Also when you say they are dying, in what way are they dying (rapid bleach, slow death, etc)? How often do you do water changes and how big (and what size is the tank). Do you dose anything? What lights do you have?