g5flier, I think you hit the nail on the head. I can't tell you how many times in the past couple of months I have seen posts almost identical to the OP. And the running theme through out most of them is the use of some amount of dead rock in the system. I myself am experiencing very similar issues, fish and nems happy, sps sad even the couple zoas I have in my tank aren't super thrilled (open but not extended, and certainly not reproducing). While I didn't used live rock, I used from from an existing tank of mine that had been essentially left to die (the rock that is) when it became completely invested with grape caulerpa. When the caulerpa was finally gone the rock was stark white, no sign of coralline, no spones, no other macro, nothing. For all intents and purpose it was dead When i started up my new tank, I just transferred the old now clean rock over and started up (See picture below for rock right after the transfer). At this point the tank has been running for 9 months. Any SPS i have ever tried to put in this tank will look good of a couple of days, then the polpys will close up, and the tissue will start peeling from the skeleton. I have been scratching my head and trying to figure out what is going on for months. Mind you I am not a novice. I have been keeping salt fish and corals for probably close to 15 years. I am totally on board with you this is a biodiversity thing. Reefs are very complex ecosystems.
Now I will add a small wrinkle to the story.... I have actually been able to save sps frags that have initially been placed in my main system and took a turn down hill to the point of some serious tissue loss. And how did I do this..... I transferred them to my small 15 gallon bare bottom frag tank that contains zero rock beyond the couple pieces that frags are mounted to. This system is completely isolated, just 15 gallons of water. What is interesting is that I can take a frag that is looking not so good and place it in there and with hours the polys are back out the coral is happy. SPS actually thrive in this tank, new frags are encrusting within days and so far I am getting excellent growth. Why it works so well i don't know. And it certainly doesn't support the biodiversity argument.
But what if we think about it as a numbers game..... the ratio of mirco fauna to inhabitable surfaces. Is it possible that if you have a low level of mico fauna and a high "inhabitable" surface area of substrate (rock, stand, etc) there is some period of "colonization" that occurs that creates an unstable environment. Now I am not talking bacterial necessarily, I am talking about all the of other things. Bacteria should spread to every surface in an aquarium very quickly. In this theory it wouldn't be the lack of creatures, it would be the actual expansion of these creatures that is actually causing the issue, and once you reach some critical mass where the actual colonization has leveled off or isn't expanding things start working. Which bring us back to the barren frag tank.... and why it works.... there is no other creatures trying to colonize the tank, so the "balance" has been reached. Could this all be complete BS.... sure, but it does seem to explain why one of my systems works and one of my systems doesn't and why using dead rock that has been seeded seems to take much longer time to support SPS than a tank that is filled with true live from with its surfaces covered with critters.
Would love to hear peoples thoughts.....