Part of fish health is providing a good ecosystem for them to live in. Even though P does not affect fish in the water, I have always found fish to be more peaceful and even more colorful with N and P at reasonable levels (not low, but also not high) to where pods and microfauna can grow, coralline covers the rocks and glass and a fuge can grow grow macro algae.
I suspect that your lack of algae is twofold... first, your N and P are probably so high that they will not live, and two, they require traces that they are not likely getting if you stopped changing water. The algae might come back when you lower them, but this is kinda a good thing... not having algae because of super high N and P is kinda like being happy that you don't have to sweep the floor anymore because there is so much dog poop that you cannot sweep around it.