I did the toothbrush treatment to some of the rock last water change, it all comes off fairly easily. I didn't do the rock in the photos because the blenny was hiding in it at the time.

And I have scraped it off the back wall...3 times now it's comeback has slowed, but I also was using Vibrant which I've stopped for the time being. The water is staying clear, but the rusty algae is coming back on the glass more frequently.
I have been under the assumption this is just a hair algae issue...I hope? As that's easier to kick than bryopsis.
I am going to guess pH is low, as it's low in my other tank with a probe. As in...7.6 to 7.9 depending. Lows in the morning, "highs" after the photoperiod. I never have pH readings at 8.0 or above now that windows are closed full time in the house.
I do have some MarinePure spheres in one of the back chambers...if nutrients are low (maybe? maybe not?) should I pull them?