So just to update, I went ahead and did it. I ended up using cheap, Minwax oil based polyurethane from Lowes and it went just fine. 2 weeks later all my inhabitants are doing well with no sign of stress either at the time or now.
I don't know if my thought process was correct, but it went something like this:
All polyurethane varnish has some sort of volatile solvent that evaporates off, leaving the polyurethane behind along with fumes from the evaporated solvent. For traditional, oil-based polyurethane, it's a some sort of hydrocarbon. For water-based, it's generally a glycol ester, although I've used some that at least smell like they have a component of ammonia in them as well.
For water-based coatings, the solvent is by definition water soluble, where as for oil-based coatings it is water insoluble. As such, the fumes created by a water based coating would be much more likely to get absorbed into the tank water and potentially cause issues, where as the opposite would be true with fumes from the oil-based solvent. Of course that assumes equal toxicity at equal concentrations, etc, so it could be totally wrong, but it sounded good in my head, and in the end things went fine.
Thanks for your help!