One thing I forgot to mention - A lot of people buy a smaller tank, and then want to upgrade because they find out the fish they like need to be in a larger system. This happens a lot with tangs, who need a tank with plenty of swimming room. So, it all comes down to knowing yourself and what you really like in this hobby. If you really like corals, and the fish are just in there for movement, then you can make just about any size tank work. Same holds true if you like blennies, gobies and other small fish. But if you find you really love tangs (especially Acanthurus), large angels and/or predator fish (triggers, groupers, lions, eels, etc.) then my recommendation would be to get the biggest tank you think you can handle/afford.
I've been doing this for 35+ years now - my dad started me out at age 5, so I'm not quite as old as it might appear.

Like you, I've done freshwater tanks even seriously breeding angels & discus for awhile. I can tell you flat out there is no comparison. I might spend 5-10 mins per day doing something on my 90 gal planted tank. Maybe clean the glass every couple of weeks. But even with my current setup - just a 150 gal mixed reef - everyday it's at least 30 mins to 1hr of something that has to be done. And then I'm always cleaning/sterilizing QTs. Some days it's almost like having a part time job.