I almost forgot of this thread, so good thing it has gotten some new messages!
Luckily I got rid of my vermetids so I will share what helped for me.
Like I said in my previous post, I tried coral snow. It did clear up my tank, but didn't help much on the vermetids. After that I started my big offense against them, and luckily I succeeded.
1. Feeding
First thing I did, was trying to cut off their food source. I used to broadcast feed my tank with all the pumps on, and that's when all the vermetids went crazy with their nets. So I changed my feeding routine to all pumps off and feeding very little so only the fish were able to eat. I read once on R2R, you need to feed the fish and not the tank. If you ask me that is a very important one to remember (also in nutrient control).
2. Manual removal
The changed feeding routine helped a lot and the population was definitely not growing anymore. So second thing I did was trying to smash as many vermetids as I could. I used a small knife to remove them from corals and pro tip: some big tweezers can be your best friend in destroying vermetids on the rocks.
3. Introducing natural predators
Last thing I did was introducing natural predators. Other people in this thread already mentioned bumble bee snails and wrasses. I thing wrasses are a hit or miss. Not all wrasses will eat them, and some are just happy eating the food you give them and snacking on some copepods instead of eating a vermetid. My yellow wrasse got a taste of the vermetids when I was crushing them, and started eating them occasionally. I also added ~8 bumble bee snails to my tank. I'm very certain not all of them ate vermetids. But I feel that if you get enough of them, at least a couple will eat vermetids.
Future prevention
All the previous steps helped eradicate the vermetids. Now it's important to make sure they don't come back. The only real way to prevent them coming back, is probably dipping all corals and putting them in a coral QT for a couple of weeks. For me that's not really an option, so I just inspect all my corals very closely and scrape off any vermetids I see on the frag. I also stopped glueing them directly, so I can easily remove them to scrape off any possible vermetid.
Final words
Hopefully my experience helps for other people to control/eradicate their vermetid population. The most important thing if you ask me, is that you stay patient in looking for a solution and ask for help (on R2R or possibly your LFS). I was on a point where I thought I would never control them and I was even considering to remove all my rocks and start over my tank. I had vermetids in my overflow box, my sump, skimmer etc. But now everything is clear luckily. Try to figure the out the cause of the problem and address that first. After that things will get easier. Good luck everyone and let me know if you have any questions!