GFO is for phosphates, but since you don’t have a major algae problem you probably don’t have a phosphate problem, especially since you don’t currently have any fish to feed. Your refugium should be taking care of that I would think. I don’t know 100% what the issue is, or even if you fix the issue(s) that the nem will recover, but I do know that a buildup of nitrates is bad for a reef. The article I shared lists nems as something that would be harmed by excess nitrate. (Potentially anything over 5, and you are at 40!). I would also confirm those numbers if possible, any LFS should be able to test your water for you, confirm those numbers, and help make some recommendations as well.
how old is your sand bed?