Yes, he is skinny enough to worry. Mandarins have a "point of no return" where they won't be able to recover. I don't think you are there quite yet, but I do think you need to act quickly.
Nutrimar ova was a miracle food for my Mandy. I highly suggest ordering some if you can't get it local. Also, you could try hatching live brine shrimp.