Depends on what the fish is in my opinion. Are we talking fresh or saltwater fish? And what fish are you feeding? I definitely wouldn't feed the liver of any carnivorous fish (too much uptake of other stuff gets lodged in there like mercury). On the other hand, fish roe is amazing, since all those eggs are lipid-y pearls of goodness. There's usually a nice chunk of meat in the fish "cheek" (I've been watching too many cooking shows again...), which is useful. And you can probably use the flesh that's left on the spine as well. The fins and gills are useless unless feeding predator fish. I'm willing to bet that the large/small intestines would make a good probiotic food as well; just make sure to remove the colon.
Whatever you do, make sure you freeze the fish flesh for an appropriate amount of time in order to kill off as many internal parasites as possible. Farm raised may come with chemicals (ooh, spooky) and antibiotics, but wild caught have all those happy bacteria and parasites and such.