"Parasite" in biology generally refers to a creature which lives on and feeds off of other creatures, and is adapted to that lifestyle. For example, mistletoe. It's a parasitic plant that grows on trees, feeds on their sap, and is rooted into the bark instead of into any dirt. Or there's ich, which lives in the skin of fish to feed off of them. Many parasites don't kill their host, but some may be lethal if there are too many (fleas can kill a severely infested animal via anemia), and some do wind up killing the host.
It is theoretically possible for a rock to bring in a parasite, but you wouldn't expect to see the parasite then grow and reproduce while living exclusively on the rock. You might have been looking for the term "pest"? That just refers to any unwanted organism.
Hopefully this has been at least a bit interesting to read. I like to tell people things about nature, particularly the lesser-known or less enjoyed types. Here's a tidbit; there are species of parasitic wasp which lay their eggs on the eggs, larvae, or pupae of other species. The larvae hatch and burrow into whatever they're on, then eventually pupate (chrysalis) and grow into wasps. It's thought that, for every single species of insect, there is an individual species of parasitic wasp that feeds on only that species of insect.