If your rock doesn't have bound up phosphate and you aren't feeding or adding anything with phosphate, there is no phosphate input into the tank.
If you are adding food for the nitrogen cycle, you likely aren't adding much, so there isn't much going into the system. The little that would be going in would be binding to the rocks and sand. I think I had 40-50 ml of phosphate into my 20 gallon system over a week before I was registering anything on my Salifert test kit. It didn't stay detectable for long. The phosphate will bind to the sand and rocks, so it will take a little while to get the phosphate to stay in the water column.
Phosphates will have the reverse happen, phosphate will leach from the rocks and sand, when the levels in the water column decrease, which is why people struggle to get phosphates down with water changes.