You won’t necessarily get ammonia spikes if your tank is fully cycled, but it will break down and increase your nitrates and phosphates. I periodically use a smaller diameter vinyl tube to clean out mine.
Any left over food or detritus in your overflows and sump should be cleaned when doing water changes. I don’t get all of mine every time but at least once a month.