I would not leave a pH electrode in kalkwasser constantly. it is not only a poor way to gauge potency, you will see pH drift unless you use a special electrode. The drift in the pH measurement may be substantial relative to potency changes. A drop from pH 12.54 (saturation at 25 deg C) to 12.40 is a drop of 38% potency.
Conductivity is a better way to monitor the potency. It is easy to detect a 10+% drop in potency that way. I still might not leave it 24/7 though since the electrodes may get coated with calcium hydroxide and carbonate.
This discusses the pH issue on page 29:
pH Measurements in Extreme pH or High Salt Content Samples Samples with a pH less than 2, pH greater than 12 and salt content greater than 0.1 M include battery acid, copper plating solutions and brines. These samples create measurement issues with the reference portion of the pH electrode, due to the liquid junction potential. The liquid junction potential occurs when there is a large difference in composition between the filling solution and the sample, resulting in measurement drift and slow electrode response. In samples with a pH of 2 to 12 or salt content less than 0.1 M, the liquid junction potential is rarely a problem since the filling solution is formulated for the best performance under these conditions.
In samples with a pH less than 2, pH greater than 12 or a salt content greater than 0.1 M, a pH electrode with a ROSS or double junction Ag/AgCl reference is recommended. The filling solution of the ROSS Ultra, ROSS and refillable Double Junction pH electrodes can be modified to match the sample composition, which will reduce the liquid junction potential and provide accurate, more stable measurements in these samples.
For samples with a pH greater than 12, add a dilute base to the recommended electrode filling solution to raise the filling solution pH. For samples with a pH less than 2, add a dilute acid to the recommended electrode filling solution to lower the filling solution pH. For samples with a high salt content, use a 30 | Thermo Scientific strong solution of the same salt as the electrode filling solution. For example, when measuring the pH of sodium bromide brines, use sodium bromide in the outer chamber of a double junction electrode.