reef-pi :: An opensource reef tank controller based on Raspberry Pi.

[Ranjib]

Been ramping up, but no unload this time. The CPU and memory readings, are those percentages?

CPU is percentage , memory is in MB.

 

[Ranjib]

I for one LOVE my little reef-pi. It is perfect, and I thank all of you who make this project possible! ( no warranty and all). I appreciate all the effort you have put into the software, and build guides.

Thank you. The community is great. I myself learn everyday something new .

 

[bishoptf]

this is not normal, or we should not be treating this normal. if reef-pi is being oomed (out of memory - killed) then its a serious bug and should be fixed asap. From your last stack trace what I could detect is theres vcgencmd command thats not been able to fork due to out of memory. That command is used to get cpu temperature. We used to show this data (cpu temperature) in past versions of reef-pi, i removed it from ui in 2.0. its still available via /api/info (which is the summary section in reef-pi). I could just remove the code and call it a day, but i think theres something else thats causing this effect.
Anyway, in case reef-pi is out of memory and get killed , systemd will restart reef-pi automatically, this is configured during install:
https://github.com/reef-pi/reef-pi/blob/master/build/reef-pi.service#L7
i have tried to reproduce this , but not been able to. If anyone else experiencing this, do let me know. Stability/reliability are of utmost importance to this project. I have happy to stop new feature work to get those gaps addressed to our desired level.
I'll keep a watch on this. Remember if this is happening, reef-pi will be automatically restarted , and you should see a different uptime (running since ... information in the summary section) in your reef-pi UI

Understand, that is what it did, restarted the reef-pi service but started fresh with the stats, I thought that the stats were written to the database and would just be reloaded. Is that the intent when reef-pi service restarts, I've not played with it enough to have a baseline to know the correct answer. I checked one of my other boxes thats on UPS, its running Centos with some vm's for various stuff, here is the uptime on it:

bishop@f15eagle ~]$ uptime
22:35:43 up 547 days, 7:15, 1 user, load average: 0.42, 0.32, 0.33

Once things are dialed in for reef-pi it should be able to run for a long time.

 

[dmolavi]

this is not normal, or we should not be treating this normal. if reef-pi is being oomed (out of memory - killed) then its a serious bug and should be fixed asap. From your last stack trace what I could detect is theres vcgencmd command thats not been able to fork due to out of memory. That command is used to get cpu temperature. We used to show this data (cpu temperature) in past versions of reef-pi, i removed it from ui in 2.0. its still available via /api/info (which is the summary section in reef-pi). I could just remove the code and call it a day, but i think theres something else thats causing this effect.
Anyway, in case reef-pi is out of memory and get killed , systemd will restart reef-pi automatically, this is configured during install:
https://github.com/reef-pi/reef-pi/blob/master/build/reef-pi.service#L7
i have tried to reproduce this , but not been able to. If anyone else experiencing this, do let me know. Stability/reliability are of utmost importance to this project. I have happy to stop new feature work to get those gaps addressed to our desired level.
I'll keep a watch on this. Remember if this is happening, reef-pi will be automatically restarted , and you should see a different uptime (running since ... information in the summary section) in your reef-pi UI

Could this be platform dependent, Pi zero vs 3B+?

 

[DirtDiggler2823]

Could this be platform dependent, Pi zero vs 3B+?

I'm not sure about that. Mine is behaving strangely as well. Just got this error while trying to reboot. I noticed my temperature was a bit low, so I tried to do a soft reboot, and that is the error message I received.

 

[b4tn]

Are you guys with memory issues running anything nonstandard or custom code? Non standard as in something different from the build guide? I have not messed with any of the code save for adding the pwm dtoverlay to the config file and installing the updates as they come out. I’m using a pi3 B+ the only thing I’m not running is a ph probe. All the other stuff dosers, lights, and regular functions are all running off one rpi headless. I’m using 3 channels of the pca9685. I seem to be pretty steady on memory and cpu usage.

 

[DirtDiggler2823]

Are you guys with memory issues running anything nonstandard or custom code? Non standard as in something different from the build guide? I have not messed with any of the code save for adding the pwm dtoverlay to the config file and installing the updates as they come out. I’m using a pi3 B+ the only thing I’m not running is a ph probe. All the other stuff dosers, lights, and regular functions are all running off one rpi headless. I’m using 3 channels of the pca9685. I seem to be pretty steady on memory and cpu usage.

Nope. Just running the code that was provided. Not using anything special at all. I'm running a pi3b, so I should have more resources to leverage.

 

[LionHeart2017]

My memory ramps with the timers for my lights from being turned on to off if that helps

 

[dmolavi]

Headless vs non-headless Raspian installs, maybe? I used the "Lite" version - no desktop or UI, just console, from https://www.raspberrypi.org/downloads/raspbian/.

 

[DirtDiggler2823]

Headless vs non-headless Raspian installs, maybe? I used the "Lite" version - no desktop or UI, just console, from https://www.raspberrypi.org/downloads/raspbian/.

I might not be running headless.....

 

[b4tn]

I might not be running headless.....

I used the GUI once to set up my network and never used it again so went ahead and disabled it in raspi-config.

 

[DirtDiggler2823]

I used the GUI once to set up my network and never used it again so went ahead and disabled it in raspi-config.

I might have to do the same once I get home from work

 

[dmolavi]

Also, I don't know what the "recommended software" is which gets installed with the first of those three images on the Pi site. Some of those may contain additional services, etc that eat away at memory over time. I think that the "Lite" version is the best option for our use. Setting up the network via the command line isn't difficult (even if you're not a Linux person). I made some changes to the image so that SSH and WiFi were enabled as soon as it was flashed and boots. I could probably dig those up and post them here for inclusion on the Reef-Pi site...

 

[dmolavi]

Here we go:
1. After you flash the image to the micro-SD card, open the /boot/ partition in your file manager.
2. Enable SSH by placing a file named “ssh” (without any extension) onto the /boot partition of the SD card.
3. In order to connect to a wifi network, create and edit wpa_supplicant.conf (in your /boot partition) and add the following lines:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
ssid="my-network-name"
psk="my-network-pass"
key_mgmt=WPA-PSK
}

Save the file, eject/unmount the micro-SD card and use it to boot up your Pi. From your wireless router, you can find the IP address assigned to "raspberrypi" and do:
ssh pi@<ipaddress>

 

[b4tn]

Here we go:
1. After you flash the image to the micro-SD card, open the /boot/ partition in your file manager.
2. Enable SSH by placing a file named “ssh” (without any extension) onto the /boot partition of the SD card.
3. In order to connect to a wifi network, create and edit wpa_supplicant.conf (in your /boot partition) and add the following lines:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
ssid="my-network-name"
psk="my-network-pass"
key_mgmt=WPA-PSK
}

Save the file, eject/unmount the micro-SD card and use it to boot up your Pi. From your wireless router, you can find the IP address assigned to "raspberrypi" and do:
ssh pi@<ipaddress>

This is useful! I tried to do this via a google search but it didnt work. Looks like I missed a few steps.

 

[crusso1993]

I might not be running headless.....

@DirtDiggler2823 @dmolavi @LionHeart2017 @Tom Bishop

I ran headless once. Scared all the neighbors!

Ok, seriously, if you're running 3 B+ you should have more than enough resources to run a full reef-pi system using the UI and all.

It seems to me that maybe there might some additional background service(s)/programs running. Maybe you guys downloaded the same version of the software used and therein lies the issue. Since you all appear to be having the same issue, I would think there has to be a commonality between all of you. Yeah, I realize this logic may be erroneous as we are dealing with different base models but it might be a good start for you guys to post your SPi model and the software downloaded. I mean, we have to start somewhere, right?

Or maybe I'll just continue to run around the neighborhood headless!

 

[rushbattle]

Here we go:
1. After you flash the image to the micro-SD card, open the /boot/ partition in your file manager.
2. Enable SSH by placing a file named “ssh” (without any extension) onto the /boot partition of the SD card.
3. In order to connect to a wifi network, create and edit wpa_supplicant.conf (in your /boot partition) and add the following lines:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
ssid="my-network-name"
psk="my-network-pass"
key_mgmt=WPA-PSK
}

Save the file, eject/unmount the micro-SD card and use it to boot up your Pi. From your wireless router, you can find the IP address assigned to "raspberrypi" and do:
ssh pi@<ipaddress>

Thanks for this, I keep forgetting to post it up!

Also, I did install Lite via terminal and I still needed to disable the GUI in raspi-config. It lowered mem and cpu usage quite a bit. No idea why or how that might be.

 

[stefanm]

I got the chance earlier to test the PCA 9685 again for the dosing, still not working properly, I then tried one of the pwm outputs on the pi and working as it should.
I then tested the pwm output on the PCA and it was as always on showing 3.79 volts, so after all it did get damaged on the Arduino when the rtc caught fire!

So I'm ordering a replacement, hopefully by the weekend I'll get it.

The only other thing I need to configure is the 8 channel relay, as I'll be using a separate PSU for it do I still need to tie the grounds, I had one hooked up to an Arduino and pretty sure I didn't tie the grounds, I believe the idea behind this is to completely isolate the microcontroller incase of failure, I guess it'll either work or not.

Just one question is the output active high or low, the board i have has a jumper selection for this?

 

[Ryan115]

I have been experienceing the growing memory usage for a while as well. I am running a custom release that was sent me a while back from @Ranjib so that I could address my PCA9685 (Version 2.0-2-gae06032).
I am running headless on a Pi Zero with the Adafruit Motor Control Hat. Running 2 lights, 2 temp probes, 1 float switch/ATO, 8 outlets with 5 timers.
I was able to restart through through terminal with

sudo systemctl restart reef-pi.service

Otherwise I would get the same error about not being able to allocate memory.

CPU is percentage , memory is in MB.

I know I shouldn't ask, but are you 100% on RAM being MB?
When my system was running high before this reboot, ps aux showed %RAM to be nearly the same that reef-pi was showing (~80%).

 

[Ranjib]

I have been experienceing the growing memory usage for a while as well. I am running a custom release that was sent me a while back from @Ranjib so that I could address my PCA9685 (Version 2.0-2-gae06032).
I am running headless on a Pi Zero with the Adafruit Motor Control Hat. Running 2 lights, 2 temp probes, 1 float switch/ATO, 8 outlets with 5 timers.
I was able to restart through through terminal with

sudo systemctl restart reef-pi.service

Otherwise I would get the same error about not being able to allocate memory.


I know I shouldn't ask, but are you 100% on RAM being MB?
When my system was running high before this reboot, ps aux showed %RAM to be nearly the same that reef-pi was showing (~80%).

You are right, the memory usage is in percentage, I was wrong I’m saying it’s in mb. I just crosschecked my code , and it’s in %. Do you mind sharing a summary of all the things you are running ? I’ll try to diagnose the issue. I am running couple of temperature controller , ato, light and 8 equipment and have around 25% memory usage at max.