Interest in a fully Open-Source Controller Platform?

[cobra2326]

I've been really surprised by the number of ad-hoc DIY controllers people have put together. I've done it myself also. It has me thinking?

How many of you have built your own controller, or cobbled something together to do so?

Would there be interest in an OSS platform? I've designed a few boards for mine already, as well as put together a UI, data logging, etc.

I know there are already competing products/platforms, such as Reef Angel, Reef Pi, etc. I would also be willing to contribute to those. Let's kick off a discussion here

Please post the programming languages and/or hardware you're comfortable working with or have worked with.

 

[hijinks7]

I'd really only be interested if there was also a set standard for equipment to purchase into the system. Also I think using a raspi is the best option since python is more approachable.

 

[cobra2326]

I'd really only be interested if there was also a set standard for equipment to purchase into the system. Also I think using a raspi is the best option since python is more approachable.

Yes, this is the interesting part. ReefAngel does something like this, they sell the controller and hardware, but it appears that it's still a for-profit operation. Having standardized hardware is a must, IMO, but how to accomplish it while still keeping it fully-open and DIY is where I'm looking for suggestions.

Building a shield for the Raspberry Pi with core components (temp, pH, water level, EC, etc) would be easy and I'm basically doing that myself currently. Not really interested in building them for people, but there are various Fab houses that do assembly, and group buys would work.

 

[hijinks7]

I don't care about pre-made hardware.. just buy these things to put it together type listing.

My issue with homebrew controllers is the amount of time it takes to assemble ballons the cost of a Apex setup. Its a fun project though. I hate the apex UI so much I'd be willing to help if there was momentum on something.

 

[cobra2326]

I don't care about pre-made hardware.. just buy these things to put it together type listing.

My issue with homebrew controllers is the amount of time it takes to assemble ballons the cost of a Apex setup. Its a fun project though. I hate the apex UI so much I'd be willing to help if there was momentum on something.

OK, so this is possible. I'm thinking a shield with a USB bus (that's really just I2C) for anyone who wanted to extend it.

What do you hate about the Apex UI?

 

[hijinks7]

Its ugly in my opinion. Looks like they took parts of bootstrap and just rigged it into their horrible UI. The front page is just a mess to look at. You have no idea whats going on. Its just info overload. Ideally I only care about a few things and alerts.. I don't need info overload.

Also setting things up is just tricky for non tech people. I've seen people that can use computers but are hardly techy and they stumble all over the place with the UI

 

[revhtree Verified Admin]

Following!

 

[njtiger aquariums]

nice idea. I might be willing to aid (I am working on my own project slowly). Personally I would like a system that doesn't matter what hardware you use. That is one thing I dislike about some of the off-the-shelf products out there you need the purchase set hardware which put a damper of making the system flexible to meet each person needs; I don't believe every tank is the same and needs the same setup.

 

[hijinks7]

Right.. I think what I mean by hardware is this is what you need to buy to make like a breakout box work or a power strip. Not so much you are tied to this ATO or these pumps.

 

[njtiger aquariums]

Right.. I think what I mean by hardware is this is what you need to buy to make like a breakout box work or a power strip. Not so much you are tied to this ATO or these pumps.

ok I understand that

 

[cobra2326]

nice idea. I might be willing to aid (I am working on my own project slowly). Personally I would like a system that doesn't matter what hardware you use. That is one thing I dislike about some of the off-the-shelf products out there you need the purchase set hardware which put a damper of making the system flexible to meet each person needs; I don't believe every tank is the same and needs the same setup.

I think a combination of a core set of sensor support, exposed raspberry pi pins, and an I2C/SPI interface for various additional components will be flexible enough for just about anything. There are lots of cheap I2C ADCs that would be easily chainable for extensibility. The shield I currently have supports 9 PWM channels too, which is more than enough for controlling various LEDs and pumps. The only other core thing would be a power bar. I’ve done this already too, but I may need to add TRIAC support.

 

[Ranjib]

I would love to see more opensource controller options where we can combine our effort. I started with my own controller for pretty much the same reasons (and money) mentioned above.

I started building the controller same time I started reefing. I knew upfront that patience will be key with reef keeping and I am better off accepting veteran reefer's advice than doing anything in haste. Being a very impulsive person, the controller work gave me means to stay tuned with reef keeping while not to over-engineer my tanks. I am an avid software programmer and can code in pretty much most popular languages, I am also a core maintainer of Chef and bunch of other popular opensource software, so FOSS is pretty much my defacto mode of work.

Couple of things I learned while developing reef-pi is to standardizing the hardware. In the beginning, reef-pi was called reefer and I had intended to support more than one popular IoT /SBC platform (Pi, Beaglebone black, Galileo etc) later I realized its better to have a stable system supported on only one platform instead of half-baked support on multiple platforms.

The second key lesson I learned was to focus on only the relevant things. Early on I spend considerable amount of time on touch screen and bunch of other things that are less important from reef keeping perspective. Later I decided to prioritize features according to their relevance in reef keeping. This is why the development went from power strip -> ATO -> Temperature -> Timers -> LED control. To me, this is a very good combination of basic features and I can see a simple and effective controller only with these set of features, hence our first public release aka reef-pi 1.0 was cut soon after those features were tested and documented. I had plans for image processing and motion detection when I started .

One software engineering suggestion I would make is that dont go with scripting languages (ruby, python, javacript) if you really want to controller to last for years. Though they are easy to code, they are pain to maintain in the long run, they are less stable and they have poor performance. Remember every piece of serious software you use is written in statically typed languages, examples will be web browsers, operating systems etc. This also means its easy to distribute (since you can generate a single binary) . For a standard controller, you'll; need at least 10-12 concurrent threads , concurrent programming is bit more painful in scripting languages. ..

Happy to help any way I can, reef-pi bill of materials is available here: https://reef-pi.github.io/general-guides/bom/
At the same time I could use every bit of help with reef-pi as well. I am pretty shaky at both electronics as well as UI (bootstrap /javascript) . reef-pi uses Go for backend, and bootstrap+react on the frontend.

 

[Ranjib]

Its ugly in my opinion. Looks like they took parts of bootstrap and just rigged it into their horrible UI. The front page is just a mess to look at. You have no idea whats going on. Its just info overload. Ideally I only care about a few things and alerts.. I don't need info overload.

Also setting things up is just tricky for non tech people. I've seen people that can use computers but are hardly techy and they stumble all over the place with the UI

Agree. As much as I admire what it can do, I just didnt like the UI. Its super complicated. I dont blame it on them though. Its very hard to find good UX engineers. And since its a niche domain, its probably acceptable. It reminds me of lotus notes or sharepoint server style UX :0-)

This is something close to my heart, I want to keep reef-pi UI as intuitive as possible, which is going to be very hard.. particularly once programming features are introduced (i.e. internal DSLs like 'do this when that happens')

 

[Ranjib]

I think a combination of a core set of sensor support, exposed raspberry pi pins, and an I2C/SPI interface for various additional components will be flexible enough for just about anything. There are lots of cheap I2C ADCs that would be easily chainable for extensibility. The shield I currently have supports 9 PWM channels too, which is more than enough for controlling various LEDs and pumps. The only other core thing would be a power bar. I’ve done this already too, but I may need to add TRIAC support.

You dont need ADCs. I was under that impression in the beginning, but then the core set of sensor do not require any analog stuff. For example ds18b20 has one wire support in linux kernel, so you get the temperature as a text file. Water level sensors (leak detectors, photoelectric level sensors or float switches) are all digital (1/0). For ph, EC and ORP atlas scientific breakout boards will do (and they offer i2c).
The only thing I can think of that may use ADC is eTape style water level sensor or some other temperature sensor. I went through this route in the very beginning (reef-pi had support for mcp3008).

 

[njtiger aquariums]

A word of advice on UI/UX (I do AV programming for college control system so you know we have "highly educated" folks using our system so they need to be as simple and intuitive as it can be)

Keep to the KISS (keep it simple stupid) method and get someone who doesn't know anything about the system or hobby to look at it and see if they can understand it. The best type of UI/UX are system where anyone off the street can come in and use it with little to no help

 

[cobra2326]

I would love to see more opensource controller options where we can combine our effort. I started with my own controller for pretty much the same reasons (and money) mentioned above.

I started building the controller same time I started reefing. I knew upfront that patience will be key with reef keeping and I am better off accepting veteran reefer's advice than doing anything in haste. Being a very impulsive person, the controller work gave me means to stay tuned with reef keeping while not to over-engineer my tanks. I am an avid software programmer and can code in pretty much most popular languages, I am also a core maintainer of Chef and bunch of other popular opensource software, so FOSS is pretty much my defacto mode of work.

Couple of things I learned while developing reef-pi is to standardizing the hardware. In the beginning, reef-pi was called reefer and I had intended to support more than one popular IoT /SBC platform (Pi, Beaglebone black, Galileo etc) later I realized its better to have a stable system supported on only one platform instead of half-baked support on multiple platforms.

The second key lesson I learned was to focus on only the relevant things. Early on I spend considerable amount of time on touch screen and bunch of other things that are less important from reef keeping perspective. Later I decided to prioritize features according to their relevance in reef keeping. This is why the development went from power strip -> ATO -> Temperature -> Timers -> LED control. To me, this is a very good combination of basic features and I can see a simple and effective controller only with these set of features, hence our first public release aka reef-pi 1.0 was cut soon after those features were tested and documented. I had plans for image processing and motion detection when I started .

One software engineering suggestion I would make is that dont go with scripting languages (ruby, python, javacript) if you really want to controller to last for years. Though they are easy to code, they are pain to maintain in the long run, they are less stable and they have poor performance. Remember every piece of serious software you use is written in statically typed languages, examples will be web browsers, operating systems etc. This also means its easy to distribute (since you can generate a single binary) . For a standard controller, you'll; need at least 10-12 concurrent threads , concurrent programming is bit more painful in scripting languages. ..

Happy to help any way I can, reef-pi bill of materials is available here: https://reef-pi.github.io/general-guides/bom/
At the same time I could use every bit of help with reef-pi as well. I am pretty shaky at both electronics as well as UI (bootstrap /javascript) . reef-pi uses Go for backend, and bootstrap+react on the frontend.

Thanks for the feedback. I tend to agree on supporting a single platform for the core. I can think of a few ways to abstract details to support a broader range of platforms, but my guess is that would make it _less_ accessible.

The reason I wanted to start this discussion, and the goal of it, should be to end up with a generally good foundation on which to build controllers. It seems like a lot of people have cobbled things together which work for a while, then end up abandoning them and going to an off-the-shelf solution. Or do like I did and just spend 200+ hours building it for yourself. I'm guilty of it too, but the whole NIH thing is wasteful.

I definitely agree with your opinion on scripting languages. I built my controller in Rust, except for the few bits that run on an AVR, those are in C. It's been really high performance so far and the type safety has been a great help. (E.g. I started typing some units (C/F, mg/L, etc) and it has saved me several times already. Testing fills in the remaining gaps.

I'm curious as to why you feel it needs so many threads though. I do have some coordination going on between the web server and the pseudo-real time code, but otherwise I just have a simple loop which delegates to various modules using a tick concept, sort of like video games. There's plenty of downtime to where I'm sleeping the majority of the time.

I'll dig a bit more into the reef-pi code. I was skimming your thread yesterday and things are looking good. I will admit to having a slight bias against Go, mainly because of the type system, but that may be a moot point. I think I'd rather contribute than start another fork, unless there's a large, clear reason to start yet another project.

You dont need ADCs. I was under that impression in the beginning, but then the core set of sensor do not require any analog stuff. For example ds18b20 has one wire support in linux kernel, so you get the temperature as a text file. Water level sensors (leak detectors, photoelectric level sensors or float switches) are all digital (1/0). For ph, EC and ORP atlas scientific breakout boards will do (and they offer i2c).
The only thing I can think of that may use ADC is eTape style water level sensor or some other temperature sensor. I went through this route in the very beginning (reef-pi had support for mcp3008).

You're right about this, I've only needed ADC for the pressure sensor that monitors water level, and the on-board ADCs in the AVR are good enough for that. For the probes, you need a breakout anyway because they generally need an amplifier to read a useful value.

 

[hijinks7]

Maybe we should start a slack group for real time talk?

 

[Ranjib]

Thanks for the feedback. I tend to agree on supporting a single platform for the core. I can think of a few ways to abstract details to support a broader range of platforms, but my guess is that would make it _less_ accessible.

The reason I wanted to start this discussion, and the goal of it, should be to end up with a generally good foundation on which to build controllers. It seems like a lot of people have cobbled things together which work for a while, then end up abandoning them and going to an off-the-shelf solution. Or do like I did and just spend 200+ hours building it for yourself. I'm guilty of it too, but the whole NIH thing is wasteful.

I definitely agree with your opinion on scripting languages. I built my controller in Rust, except for the few bits that run on an AVR, those are in C. It's been really high performance so far and the type safety has been a great help. (E.g. I started typing some units (C/F, mg/L, etc) and it has saved me several times already. Testing fills in the remaining gaps.

I'm curious as to why you feel it needs so many threads though. I do have some coordination going on between the web server and the pseudo-real time code, but otherwise I just have a simple loop which delegates to various modules using a tick concept, sort of like video games. There's plenty of downtime to where I'm sleeping the majority of the time.

I'll dig a bit more into the reef-pi code. I was skimming your thread yesterday and things are looking good. I will admit to having a slight bias against Go, mainly because of the type system, but that may be a moot point. I think I'd rather contribute than start another fork, unless there's a large, clear reason to start yet another project.



You're right about this, I've only needed ADC for the pressure sensor that monitors water level, and the on-board ADCs in the AVR are good enough for that. For the probes, you need a breakout anyway because they generally need an amplifier to read a useful value.

Anyone who writes rust, gets free remote hugs from me :0) . Its one of the most elegant language I have seen quiet a while.
I went with go for a different reason, my options was go, rust , java or C/C++ . From my past experience with scala, i believe rust can be a significant learning curve for any new comers. It also means one can have endless bikeshedding (discussion that does not lead to any feature and performance benefits) around how to write a certain logic (since the language is rich and has many options), rust also has very limited stdlib and third party library support. Java/C++ shines on those fronts. Java would have cost significant resident memory footprint (not suitable for pi zero i'll say, though it is possible to run modern java under 15mb vmm). Go was a clear winner for me due to the combination of type system, and stellar http library . Its an extemely mediocre language, which forces you to focus on what you want from the software, while still gives some good concurrent primitives and simple build/cross compiling features.

I mentioned multiple threads because I modeled the entire controller into multiple , stackable sub systems. ATO, temperature controller, power bar, timers , lighting controller are all individual sub system and run in their own threads. Each thread can do its own independent polling for sensors etc. Some sub systems like timers can spawn sub-threads (go routines as we call them in golang).
http listener itself will be a separate thread. I have a separate main thread as well , which traps control signals etc.

reef-pi is modular, which means you can just turn on one sub-system and use it for that specific purpose, this allows users to use reef-pi just as a kessil controller or just as temperature controller.
Even when all subsystem is on, the resident cpu/memory footprint is less than 1/3rd

Here is the dashboard for one of my builds , powered by a pi zero , look at the low cpu /memory footprint

The actual controller

I bet with rust you can build a even more performant system

 

[cobra2326]

Maybe we should start a slack group for real time talk?

I'm down. Is there a reef-pi Slack (or IRC!?) channel already? I think for now I'll try to get familiar with reef-pi and contribute there. I don't plan on totally abandoning my stuff, since it's the only real way I get to play with Rust and other concepts like it, but having one solid, open source frontrunner is something I think the community needs.

 

[Ranjib]

I'm down. Is there a reef-pi Slack (or IRC!?) channel already? I think for now I'll try to get familiar with reef-pi and contribute there. I don't plan on totally abandoning my stuff, since it's the only real way I get to play with Rust and other concepts like it, but having one solid, open source frontrunner is something I think the community needs.

I just created one: https://reef-pi.slack.com
PM me your email ids so that I can invite you all.