The real Black Box light study

[dantimdad]

Sorry, I assumed (wrongly I admit) you were specifically defending that brand.

I do apologize.

 

[theKoolAidMan]

Sorry, I assumed (wrongly I admit) you were specifically defending that brand.

I do apologize.

No problem. And please don't take my responses as being hostile in any way. I am appreciative of the OP doing these tests, but just because he's taken the initiative and fronting the time and costs doesn't mean that we shouldn't be critical of the results and methodology where warranted. This should be about getting GOOD data, not just any data. Far too many best practices in this hobby have been built around shoddy data or a small kernel of science wrapped up in layers of anecdotes. Let's be better than that!

 

[Manose]

I will take pictures today of how we hold the seneye on the reef plug in order to get consistent data and if improvements can be made lets discuss.
I think a hobbyist data can be very valuable or as valuable as any real lab data run by a scientific firm, this test is about consistency and results obtained are used for a broad range of data.

I just now question what exactly is the point of this series? Is it to test how black boxes actually perform vs how name brand lights actually perform? Or is it for black box owners to get some data, no matter how inaccurate to point to in order to convince themselves they made a wise purchasing decision? My suspicion is that it has become the latter.

That is not the case at all...
Many black box owners know how well or not well their lights work in the real world application of reef keeping. This is a hobbyist study using a device that was praised by the BRS crew and was inexpensive. The accuracy may be a little off but it will be off for all lights and give us a consistent number throughout.
I welcome all the help I can get as well as some good constructive criticism, but to say this will be a biased test and put the black box in a far superior favor then please save those comments for somewhere else please.
I started this test as to maybe give something back to this wonderful community here at R2R so maybe we can learn something from it together as a community.

 

[Ariel V Rosa]

I like the direction this thread is taking, peer review is important and so is a discussion on the methods and devices used for data collection. I hope no one really uses this thread as a tool to bash brands or become keyboard heroes not willing to do the work them selves. The O.P. is putting lots of time an effort into this and that has to carry a lot of weight.

Hopefully further contributions are more constructive than inflammatory, its easy for us to get on a high perch over the internet and use language or fling oppinions we would not dare say in a setting with actual people sitting across from us.

 

[Manose]

I like the direction this thread is taking, peer review is important and so is a discussion on the methods and devices used for data collection. I hope no one really uses this thread as a tool to bash brands or become keyboard heroes not willing to do the work them selves. The O.P. is putting lots of time an effort into this and that has to carry a lot of weight.

Hopefully further contributions are more constructive than inflammatory, its easy for us to get on a high perch over the internet and use language or fling oppinions we would not dare say in a setting with actual people sitting across from us.

I will be honest and say that I am somewhat discouraged right now, but I started this and I will finish it with all the gathered knowledge here.

 

[Porpoise Hork]

In regards to the PAR results with the Kessil, is there any modification needed to the readings the SenEYE is giving? I ask this because with the Apogee units you have to multiply the sensor results by 1.32 to get the correct PAR reading. I wonder if something similar needs to be done here.

 

[Manose]

In regards to the PAR results with the Kessil, is there any modification needed to the readings the SenEYE is giving? I ask this because with the Apogee units you have to multiply the sensor results by 1.32 to get the correct PAR reading. I wonder if something similar needs to be done here.

That is a great question and I would love to know the answer.

 

[Johniejumbo]

Please don’t be discouraged! I’ve been following this for a while now and am really enjoying this project. It’s been my favorite thread! Keep up the good work.

 

[theKoolAidMan]

I will take pictures today of how we hold the seneye on the reef plug in order to get consistent data and if improvements can be made lets discuss.
I think a hobbyist data can be very valuable or as valuable as any real lab data run by a scientific firm, this test is about consistency and results obtained are used for a broad range of data.


That is not the case at all...
Many black box owners know how well or not well their lights work in the real world application of reef keeping. This is a hobbyist study using a device that was praised by the BRS crew and was inexpensive. The accuracy may be a little off but it will be off for all lights and give us a consistent number throughout.
I welcome all the help I can get as well as some good constructive criticism, but to say this will be a biased test and put the black box in a far superior favor then please save those comments for somewhere else please.
I started this test as to maybe give something back to this wonderful community here at R2R so maybe we can learn something from it together as a community.

My suggestion that there were ulterior motives in the tests was unfounded and disrespectful. Please accept my apology. I honestly do believe you're doing this for the right reasons and should not have said that.

 

[theKoolAidMan]

I will be honest and say that I am somewhat discouraged right now, but I started this and I will finish it with all the gathered knowledge here.

Please accept my apology if my comments about the motives for the test are what have discouraged you. That was uncalled for on my part.

 

[Manose]

Please accept my apology if my comments about the motives for the test are what have discouraged you. That was uncalled for on my part.

Do not worry as lets work together as to make this a really good study.

 

[theKoolAidMan]

Do not worry as lets work together as to make this a really good study.

Thank you for being gracious. I fear that my off the cuff quip made in bad judgement will cloud some good points numerous posters have been bringing up about the Kessil results.

 

[Manose]

I am here now as I stayed in a hotel as to avoid driving home late at night. I have the Noobsyche light hooked up and waiting to get some more readings. I will take pics of how we use the wand and hook the meter to the frag plugs so we can keep it the same level each time.

 

[naterealbig]

You didn't at all. Listen, if we all want to play scientist here...then you go the whole hog. We need to be able to have these open and frank discussions. There's a reason real scientific papers are peer reviewed.

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all LED channels at 100%.

 

[Manose]

Thank you for being gracious. I fear that my off the cuff quip made in bad judgement will cloud some good points numerous posters have been bringing up about the Kessil results.

No way as Kessil 360x is outstanding in my opinion as I noticed the huge area this light covers and covers very well. I am going test area on the light later on with a 48in x 24in frag tank.

 

[theKoolAidMan]

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all LED channels at 100%.

I would also add that Ryan has openly stated on camera on multiple occasions that they do not carry certain brands or certain products from certain brands due to poor reviews or them themselves not finding the product to be effective. He has said in the past that if a product starts getting poor reviews, they look into it, and if they find those reviews to be justified, they pull the product and refuse to sell it.

 

[Manose]

I agree 100% @theKoolAidMan. If there is a large margin of error already, then it will push to the resulting data sets for all tested lights - there is no way around it. Let me play devil's advocate here though: what if BRS test results are incorrect?. There will always be a margin of error that is exacerbated (or extrapolated) by using different test processes and equipment; the question is, what margin are we willing to accept?.

To what seems to be the original point of this thread: accuracy and precision is a must. Without accuracy, sure, the lights can be tested in relation to one-another, but this does not answer the original question - what is the efficacy of the Chinese Black Boxes on our tanks. How they relate to one another tells us nothing without an accurate baseline.

Meh to the accusations regarding BRS test results surrounding financial gain. To those touting this, have you actually watched the videos in their entirety?. In many videos, Ryan bluntly suggests that several of the products they sell (lighting, GFO, etc) are not necessary, or are only necessary under certain circumstances. He also routinely challenges vendors and OEMs of the products BRS sells.

In the Kessil video he does admit that they tested the new light differently than the old. The difference was the spectrum they used in the latest test was comparable to what we would see in your average reef tank. So, the results of this test are much closer to what we would see when the lights are actually being used. This data is much more valuable to me, as I would postulate that 0.0% of reef keepers use all uld findLED channels at 100%.

This is not about BRS as this is about what we can do to make this study better. I will say that BRS did inadvertently create a bad stereotype when they chose the cheapest made Chinese light they could find to do their study with. We have all seen the comments people make when you say you have a "Black Box" light. "OMG dude your house is going to burn down" NOT all lights made in China are crap and we need to use this study to see what is good or maybe good with more advanced testing. To do this we need to use some lights that WE KNOW will perform such as the Kessil 360x,Hydra 26,and Radion xr 15pro gen 4 to get some numbers to go by as to formulate which of the BB lights might be worth a better look.
BRS has done some amazing things for this hobby and I support them 1000% and will always appreciate what they do for us.

 

[madweazl]

...The only thing I am not seeing from the new X series is the claimed more light output...

I've done a fair amount of testing on the A360X and there actually is more output over the WEs but it comes from the ability to run the light lower to the water than the WE because of how wide the new lens projects light. To cover the same are as a WE, the X model can be set three inches lower and doing so increases PAR by a huge margin (approximately 50% for the configuration I used). With the X mounted 2.25" off the top of a 24" cube, it will cast light to the top of all sides (missing the corners a bit); the WE had to be mounted 5.5" above the tank to cover the same area. The down side to this is that you may end up dangerously close to the water if you decide to take advantage of the huge spread for the increase in PAR (I ran the WEs 5" above the water for almost two years without issue and would be comfortable running the X at the same height).

Dropping the light 3" from its current mounting height in this thread would produce at least 30% higher PAR results with no compromise to coverage (it would likely be even better with the additional light being reflected off the glass vice falling outside the tank altogether).

 

[naterealbig]

This is not about BRS as this is about what we can do to make this study better. I will say that BRS did inadvertently create a bad stereotype when they chose the cheapest made Chinese light they could find to do their study with. We have all seen the comments people make when you say you have a "Black Box" light. "OMG dude your house is going to burn down" NOT all lights made in China are crap and we need to use this study to see what is good or maybe good with more advanced testing. To do this we need to use some lights that WE KNOW will perform such as the Kessil 360x,Hydra 26,and Radion xr 15pro gen 4 to get some numbers to go by as to formulate which of the BB lights might be worth a better look.
BRS has done some amazing things for this hobby and I support them 1000% and will always appreciate what they do for us.

Thank you for your reply Manose, and my comments on BRS were not directed at you. I see some of the comments in the thread suggesting that some suppliers aren't being honest - which I will vehemently disagree with. I cannot imagine the money they are spending to do real testing for products, including the capital expenditures on setup, equipment, and film-making, and as you eluded to - these things should be recognized and respected. These are also concepts that cannot get lost, with regards to the work, time, money, and effort that you and others are devoting to this study. I'm sure it is discouraging to listen to critical comments, and I should have included some appreciative words in my last post along with the others. Thank you for the work that you are doing!

I agree, there was an insinuated and unfair stereotype for the CBB's that I'm sure many watchers walked away with from that video. I also agree that some products are better than others, and we cannot extrapolate a single test to all less expensive lighting options. I am planning on XR15's for my current build, but I am not married to a particular product or brand. One thing that I refuse to support however, is the blatant copy-catting of products with little-to-know research and development and quality control. Unfortunately, products like this feed the "cheap chinese" stereotype, and are incredibly detrimental to the companies that are doing the hard work and innovation, and to the end-user who is swayed by deceptive marketing and ignorance regarding what is required to produce a quality product.

Either way, I am interested to follow along and see what the results are. As others have mentioned, I think there are some critical aspects that should be measured that carry just as much weight as PAR (quality construction goes without mention). These are:

1. Spectrum blending (getting rid of the color separation)
2. Even coverage over a specific area

I would highly recommend tuning each light to spectrum that is likely to be used on a reef tank, and taking data points from there. This definitely introduces many variables into the testing, however, I believe the results will be much more useful to the end-user.

 

[symon_say]

Keep the good work, waiting for more results.