Self test failures

Hi guys,

I have been getting really crazy OD measurements in two of my reactors and i tried running a self test in one of them, but the self test doesn’t start and i get these messages:

What is this about? I would expect the at least some of the tests would be successful even of some fail. But they all remain grey, like the testing doesn’t even start.

what are the positive correlations between pds and leds?

Hm, so it sounds like the self-test is starting, but it’s not telling you that in the UI. Can you check that your UI version is >= 24.5.20 (we made some changes to the self-test UI that could cause this). To check your UI version, go to the Updates page, and hover over the dotted version line:
Screenshot 2024-06-03 at 8.51.52 AM

(or you can check by SSHing into your leader and typing pio version -v). Let me know what your App version is too.

This test checks that if we increase the IR intensity from LED channel A, we should see an increase in the PD signal (i.e. checking for a linear correlation). In your case, it looks like PD channel 2 doesn’t have a large enough correlation with the IR LED.

You’re also getting a “blank signal is too high error”, suggesting that there’s too much stray light, or a bad wire connection on the PD (check that the PD leads aren’t touching).

Hi Cameron,

Thanks for looking into this! So:

App Version: 24.5.22
UI version: 24.5.21

I put new covers on the PDs and the IR led to see if this helps.

How can I change the intensity of the IR led? I tried changing the intensity in the configuration from 50 to auto or to 70, but the output i get in the Pioreactors LED intensity for channel A is always 50%.

Also how about the first warning message about correlation between temperature and heating?

In the self-test, we hardcode 50% intensity, so it can’t be changed.

A value of > 5.0 (looking at your logs) seems very high, and makes me suspect that your REF value is actually too high. (Similar to the issue here). In the most recent version of our software, released last Friday, we made a change that removed the REF component from this self-test. Like in the linked post, if you plotted both PD signals, do they look “expected”?

Also how about the first warning message about correlation between temperature and heating?

Hm, this may be setup dependent, since I recall you were heating with an external source. Where is your heater PCB - is it near something that is warm?

App Version: 24.5.22
UI version: 24.5.21

Okay the UI looks correct - is the software updated to >= 24.5.22 on the worker who is running the self test? (Check on the inventory page)

Ok now that I updated all reactors to the latest software the self test is running and only failing for the photodiods and the heating PCB:

I did the test you proposed with the PD and i get the expected result i think.

As for the heating PCB the external heating has been off since yesterday and the Pioreactor is cold. I have changed the flat cable to a longer one, maybe this has something to do with it.

While interesting, I don’t expect it’s that. When you replace the cable, did you notice any damage to the heater PCB? Double check connections on both sides of the flat cable. From your logs, it looks like your reading temperature data correctly, but not distributing any heat.

In your config.ini under [PWM], is “heating” assigned to channel 5? (Make sure it’s not overwritten in the unit-specific config.ini, too)

For the OD failures: In your chart, I’m going to assume that the photodiode in the “REF” channel is the higher of the two (i.e. REF is the line at 0.037)? If so, that looks okay.

  • double check that [od_config.photodiode_channel] in your config has the correct assignments (Make sure it’s not overwritten in the unit-specific config.ini, too).

It may help for me to see the raw logs. Can you run

pio run self_test -k test_PD_is_near_0_volts_for_blank

on the worker this is failing on and provide the entire output? (here or email is fine).