Very low OD readings map to max

One of the release notes of software version 24.9.19 is:

Fixed an issue where a calibrated OD reading would be mapped to max OD signal if it was too low.

It seems like I am still getting this issue. My pioreactors are filled with water, but one of them supposedly has an OD of 4.061, which is exactly the maximum of my OD calibration. I have noticed this a few times previously. (Currently running the latest version v24.12.10.)

I have had a few errors like this one, which confuse me. The error says that the signal is outside the calibration range, but the observed voltage is actually withing the calibration range. Note that these errors also occur for a few pioreactors that do not map to the maximum signal.

Signal outside suggested calibration range. Trimming signal. Calibrated for OD=[0, 4.06], V=[0, 0.546]. Observed 0.002V.

Do you have idea how to solve this?

@mandy-twig for this pioreactor, can you send me the calibration data? Ex:

  1. SSH into that worker, run
     pio run od_calibration display
    
  2. Copy the JSON blob at the end (The bits between the { and }).

While you’re there, can you confirm the version is 24.12.10 as well:

pio version

I’m going to try to reproduce the error here.

Data for od-cal-2024-12-02-M9

{
“type”: “od_90”,
“created_at”: “2024-12-05T15:27:38.333562Z”,
“pioreactor_unit”: “pio24”,
“name”: “od-cal-2024-12-02-M9”,
“angle”: “90”,
“maximum_od600”: 4.061,
“minimum_od600”: 0.0,
“minimum_voltage”: 0.0,
“maximum_voltage”: 0.5463,
“curve_type”: “poly”,
“curve_data_”: [
-0.03112259838616315,
0.14606367297714123,
0.05224678328234911,
0.009665339167023364
],
“voltages”: [
0.0,
0.0158,
0.0322,
0.0589,
0.1002,
0.1648,
0.4045,
0.5463
],
“od600s”: [
0.0,
0.139,
0.155,
0.378,
0.671,
0.993,
1.82,
4.061
],
“ir_led_intensity”: 50.0,
“pd_channel”: “2”
}

pioreactor@pio24:~ $ pio version
24.12.10

Thanks @mandy-twig, I know what’s going on. It’s a consequence of how calibrations are “fit”. The polynomial fitted to the dataset, y=-0.03x³ + 0.15x² + 0.05x + 0.01, looks like:

but if I zoom in on the x-axis,

So when we do the inference in the code, we are solving 0.002 = -0.03x³ + 0.15x² + 0.05x + 0.01, which is the same as drawing a horizontal line at 0.002 and seeing where it intersects the curve. As you can see, the only (positive) intersection occurs x~=5.5, and that’s why you see a large value + warning in the Pioreactor (truncated to 4.061).


Our fitted polynomial is silly: your inputted data has the point (0,0), so our fitted curve should have the property that f(0) ~= 0. We can make some tweaks to the fitting algorithm to bias this.

Let me think more a solution for you now, and how we can generally avoid this in the future.

So I think the bug is in our polyfit, where we have a bias that weighs the regression towards the largest value, where it should in fact be the smallest value (the blank). This generally isn’t an issue, compare these two calibration curves before (old) and after (new) the bug fix:

however the error you are seeing above can creep up: notice the slight difference in the curves near x=0 - that’s the problem!

Problem is diagnosed, next is to solve it for existing calibrations!