Inconsistent Dosing Behavior with multiple media

Software
1

Hello. I believe I have identified some inconsistent behavior with the /automations/dosing/base.py execute_io_action() function (line 281). I wanted to bring this to your attention so that you are aware of the possible bug. I have included the code I am running and the log at the bottom of my post.

I wanted to test how the pioreactor handled a main media and an alternative media, so I wrote an automation that should exchange 1 mL main media and 1 mL alternative media every 2 minutes. Here is how the pioreactor performs this automation:

[add_media] add 0.5 mL of main media
[remove_media] remove 0.5 mL of media
[remove_media] remove 1.0 mL of media

[add_media] add 0.5 mL of main media
[add_alt_media] add 0.5 mL of alt media
[remove_media] remove 1.0 mL of media
[remove_media] remove 2.0 mL of media

[add_alt_media] add 0.5 mL of alt media
[remove_media] remove 0.5 mL of media
[remove_media] remove 1.0 mL of media

The issue is that the Pioreactor only checks to see if the volume of any single media added is above the maximum volume limit, but does not check whether the total volume of media added is also above that limit. If, for example, I added 5 different pumps/sources of media, the Pioreactor would not detect that I am adding 2.5 mL at a single time and am risking overflow. I have provided two examples below of what I think a more consistent dosing behavior would look like.

Currently, it looks like this:
Exchange 1 - [add_media = 0.5 mL, add_alt_media = 0 mL, remove_waste = 0.5 mL]
Exchange 2 - [add_media = 0.5 mL, add_alt_media = 0.5 mL, remove_waste = 1.0 mL]
Exchange 3 - [add_media = 0 mL, add_alt_media = 0.5 mL, remove_waste = 0.5 mL]

Solution 1:
Exchange 1 - [add_media = 0.5 mL, add_alt_media = 0.5 mL, remove_waste = 1.0 mL]
Exchange 2 - [add_media = 0.5 mL, add_alt_media = 0.5 mL, remove_waste = 1.0 mL]

Solution 2:
Exchange 1 - [add_media = 0.5 mL, add_alt_media = 0 mL, remove_waste = 0.5 mL]
Exchange 2 - [add_media = 0 mL, add_alt_media = 0.5 mL, remove_waste = 0.5 mL]
Exchange 3 - [add_media = 0.5 mL, add_alt_media = 0 mL, remove_waste = 0.5 mL]
Exchange 4 - [add_media = 0 mL, add_alt_media = 0.5 mL, remove_waste = 0.5 mL]

Here is the code I am running via ssh.

# -*- coding: utf-8 -*-
from __future__ import annotations

from pioreactor.automations import events
from pioreactor.automations.dosing.base import DosingAutomationJobContrib
from pioreactor.exc import CalibrationError
from pioreactor.utils import local_persistant_storage
 

class ChemostatAltMedia(DosingAutomationJobContrib):
    """
    Chemostat mode - try to keep [nutrient] constant.
    """

    automation_name = "chemostat_alt_media"
    published_settings = {
        "volume": {"datatype": "float", "settable": True, "unit": "mL"},
        "fraction_alt_media": {"datatype": "float", "settable": True, "unit": "%"},
        "duration": {"datatype": "float", "settable": True, "unit": "min"},
    }

    def __init__(self, volume: float, fraction_alt_media: float, **kwargs):
        super(ChemostatAltMedia, self).__init__(**kwargs)

        self.volume = float(volume)
        self.fraction_alt_media = float(fraction_alt_media)

    def execute(self) -> events.DilutionEvent:
        
        alt_media_ml = self.fraction_alt_media * self.volume
        media_ml = (1 - self.fraction_alt_media) * self.volume
        
        self.execute_io_action(alt_media_ml=alt_media_ml, media_ml=media_ml, waste_ml=self.volume)
        volume_actually_cycled = self.execute_io_action(alt_media_ml=alt_media_ml, media_ml=media_ml, waste_ml=self.volume)
        return events.DilutionEvent(
            f"exchanged {volume_actually_cycled[0]}mL",
            data={"volume_actually_cycled": volume_actually_cycled[0]},
        )
    
if __name__ == "__main__": 
    from pioreactor.background_jobs.dosing_control import DosingController
    
    dc = DosingController(
        "chemostat_alt_media",
        duration=2,
        fraction_alt_media=0.5,
        volume=2.0,
        unit="test_unit",
        experiment="test_experiment"
        )
    dc.block_until_disconnected()

Here is a portion of the Log file.

2023-01-04T00:12:17+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:12:29+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:12:36+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:12:50+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:13:02+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:13:10+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:13:23+0000 [remove_waste] [app] INFO 2.0mL
2023-01-04T00:13:48+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:13:57+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:14:04+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:14:17+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:14:29+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:14:37+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:14:50+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:15:02+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:15:10+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:15:24+0000 [remove_waste] [app] INFO 2.0mL
2023-01-04T00:15:48+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:15:56+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:16:04+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:16:17+0000 [dosing_automation] [app] INFO DilutionEvent: exchanged 1.0mL
2023-01-04T00:18:17+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:18:29+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:18:36+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:18:49+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:19:02+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:19:10+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:19:23+0000 [remove_waste] [app] INFO 2.0mL
2023-01-04T00:19:48+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:19:56+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:20:03+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:20:16+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:20:29+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:20:36+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:20:49+0000 [add_media] [app] INFO 0.5mL
2023-01-04T00:21:01+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:21:09+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:21:23+0000 [remove_waste] [app] INFO 2.0mL
2023-01-04T00:21:48+0000 [add_alt_media] [app] INFO 0.5mL
2023-01-04T00:21:56+0000 [remove_waste] [app] INFO 0.5mL
2023-01-04T00:22:04+0000 [remove_waste] [app] INFO 1.0mL
2023-01-04T00:22:17+0000 [dosing_automation] [app] INFO DilutionEvent: exchanged 1.0mL

2

!!!

You’re really doing incredible things here! Thanks for providing a detailed summary (with logs and code) of the issue you are seeing. Let’s take a peek:

  1. First thing, you are running self.execute_io_action twice in your execute - I’m guessing this isn’t intended and maybe a leftover from debugging? As currently written, the code is dosing 4ml instead of 2ml.

  2. I believe I understand your suggestion. Correct me if I’m wrong, but the issue isn’t possible atm. Currently theres only 2 media pumps, and the theoretical max you could add before removing waste is 0.625ml * 2 = 1.25ml. (The 0.625ml is hardcoded in the function)

  3. However, if I’m understanding correctly, we just want to remove X ml immediately after we add X ml - is that right? This is a good suggestion, and isn’t hard to add! (Since you’re digging into the source code: in this final else statement in execute_io_action, we’ll copy the remove waste logic to act after media and alt media)

  4. Another way to accomplish this is to seperate the additions into two calls to execute_io_action:

    def execute(self) -> events.DilutionEvent:
        alt_media_ml = self.fraction_alt_media * self.volume
        media_ml = (1 - self.fraction_alt_media) * self.volume
        
        alt_media_cycled = self.execute_io_action(alt_media_ml=alt_media_ml, media_ml=0, waste_ml=alt_media_ml)
        media_cycled = self.execute_io_action(alt_media_ml=0, media_ml=media_ml, waste_ml=media_ml)
        return events.DilutionEvent(
            f"exchanged {media_cycled[0]}mL media, and {alt_media_cycled[1]}ml alt media",
        )
3

Thanks for the detailed response!

Regarding #1, you are correct and thanks for catching that.

For #2, you are correct and the issue isn’t possible, but I wanted to bring it up because I realized it could be a potential future issue due to the way execute_io_action was conducting the checks. Additionally, I have been planning on attempting to add additional pumps at some point in the next couple months, so I figured I would mention it.

Lastly, the issue I wanted to bring up was how the execute_io_action could have inconsistent dosing.

To try and clarify, if you input self.execute_io_action(media_ml=2.0, media_alt_ml=2.0, waste_ml=4.0), the recursion transforms this into:
self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0.5, waste_ml=1.0) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5)

Because the command would be removing waste after every call to execute_io_action, it will not be removing media_ml and media_alt_ml equally.

This would be in contrast to inputting self.execute_io_action(media_ml=2.0, media_alt_ml=2.0, waste_ml=4.0) and having this behavior:

self.execute_io_action(media_ml=0.5, media_alt_ml=0.5, waste_ml=1.0) self.execute_io_action(media_ml=0.5, media_alt_ml=0.5, waste_ml=1.0) self.execute_io_action(media_ml=0.5, media_alt_ml=0.5, waste_ml=1.0) self.execute_io_action(media_ml=0.5, media_alt_ml=0.5, waste_ml=1.0)

Or, alternatively:

self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5) self.execute_io_action(media_ml=0.5, media_alt_ml=0, waste_ml=0.5) self.execute_io_action(media_ml=0, media_alt_ml=0.5, waste_ml=0.5)

Thanks again.

4

Yes, you’re right. Under the current algorithm, for example, if the vial’s current ratio is 0.5 between media and alt_media, then our execute_io_action with equal media and alt_media volumes won’t preserve that ratio. I’ve written a local test that demonstrates this.

For historical context: I’ve never thought of this use case because my applications were morbidostat-like operations, where one is adding small amounts of an alternative media with some killing agent in it. The ratio in the vial is important (and computed internally), but the ratio wasn’t what we were trying to control. We would just say “add more alt_media than last time”, and the ratio would increase (as expected).

I’ll spend time today modifying the algorithm to address this, and report back. It’ll be in the next release, which is coming soon!

5

Okay, so I’ve made a number of changes, improvements, and bug fixes. FYI if you want to try them out, you can run pio update --app -b develop on the command line to grab these changes, but this is my development branch, so it may have some bugs.¹

  1. execute_io_action should behave as expected. Specifically, it will add volumes of media and alt_media at ratio equal to the initial ratio each “batch”. Here’s a test with your custom class that tests the requested behaviour. Specifically these lines show that the ratio will be respected.

A number of other changes which you may find useful:

  1. We now track a dynamic vial_volume on the DosingAutomationJob class, so this is available to be used in automations.
  2. It’s possible to specify the initial volume in the vial, and initial fraction of alt media. For example, the below command will start a chemostat to dose 0.5ml / 2min, but all metrics are computed with the assumption that the initial volume in the vial is 0.
pio run dosing_control --automation-name chemostat --volume 0.5 --duration 2 --initial-vial-volume 0

Another example, suppose we start the experiment by pre mixing 7ml media and 7ml alt media, then start a chemostat:

pio run dosing_control --automation-name chemostat --volume 0.5 --duration 2 --initial-alt-media-fraction 0.5

(see config.ini below for another way to specify this globally).

None of these changes will affect your code, except that the execute_io_action will have the modified dosing algorithm.

¹ you’ll need to make an addition to your config.ini, too, adding the following:

[bioreactor]
max_volume_ml=14
initial_volume_ml=14
initial_alt_media_fraction=0

The existing volume_ml is deprecated.

6

Thanks! I really appreciate the help.