Questions developing a dosing automation

Software
,
1

Hi!

I have been developing a dosing automation for our Pioreactors, and have come up with some questions about software and plugins. Also, I tested my code last week and it did not work, so I would be super grateful if I could get more input into these topics to figure out how to write the code correctly.

  • Each worker has their own config_.ini file, is it normal that they are empty? and only the leader config file is not?

  • What is the most common way to run a customized dosing automation: as a one-time run, as a background job, have it as a plugin and run it from the CL, as a plugin run from the UI? Or how do I know which is the best way to run it for our setup?

  • To run my script as a background job, I found that I can use either of these next commands, but the nohup command preserves the output in a file. Is this correct? and are these the correct commands?

python your_script.py >/dev/null 2>&1 & disown
nohup python3 your_script.py &

  • How do you stop a background job? just shuting down the pioreactor? or some other way?

  • Can I restart the automation if it was already running?

I was checking the output file to see if the automation was running correctly, by writing cat nohup.out, and the error messages were not clear to me. Sometimes it said that some input was missing (which was not the case), other times some syntax errors, which I corrected in the file directly in the CL and saved it, but it seemed like the pioreactor was trying to run some older version of the .py file instead of the latest one. Sometimes it said some errors mentioning “exiting”, so I did not know if the automation was already running and I had force it to start again and instead the initial run was just stopped.

  • So would it be better to stop the background job and then restart it again? Or what is recommended here?
  • Does the output file get overwritten every time the automation is run? or should i delete it so that a new version of it can be created?

At the moment I do not know how to correct my .py file so that the automation works correctly, since I dont really understand the last error messages I was getting in the nohup.out file. Would it be better to make a .yaml file and upload it to the UI as a plugin, so the errors can be handled easier? or that is not really the case? Thought maybe having it in the UI would make it easier to see if the automation is at least running, for example, or how it gives back the errors encountered.

Thank you very much for the help! :smile:

2

Hi @aiseor, I’m happy to help:

  • configuration is shared between all workers and leader: this is the config.ini file (which is what you may be referring to as the leader config file - it’s not the leader’s config, it’s everyones default config).

    Screenshot 2024-02-26 at 4.20.23 PM

    However, sometimes you want custom configuration for just a single worker - that’s why there are worker-specific configuration files. Note that you only need to add changes to the worker config files, not the entire configuration. See example here.

  • What’s the most common way to run a dosing automation? Here’s how I would do it:

  1. Add the Python file with your automation to ~/.pioreactor/plugins.
  2. Next, I want to test it. From the command line, run the following:
    pio run dosing_control --automation-name my_name --possible-arg 1 --possible-arg 2
    For example, the chemostat automation would look like:
    pio run dosing_control --automation-name chemostat --volume 1.5 --duration 30
    1. Make any changes / fixes based on the output of ^. Once it works, (eg: no errors, output looks generally correct, etc), then I would add the YAML file for it under ~/.pioreactor/plugins/ui/contrib/automations/dosing/. See here for the documentation. This YAML only needs to go on the leader Pioreactor btw.
    2. Finally, you can distribute that Python file to all other workers with:
    pios cp ~/.pioreactor/plugins/my_automation.py

  • To run a script in the background, yes, both work. However the >/dev/null 2>&1 part of the first command will clobber any output. So remove that. However, I don’t suggest you run automations like this. You should use the code / UI (see above answer).

  • How do you stop a background job? If it’s a job in the UI, just turn if off from there. Otherwise, you can run pio kill <job name>. So to stop the a dosing automation, you kill the dosing_control (since that controls automations). Ex: pio kill dosing_control.

  • Can you restart the automation if it was already running? If you changed the code, I suggest stopping (killing) the dosing_contol job, and restarting it. Generally, when you start a job, it puts the code from disk into memory, and hence any changes to files on disk (including .py or config changes), don’t get updated in the running code.

  • Does the output file get overwritten every time the automation is run? Again, my suggestion is to avoid using nohup in your workflow, as it will just cause you to develop more slowly.

I’d be happy to review your code / automation here, or privately at cam@pioreactor.com. I’m also happy to jump on a video call and we can talk live :slight_smile: