Did an export of my sql before transferring leader to a new Pi, so the original is now gone. Looking through the calibrations table and the calibrations don’t seem to have been updated from the first ones that were used, with UNIQUE (pioreactor_unit, type, name). Wouldn’t it be better for this table to keep a log of when calibrations were changed and what they were, like with the config log? Otherwise we also have to keep all of the calibration files as well to ensure proper logging. Or is that the intention? This also doesn’t include an extra calibration Ive added for my pH probe.
Related SQL question: How come alt_media_fractions and liquid_volumes just use (experiment) for the index not also pioreactor unit and timestamp like most others?
You’re asking the tough questions here!
How come alt_media_fractions and liquid_volumes just use (experiment) for the index not also pioreactor unit and timestamp like most others?
These are pretty small-sized tables (relatively), so we can keep the index more coarse. They also aren’t used in the UI (that data comes from MQTT), so we don’t need high performance from them. Very little harm in adding pioreactor and timestamp though.
Looking through the calibrations table and the calibrations don’t seem to have been updated from the first ones that were used
The calibrations table isn’t populated anymore. In ~Jan 2025, we moved to flat-file based calibrations, and we stopped sending the calibration data to the leader. It was too hard to keep track of (ex: a user edits a calibration file on a worker manually - how will the leader know?).
Maybe in the future we can reintroduce this though by exposing and encouraging a pio calibrations edit -device D --name N tool.
1 Like
Great, thanks! I did pgloader to migrate the sqlite db to a postgres db on google cloud sql but it didn’t move the indexes/keys etc so it’s helpfully made me look through everything in a bit more detail.
So to confirm for calibrations the only way currently to know what calibration was used for an experiment is to check the file names for the dates, assuming that they weren’t edited manually? When I calibrate my pH probe the logs show that a new file has been set as the active calibration, but this doesn’t seem to happen for the pumps.
"experiment","pioreactor_unit","timestamp","message","source","level","task"
"$experiment","pioreactor01","2025-10-21T16:35:35.93Z","Set pH-2025-10-21 as active calibration for ph_probe","app","INFO","calibrations"
"$experiment","pioreactor01","2025-10-21T16:35:33.271Z","Saved calibration pH-2025-10-21 to /home/pioreactor/.pioreactor/storage/calibrations/ph_probe/pH-2025-10-21.yaml","app","INFO","calibrations"
"$experiment","pioreactor01","2025-09-30T18:44:08.389Z","Starting pump calibration.","app","INFO","pump_calibration"
Couldn’t the yaml for calibration be sent over mqtt when it’s set as active like it is when the config file is updated?
If there’s a reason not to do that natively, I will probably write it into the plugin I’m writing for streaming MQTT to API. We can convert the calibration yamls and the config.ini to JSONB.
Hm, let me get back to you. It sounds like we do want a native solution.