I’d like to install a plugin from source but with dependencies being installed via PyPI. I think perhaps the easiest way to do this is to provide an index-url in the pip3 install:
sudo pip3 install --force-reinstall --no-index "$source" to
Currently the pio plugins install predict-fluorescence --source predict_fluorescence-1.0.0-py3-none-any.whl fails:
2024-11-14T13:59:13+0000 DEBUG [install_plugin] Installing plugin predict-fluorescence.
2024-11-14T13:59:13+0000 DEBUG [install_plugin] bash /usr/local/bin/install_pioreactor_plugin.sh predict-fluorescence predict_fluorescence-1.0.0-py3-none-any.whl
2024-11-14T13:59:14+0000 ERROR [install_plugin] Failed to install plugin predict-fluorescence. See logs.
2024-11-14T13:59:14+0000 DEBUG [install_plugin] b'Processing ./predict_fluorescence-1.0.0-py3-none-any.whl\nINFO: pip is looking at multiple versions of predict-fluorescence to determine which version is compatible with other requirements. This could take a while.\n'
2024-11-14T13:59:14+0000 DEBUG [install_plugin] b"+ export LC_ALL=C\n+ LC_ALL=C\n+ plugin_name=predict-fluorescence\n+ source=predict_fluorescence-1.0.0-py3-none-any.whl\n+ clean_plugin_name=predict-fluorescence\n+ clean_plugin_name_with_dashes=predict-fluorescence\n+ clean_plugin_name_with_underscores=predict_fluorescence\n++ python3 -c 'import site; print(site.getsitepackages()[0])'\n+ install_folder=/usr/local/lib/python3.11/dist-packages/predict_fluorescence\n++ crudini --get /home/pioreactor/.pioreactor/config.ini cluster.topology leader_hostname\n+ leader_hostname=lead1\n++ hostname\n+ '[' lead1 = lead1 ']'\n+ am_i_leader=true\n+ '[' -n predict_fluorescence-1.0.0-py3-none-any.whl ']'\n+ sudo pip3 install --force-reinstall --no-index predict_fluorescence-1.0.0-py3-none-any.whl\nERROR: Could not find a version that satisfies the requirement schedule==1.2.2 (from predict-fluorescence) (from versions: none)\nERROR: No matching distribution found for schedule==1.2.2\n"
Traceback (most recent call last):
File "/usr/local/bin/pio", line 8, in <module>
sys.exit(pio())
^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/pioreactor/plugin_management/install_plugin.py", line 44, in click_install_plugin
install_plugin(name_of_plugin, source)
File "/usr/local/lib/python3.11/dist-packages/pioreactor/plugin_management/install_plugin.py", line 33, in install_plugin
raise BashScriptError(f"Failed to install plugin {name_of_plugin}. See logs.")
pioreactor.exc.BashScriptError: Failed to install plugin predict-fluorescence. See logs.
It’s been a while since I looked at this bash script. I’ll take a fresh look make improvements, including handling your use case.
Unrelated to the main issue, but the pioreactor software has a similar function to what scheduale provides. See pioreactor.utils.timing.RepeatedTimer - but don’t let this stop you from progressing!
PS I have recreated the shell script but without the --no-index and without the broadcasting to the cluster (it’s a leader-only background job). The graph appears on the Overview page, the buttons appear on the “Manage Pioreactors” card and the new table appears in the sqlite database but I can’t get the job to run i.e. if I click on the “Start” button on the “Manage Pioreactors” card, it doesn’t start and if I do pio run --help on the command line, it is not one of the jobs that appears. If I run python predict_fluorescence.py in the plugins folder, it runs fine (publishing topics to the mqtt). I have rebooted the leader and the cluster. Do you have any tips?
From the run function, I can see that I need a click command. I have this in my background job:
NB I’ve put pio01 even though it’s leader only - the leader has the leader-only image and so it didn’t recognise the leader as part of the cluster. The job needs historical data and so queries the sqlite database for all pioreactors and publishes them to separate MQTT topics.
Here is the shell script:
#!/bin/bash
# Check if the package name is provided
if [ -z "$1" ]; then
echo "Error: Package name not provided."
echo "Usage: ./install_on_lead.sh <PACKAGE_DIR> <PACKAGE_UNDERSCORE>"
exit 1
fi
PACKAGE_DIR="$1"
PACKAGE_UNDERSCORE="$2"
# Define the plugin directory
PLUGIN_DIR="/home/pioreactor/.pioreactor/plugins"
# Navigate to the dist directory of the package
cd "$PLUGIN_DIR/$PACKAGE_DIR/dist" || {
echo "Error: Directory $PLUGIN_DIR/$PACKAGE_DIR/dist does not exist."
exit 1
}
echo "Changed to directory: $(pwd)"
# Find the wheel file
WHEEL_FILE=$(ls *.whl | tail -n 1)
echo "WHEEL_FILE is $WHEEL_FILE"
if [ -n "$WHEEL_FILE" ]; then
echo "Installing $WHEEL_FILE"
pip3 install "$WHEEL_FILE" --force-reinstall --ignore-installed --index-url https://pypi.org/simple/ || true
echo "Installation complete, proceeding to check for LEADER_ONLY file"
# Check if LEADER_ONLY file exists
if [ -f "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/LEADER_ONLY" ]; then
echo "LEADER_ONLY file exists"
# Merge new config.ini if it exists
if [ -f "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/additional_config.ini" ]; then
echo "Merging additional config"
crudini --merge /home/pioreactor/.pioreactor/config.ini < "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/additional_config.ini"
fi
# Add any new SQL and restart mqtt_to_db job if SQL exists
if [ -f "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/additional_sql.sql" ]; then
echo "Adding additional SQL and restarting service"
sqlite3 "$(crudini --get /home/pioreactor/.pioreactor/config.ini storage database)" < "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/additional_sql.sql"
sudo systemctl restart pioreactor_startup_run@mqtt_to_db_streaming.service
fi
# Merge UI contributions if contrib directory exists
if [ -d "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/ui/contrib/" ]; then
echo "Merging UI contributions"
rsync -a "$PLUGIN_DIR/$PACKAGE_DIR/$PACKAGE_UNDERSCORE/ui/contrib/" /home/pioreactor/.pioreactor/plugins/ui/contrib/
fi
else
echo "LEADER_ONLY file does not exist"
fi
else
echo "No .whl file found for installation"
fi
Hm, if you execute pio run --help, is there an error message at the top of the output? Any errors when a plugin loads are captured and spit out with pio x is run.
Otherwise, that looks okay to me. Can you share the entire script?
Hi Cameron, thanks for taking a look. When running pio run --help, I get the error:
pio_data_daemon plugin load error: No module named 'pio_data_daemon'
And when I run pio plugins list, I get a similar error:
pio_data_daemon plugin load error: No module named 'pio_data_daemon' pio_data_daemon plugin load error: No module named 'pio_data_daemon'
pio_data_daemon is another plugin we’re working on. I’ve deleted the directory from the .pioreactor/plugins directory and tried pio plugins uninstall pio_data_daemon (which errors as it can’t find it) followed by a reboot but it looks like it’s not uninstalling properly… This might be the issue but the other installed plugins show up in the pio run --help as subcommands (and I’m reasonably sure it was a problem before my colleague started working on the other plugin).
I’ll email across the package and the shell script to you too in case there’s anything I’m doing that’s unexpected. Many thanks!
we’ve run pip uninstall pio-data-daemon and the errors I mentioned above have disappeared
we tried testing installing a community plugin log2discord to try to work out whether it was something specific with how we’re setting up plugins. We installed it via the UI, rebooted and that one doesn’t appear in the pio plugins list either.
Will keep trying a few things and keep you updated here if we see anything different. Thanks!
