Extremely slow pioreactors and UI

Using the Pioreactor
1

Hey,

After coming back from my holiday I am experiencing extremely slow responsiveness in the Pioreactors and online interface. Things like turning on stirring takes 2 minutes to have an effect. And it takes more than a minute for the webpage to correctly show pioreactor status.

Today we even experienced that giving several commands in succession resulted in the pios not performing some of them. Like not starting to calculate growth rate despite the interface saying they are.

We are using a setup with an external leader and 12 connected Pios. Leader is a Raspberry 5 - 8 GB.

Anybody experiencing similar issues or with any idea what might be going on here. Maybe recent updates or anything else?

Thanks!

2

Hi @Stkral, sorry about the troubles

It sounds like connection issues to MQTT, since commands to start jobs and pioreactor status all use MQTT.

Can you run a few tests for me?

  1. Is SSHing in to the leader Pioreactor still fast?
  2. Once SSHed in, can you try pio mqtt - does that respond quickly (sub 1 second), or take minutes? (ctrl-c to exit)

In your shared configuration, there’s the section [mqtt] and parameter broker_address. Can you try changing the broker_address to the leader’s IPv4 address? You can get that IPv4 address from the hostname -I on the leader’s command line (or the UI’s Inventory page, if it’s displayed there).

Save that config - does that improve the response time at all?

3
  1. SSH into leader is slow
  2. pio mqtt is fast

Changing the broker_address didnt change anything.

Thanks once again for helping out so quickly!

4

Changing the broker_address made me lose the connection to all Pios so I changed it back

5

Also, not sure if this is connected or not. But I realized that the normalized OD and growth rate are not being shown/calculated.

6

Hm, ssh into leader is slow eh… maybe it’s a network congestion issue? Are you using the local-access-point or your institution’s network?

  1. What if you ssh using the IP address: ssh pioreactor@<leader's ipv4 address> - is that fast?
  2. Try SSHing into a worker, and running a command like pio blink. Is that very slow?
7

We’re using an (external) local access point which in turn is connected to a mobile router (for updates etc).

When trying your first suggestion using the IP address given by the hostname -I command, I get the following output:

C:\Users\labuser>ssh pioreactor@192.168.1.35.local
ssh: Could not resolve hostname 192.168.1.35.local: No such host is known.

Pio Blink is also very slow

8

Sorry, don’t put the .local at the end:

ssh pioreactor@192.168.1.35