Running a large cluster

Using the Pioreactor
1

I’m setting up a fairly large pioreactor cluster for a course that I am running (16 reactors). I’ve got a small cluster running now (on pi zero 2W’s), and have noticed that with each added worker the setup gets increasingly laggy. I assume that a more powerful pi running as the leader may help. Is that the case? If so, would a 3B be sufficient for 16 pioreactors or should I look at a 4? And how much memory (2GB, 4GB, more?).

Also, the cluster documentation seems to suggest that you can have a dedicated pi for a leader (e.g. without the pioreactor hardware added). Is that a better option for larger clusters?

2

Hi Bryan,

(working on an email to you now btw)

Likely you’ll start to experience more lag as your cluser expands, especially over WIFI. Here are some ideas to improve performance (in no particular order).

  1. Offload the WIFI to a separate dedicated WIFI router, and connect all your Pioreactors to this router. I can supply a command you can run on each Pi to reconnect to this new network.

  2. Bumping up the leader to an RPi3 or RPi4 is a good idea, probably even worth it to buy a RPi4 (even an RPi5 if you really want to see performance improvements). I would say 2GB of RAM is fine. If you go for a RPi4 or RPi5, you have the option to connect a monitor + mouse + keyboard to the leader and control the cluster from there.

  3. A dedicated leader is a good option: we produce leader-only images that can be used: the pioreactor_leader.zip option here is what you are looking for. A dedicated leader means it can spend more time handling requests and networking vs spinning a fan, etc.

3

The pioreactors are on their own dedicated wifi network, and are connecting to it fine.

I think a dedicated leader on a pi 3B or 4 is probably a good option for us. I have a couple of questions about that option:

  1. If I get a 3B or 4, they have a LAN port - would I see a speed improvement if I plug the leader directly into the wireless router rather than connect via wifi? The workers would still connect to the router via wifi.

  2. If I run these as a leader-only, can they run as a stand-alone computer (e.g. in a case), or do they need to be installed into a pioreactor with the header board/etc?

4

Yes definitely, I should have suggested that as well.

They can be stand-alone. The hardware is only required when they are a worker.