Here’s an outline of a few ways realtime could be done:
Using MQTT (no Python, but lots of networking)
The Pioreactor uses Mosquitto, a software that implements MQTT, for realtime data streams. A Mosquitto broker lives on the leader in each Pioreactor cluster.
It’s possible to define a “bridge” between Mosquitto brokers so that they can share messages. In this case, we just want a one way stream between all users and a central Mosquitto broker. On the central server with the central broker, we can just write those messages to disk (or duplicate what we do in the Pioreactor, and write to tables in SQLite3).
The central server needs to be accessible, which suggests a public API (with credentials), but you could also use a project like Tailscale to create a private VPN with users’ leaders.
Using plugins (more Python, but less networking)
You can write a Python script that
- listens to all topics in MQTT, and
- serializes the topic and message and pushes it to an API on some central server.
The server is set up as a webserver and knows how to handle incoming requests (writes to disk, saves to a db, etc).
This script can be distributed to users to run in the background. More formally, the Python script could be a proper Pioreactor plugin if you wanted, too.