I am trying to export the data from my experiment through the “export” menu, but every time I try, the zip file downloaded is empty. what could be the problem?
Hm, a bit odd. You are positive there is data for that experiment? Can you tell me what you dataset you are trying to download?
Actually I’m seeing the same problem. I’ll investigate.
Hey, if you managed to fix the problem I’ll be glad to hear how
Thank you
I think my problem was unrelated (we are actively changing the export code/logic, and my problem occurred in a development environment). Can you tell me more about which dataset you are exporting? Does this even happen if you choose “logs” (which should always have something).
I wonder if data is correctly being saved to your database. When you refresh the Overview page, do you see data on the charts disappear?
I have tried to export different kinds of datassets (Alt. media fraction, OD, growth rate) and none of them worked, not even the “logs”.
It also seems that the data is being saved correctly, even when I close the website and open it again - the charts are still avalible.
Hm, not sure then. What browser are you using? Can you try the same with “”? If it’s still the same problem, can you SSH into your leader, and try:
pio run export_experiment_data --tables logs
Does this succeed okay? If so, a zip file output.zip should have been created. Inspect it’s size using:
ls -lh output.zip
produces some like:
-rw-r--r-- 1 pioreactor pioreactor 857K Nov 28 10:59 output.zip
(so the zip file is 857 K - ideally not empty which is ~22 !)
Also, try updating to the latest 24.12.5. We redid the backend (and frontend) for exporting data, which may solve this problem.
thank you, after the update it seems to be fixed
Hey @CamDavidsonPilon,
i’ve ran into the same problem on 24.12.10. The graphs on the website are empty when the website is refreshed, until some points of measurement are gathered. I assumed that the scaling of the graphs changed with the last update. Is corruption of the SD-Card possible?
If i inspect output.zip via ls the size is 0. If I try the to export the data of the whole experiment, I get the following error message:
pio run export_experiment_data --experiment 111224Turbidostat_electroautotroph --dataset-name od_readings 2024-12-16T17:15:01+0100 INFO [export_experiment_data] Starting export of dataset: od_readings. 2024-12-16T17:15:01+0100 DEBUG [export_experiment_data] SELECT 1 FROM sqlite_master WHERE (type=‘table’ or type=‘view’) and name = ‘od_readings’ 2024-12-16T17:15:01+0100 DEBUG [export_experiment_data] SELECT *, datetime(timestamp, ‘localtime’) as timestamp_localtime FROM (od_readings) WHERE experiment IN (‘111224Turbidostat_electroautotroph’) ORDER BY ‘timestamp’ Traceback (most recent call last): File “/usr/local/bin/pio”, line 8, in sys.exit(pio()) ^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 1157, in call return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/click/core.py”, line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/usr/local/lib/python3.11/dist-packages/pioreactor/actions/leader/export_experiment_data.py”, line 275, in click_export_experiment_data export_experiment_data( File “/usr/local/lib/python3.11/dist-packages/pioreactor/actions/leader/export_experiment_data.py”, line 250, in export_experiment_data logger.debug(f"Exported {i} rows from {dataset_name}.") ^ UnboundLocalError: cannot access local variable ‘i’ where it is not associated with a value
It sounds like data isn’t being saved to disk! If you ssh into the leader Pioreactor, can you run:
ps x | grep mqtt
and does pio run mqtt_to_db_streaming show up?
If not, can you run
pio run mqtt_to_db_streaming
And see if that errors?
1 Like
@EBTPio try this solution: Graphs on Overview page clearing on refresh / Not able to export data with newest version - #5 by CamDavidsonPilon
Hello, I am having trouble with exporting again. perhaps you re-updated it? for some reason it fails to connect to the server, although it is connected to the pioreactors and I see the data in the “overview” section.
With other experiments I manage to export, so maybe it’s because this experiment is running for 5 days so it’s a lot of data.
With other experiments I manage to export, so maybe it’s because this experiment is running for 5 days so it’s a lot of data.
It’s possible… what datasets are you exporting? Some are much larger than others.