Rebuild Combo table of detections manually #254

petri-o-ojala · 2022-05-08T16:54:21Z

petri-o-ojala
May 8, 2022

I just installed BirdNET-Pi a week ago so new to the game.. great work Patrick and contributors, we moved to more countryside last summer and been enjoying the birds around without really knowing anything about the topic until now :-)

For some reason the detection has stopped during the last few days, when restarted, it does start to analyze the older recordings and update the database accordingly. I haven't yet looked what exactly is going on.

Is there a script somewhere that one could run to update yesterday's Combo table of detections? I saw in the logs that it did go through detection for them but the chart didn't get updated for yesterday, only today. daily_plot.py seems to be based on current timestamp or is there a way to run it for a specific date somehow?

CaiusX · 2022-05-08T19:15:16Z

CaiusX
May 8, 2022
Collaborator

hi @petri-ojala-tieto

Easiest way is to open the daily_plot.py in Thonny, save as another name (daily_plot_date.py say and change line 35 from

now.strftime("%Y-%m-%d")

to

"2022-05-07"

or whatever date you need to re-run

and re-run.

If the libraries can't be imported, then make sure the birdnet virtual environment is loaded.

1 reply

mcguirepr89 May 8, 2022
Maintainer

Or, without needing to install the Raspberry Pi desktop to install Thonny, you can use "Tools" > "File Manager" and then locate the script in ~/BirdNET-Pi/scripts/daily_plot.py and edit it straight from BirdNET-Pi.

If you encounter any permissions errors when trying save changes to daily_ploy.py using that method, you can run this in the "Tools" > "Web Terminal" and then try again:

sudo chown -R $USER:$USER ~/Bird*
sudo chmod -R g+wr ~/Bird*

petri-o-ojala · 2022-05-08T20:11:26Z

petri-o-ojala
May 8, 2022
Author

Thanks, had to google what Thonny is :-) I haven't done python for several years now, switched to golang for pretty much everything.

ssh to the RasPi and vi is a good IDE.. but looks like it needs a bit more changes, the SELECT from the database and savename for the filenames (Combo and Combo2) are also using now directly. Anyway, that's fine, I'll make a new version of the script with relevant changes.

1 reply

CaiusX May 9, 2022
Collaborator

Hi @petri-ojala-tieto

Hope you got this sorted out?

Good to have a golang coder onboard - do you have any ideas where it could be better than python/bash scripts?

petri-o-ojala · 2022-05-09T06:13:28Z

petri-o-ojala
May 9, 2022
Author

As it failed again last night, looked a bit at the log files:

May  8 23:21:29 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3192.
May  8 23:21:29 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:21:29 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:21:29 birdnet spectrogram.sh[538387]: sox WARN wav: Premature EOF on .wav input file
May  8 23:21:29 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May  8 23:21:33 birdnet birdnet_analysis.sh[538236]: analyze.py --i /home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:16.wav --o /home/ojala/BirdSongs/May-2022/08-Sunday/2022
-05-08-birdnet-23:21:16.wav.csv --lat 60.3708 --lon 25.2787 --week 18 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7 --birdweather_id "IN_USE"
May  8 23:21:33 birdnet server.py[394]: ANALYZING AUDIO... DONE! Time 1.5 SECONDS
May  8 23:21:33 birdnet server.py[394]: WRITING RESULTS TO /home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:01.wav.csv ... DONE! WROTE 0 RESULTS.
May  8 23:21:33 birdnet server.py[394]: [NEW CONNECTION] ('127.0.0.1', 46380) connected.
May  8 23:21:33 birdnet server.py[394]: [ACTIVE CONNECTIONS] 1
May  8 23:21:33 birdnet server.py[394]: READING AUDIO DATA... DONE! READ 5 CHUNKS.
May  8 23:21:33 birdnet server.py[394]: FULL FILENAME: -/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:16.wav-
May  8 23:21:33 birdnet server.py[394]: Date-time: 2022-05-08 23:21:16
May  8 23:21:35 birdnet birdnet_analysis.sh[538438]: 0.0;3.0-('Botaurus stellaris_Kaulushaikara', 0.066880025)
May  8 23:21:35 birdnet birdnet_analysis.sh[538438]: 3.0;6.0-('Botaurus stellaris_Kaulushaikara', 0.14644822)
May  8 23:21:35 birdnet birdnet_analysis.sh[538438]: 6.0;9.0-('Dryocopus martius_Palokärki', 0.04091612)
May  8 23:21:35 birdnet birdnet_analysis.sh[538438]: 9.0;12.0-('Botaurus stellaris_Kaulushaikara', 0.038738806)
May  8 23:21:35 birdnet birdnet_analysis.sh[538438]: 12.0;15.0-('Circus macrourus_Arosuohaukka', 0.0464693)
May  8 23:21:35 birdnet systemd[1]: birdnet_analysis.service: Succeeded.
May  8 23:21:37 birdnet systemd[1]: birdnet_analysis.service: Scheduled restart job, restart counter is at 1144.
May  8 23:21:37 birdnet systemd[1]: Stopped BirdNET Analysis.
May  8 23:21:37 birdnet systemd[1]: Started BirdNET Analysis.
May  8 23:21:38 birdnet birdnet_analysis.sh[538482]: Files loaded
May  8 23:21:39 birdnet birdnet_analysis.sh[538482]: RECORDING_LENGTH set to 15
May  8 23:21:39 birdnet env[538489]: Line = 2022-05-08-birdnet-23:21:16.wav.csv
May  8 23:21:39 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3193.
May  8 23:21:39 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:21:39 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:21:39 birdnet spectrogram.sh[538582]: sox WARN wav: Premature EOF on .wav input file
May  8 23:21:39 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May  8 23:21:50 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3194.
May  8 23:21:50 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:21:50 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:21:50 birdnet spectrogram.sh[538686]: sox WARN wav: Premature EOF on .wav input file
May  8 23:21:58 birdnet birdnet_analysis.sh[538482]: analyze.py --i /home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav --o /home/ojala/BirdSongs/May-2022/08-Sunday/2022
-05-08-birdnet-23:21:31.wav.csv --lat 60.3708 --lon 25.2787 --week 18 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7 --birdweather_id "IN_USE"
May  8 23:21:58 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May  8 23:21:58 birdnet server.py[394]: ANALYZING AUDIO... DONE! Time 1.6 SECONDS
May  8 23:21:58 birdnet server.py[394]: WRITING RESULTS TO /home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:16.wav.csv ... DONE! WROTE 0 RESULTS.
May  8 23:21:58 birdnet server.py[394]: [NEW CONNECTION] ('127.0.0.1', 46382) connected.
May  8 23:21:58 birdnet server.py[394]: [ACTIVE CONNECTIONS] 1
May  8 23:21:58 birdnet server.py[394]: READING AUDIO DATA... /home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/util/decorators.py:88: UserWarning: PySoundFile failed. Tryin
g audioread instead.
May  8 23:21:58 birdnet server.py[394]:   return f(*args, **kwargs)
May  8 23:21:59 birdnet server.py[394]: Exception in thread Thread-8291:
May  8 23:21:59 birdnet server.py[394]: Traceback (most recent call last):
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 155, in load
May  8 23:21:59 birdnet server.py[394]:     context = sf.SoundFile(path)
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 629, in __init__
May  8 23:21:59 birdnet server.py[394]:     self._file = self._open(file, mode_int, closefd)
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 1183, in _open
May  8 23:21:59 birdnet server.py[394]:     _error_check(_snd.sf_error(file_ptr),
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 1357, in _error_check
May  8 23:21:59 birdnet server.py[394]:     raise RuntimeError(prefix + _ffi.string(err_str).decode('utf-8', 'replace'))
May  8 23:21:59 birdnet server.py[394]: RuntimeError: Error opening '/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': System error.
May  8 23:21:59 birdnet server.py[394]: During handling of the above exception, another exception occurred:
May  8 23:21:59 birdnet server.py[394]: Traceback (most recent call last):
May  8 23:21:59 birdnet server.py[394]:   File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
May  8 23:21:59 birdnet server.py[394]:     self.run()
May  8 23:21:59 birdnet server.py[394]:   File "/usr/lib/python3.9/threading.py", line 892, in run
May  8 23:21:59 birdnet server.py[394]:     self._target(*self._args, **self._kwargs)
May  8 23:21:59 birdnet server.py[394]:   File "/usr/local/bin/server.py", line 293, in handle_client
May  8 23:21:59 birdnet server.py[394]:     audioData = readAudioData(args.i, args.overlap)
May  8 23:21:59 birdnet server.py[394]:   File "/usr/local/bin/server.py", line 123, in readAudioData
May  8 23:21:59 birdnet server.py[394]:     sig, rate = librosa.load(path, sr=sample_rate, mono=True, res_type='kaiser_fast')
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/util/decorators.py", line 88, in inner_f
May  8 23:21:59 birdnet server.py[394]:     return f(*args, **kwargs)
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 174, in load
May  8 23:21:59 birdnet server.py[394]:     y, sr_native = __audioread_load(path, offset, duration, dtype)
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 198, in __audioread_load
May  8 23:21:59 birdnet server.py[394]:     with audioread.audio_open(path) as input_file:
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/__init__.py", line 111, in audio_open
May  8 23:21:59 birdnet server.py[394]:     return BackendClass(path)
May  8 23:21:59 birdnet server.py[394]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/rawread.py", line 62, in __init__
May  8 23:21:59 birdnet server.py[394]:     self._fh = open(filename, 'rb')
May  8 23:21:59 birdnet server.py[394]: FileNotFoundError: [Errno 2] No such file or directory: '/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav'
May  8 23:22:08 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3195.
May  8 23:22:08 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:08 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:08 birdnet spectrogram.sh[538835]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:08 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:08 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:22:18 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3196.
May  8 23:22:18 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:18 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:18 birdnet spectrogram.sh[538964]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:18 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:18 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:22:28 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3197.
May  8 23:22:28 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:28 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:28 birdnet spectrogram.sh[539100]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:28 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:28 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:22:39 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3198.
May  8 23:22:39 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:39 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:39 birdnet spectrogram.sh[539192]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:39 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:39 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:22:42 birdnet systemd[1]: chart_viewer.service: Scheduled restart job, restart counter is at 247.
May  8 23:22:42 birdnet systemd[1]: Stopped BirdNET-Pi Chart Viewer Service.
May  8 23:22:42 birdnet systemd[1]: chart_viewer.service: Consumed 13.648s CPU time.
May  8 23:22:42 birdnet systemd[1]: Started BirdNET-Pi Chart Viewer Service.
May  8 23:22:49 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3199.
May  8 23:22:49 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:49 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:49 birdnet spectrogram.sh[539324]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:49 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:49 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:22:54 birdnet systemd[1]: chart_viewer.service: Succeeded.
May  8 23:22:54 birdnet systemd[1]: chart_viewer.service: Consumed 12.695s CPU time.
May  8 23:22:59 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3200.
May  8 23:22:59 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May  8 23:22:59 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May  8 23:22:59 birdnet spectrogram.sh[539453]: sox FAIL formats: can't open input file `/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': No such file or directory
May  8 23:22:59 birdnet systemd[1]: spectrogram_viewer.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
May  8 23:22:59 birdnet systemd[1]: spectrogram_viewer.service: Failed with result 'exit-code'.
May  8 23:23:09 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 3201.
May  8 23:23:09 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.

Recovers only after doing restart all services.

I added a test to the spectrogram.sh script to run sox only if the file exists as a quick workaround. Not sure if that alone is going to help or there's something else failing in birdnet_analysis.sh as well nor yet any idea what's the root cause.. (not sure if there are potential TZ issues in the code but running the Pi in EEST)

5 replies

mcguirepr89 May 9, 2022
Maintainer

May 8 23:21:58 birdnet server.py[394]: READING AUDIO DATA... /home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/util/decorators.py:88: UserWarning: PySoundFile failed. Trying audioread instead.

May 8 23:21:59 birdnet server.py[394]: RuntimeError: Error opening '/home/ojala/BirdSongs/May-2022/08-Sunday/2022-05-08-birdnet-23:21:31.wav': System error.

These errors are totally new to me 🤔

Please do update (twice) to ensure you're running the most recent scripts so we can be sure we're troubleshooting a new issue: "Tools" > "System Controls" > "Update"

Thank you!

petri-o-ojala May 10, 2022
Author

Yes, something odd going on (the configuration is pretty much "as-is", haven't done anything special on the Pi yet..). My workaround just moved the issue to server.py last night so it's in a loop complaining about the missing file.

Code was probably up-to-date but did run update a few times,

Already up to date.
On branch main
Your branch is up to date with 'origin/main'.

as of right now.

I'll look at the log files later today / tomorrow morning to see what's happens. BTW, is there a diagram of the script / server execution, what happens and where yet? Takes some effort to figure out all the logic.

petri-o-ojala May 12, 2022
Author

On the web interface, it pretty much died the other night due to the missing file issue.

May 10 21:08:22 server.py[2211103]:     with audioread.audio_open(path) as input_file:
May 10 21:08:22 server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/__init__.py", line 111, in audio_open
May 10 21:08:22 server.py[2211103]:     return BackendClass(path)
May 10 21:08:22 server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/rawread.py", line 62, in __init__
May 10 21:08:22 server.py[2211103]:     self._fh = open(filename, 'rb')
May 10 21:08:22 server.py[2211103]: FileNotFoundError: [Errno 2] No such file or directory: '/home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:46.wav'
May 11 00:00:01 env[2804059]: Removing old directories
May 11 00:00:01 env[2804059]: Removing old directories
00:00:00 env[3544297]: Removing old directories
00:00:00 env[3544297]: Removing old directories

daemon.log is showing

May 10 21:07:46 birdnet birdnet_analysis.sh[2677151]: analyze.py --i /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:31.wav --o /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-bir
dnet-21:07:31.wav.csv --lat 60.3708 --lon 25.2787 --week 18 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7 --birdweather_id "IN_USE"
May 10 21:07:46 birdnet server.py[2211103]: ANALYZING AUDIO... DONE! Time 1.5 SECONDS
May 10 21:07:46 birdnet server.py[2211103]: WRITING RESULTS TO /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:16.wav.csv ... DONE! WROTE 0 RESULTS.
May 10 21:07:46 birdnet server.py[2211103]: [NEW CONNECTION] ('127.0.0.1', 57688) connected.
May 10 21:07:46 birdnet server.py[2211103]: [ACTIVE CONNECTIONS] 1
May 10 21:07:46 birdnet server.py[2211103]: READING AUDIO DATA... DONE! READ 5 CHUNKS.
May 10 21:07:46 birdnet server.py[2211103]: FULL FILENAME: -/home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:31.wav-
May 10 21:07:46 birdnet server.py[2211103]: Date-time: 2022-05-10 21:07:31
May 10 21:07:48 birdnet birdnet_analysis.sh[2677361]: 0.0;3.0-('Botaurus stellaris_Kaulushaikara', 0.057368323)
May 10 21:07:48 birdnet birdnet_analysis.sh[2677361]: 3.0;6.0-('Picoides tridactylus_Pohjantikka', 0.038595065)
May 10 21:07:48 birdnet birdnet_analysis.sh[2677361]: 6.0;9.0-('Scolopax rusticola_Lehtokurppa', 0.04885064)
May 10 21:07:48 birdnet birdnet_analysis.sh[2677361]: 9.0;12.0-('Passer montanus_Pikkuvarpunen', 0.117073886)
May 10 21:07:48 birdnet birdnet_analysis.sh[2677361]: 12.0;15.0-('Motacilla alba_Västäräkki', 0.04219253)
May 10 21:07:48 birdnet systemd[1]: birdnet_analysis.service: Succeeded.
May 10 21:07:49 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41167.
May 10 21:07:49 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:07:49 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:07:49 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:07:50 birdnet systemd[1]: birdnet_analysis.service: Scheduled restart job, restart counter is at 1332.
May 10 21:07:50 birdnet systemd[1]: Stopped BirdNET Analysis.
May 10 21:07:50 birdnet systemd[1]: Started BirdNET Analysis.
May 10 21:07:50 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 19276.
May 10 21:07:50 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May 10 21:07:50 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May 10 21:07:51 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May 10 21:07:51 birdnet birdnet_analysis.sh[2677406]: Files loaded
May 10 21:07:52 birdnet birdnet_analysis.sh[2677406]: RECORDING_LENGTH set to 15
May 10 21:07:53 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41168.
May 10 21:07:53 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:07:53 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:07:53 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:07:54 birdnet env[2677464]: Line = 2022-05-10-birdnet-21:07:31.wav.csv
May 10 21:07:56 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41169.
May 10 21:07:56 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:07:56 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:07:56 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:07:59 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41170.
May 10 21:08:20 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:20 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:20 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 19277.
May 10 21:08:20 birdnet spectrogram.sh[2677569]: sox WARN wav: Premature EOF on .wav input file
May 10 21:08:20 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May 10 21:08:20 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May 10 21:08:20 birdnet birdnet_analysis.sh[2677406]: analyze.py --i /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:46.wav --o /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-bir
dnet-21:07:46.wav.csv --lat 60.3708 --lon 25.2787 --week 18 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7 --birdweather_id "IN_USE"
May 10 21:08:20 birdnet server.py[2211103]: ANALYZING AUDIO... DONE! Time 1.5 SECONDS
May 10 21:08:20 birdnet server.py[2211103]: WRITING RESULTS TO /home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:31.wav.csv ... DONE! WROTE 0 RESULTS.
May 10 21:08:20 birdnet server.py[2211103]: [NEW CONNECTION] ('127.0.0.1', 57690) connected.
May 10 21:08:20 birdnet server.py[2211103]: [ACTIVE CONNECTIONS] 1
May 10 21:08:21 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May 10 21:08:21 birdnet server.py[2211103]: READING AUDIO DATA... /home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/util/decorators.py:88: UserWarning: PySoundFile failed. Trying audiore
ad instead.
May 10 21:08:21 birdnet server.py[2211103]:   return f(*args, **kwargs)
May 10 21:08:21 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:22 birdnet server.py[2211103]: Exception in thread Thread-4544:
May 10 21:08:22 birdnet server.py[2211103]: Traceback (most recent call last):
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 155, in load
May 10 21:08:22 birdnet server.py[2211103]:     context = sf.SoundFile(path)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 629, in __init__
May 10 21:08:22 birdnet server.py[2211103]:     self._file = self._open(file, mode_int, closefd)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 1183, in _open
May 10 21:08:22 birdnet server.py[2211103]:     _error_check(_snd.sf_error(file_ptr),
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/soundfile.py", line 1357, in _error_check
May 10 21:08:22 birdnet server.py[2211103]:     raise RuntimeError(prefix + _ffi.string(err_str).decode('utf-8', 'replace'))
May 10 21:08:22 birdnet server.py[2211103]: RuntimeError: Error opening '/home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:46.wav': System error.
May 10 21:08:22 birdnet server.py[2211103]: During handling of the above exception, another exception occurred:
May 10 21:08:22 birdnet server.py[2211103]: Traceback (most recent call last):
May 10 21:08:22 birdnet server.py[2211103]:   File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
May 10 21:08:22 birdnet server.py[2211103]:     self.run()
May 10 21:08:22 birdnet server.py[2211103]:   File "/usr/lib/python3.9/threading.py", line 892, in run
May 10 21:08:22 birdnet server.py[2211103]:     self._target(*self._args, **self._kwargs)
May 10 21:08:22 birdnet server.py[2211103]:   File "/usr/local/bin/server.py", line 293, in handle_client
May 10 21:08:22 birdnet server.py[2211103]:     args.min_conf = float(inputvars[1])
May 10 21:08:22 birdnet server.py[2211103]:   File "/usr/local/bin/server.py", line 123, in readAudioData
May 10 21:08:22 birdnet server.py[2211103]:     # Open file with librosa (uses ffmpeg or libav)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/util/decorators.py", line 88, in inner_f
May 10 21:08:22 birdnet server.py[2211103]:     return f(*args, **kwargs)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 174, in load
May 10 21:08:22 birdnet server.py[2211103]:     y, sr_native = __audioread_load(path, offset, duration, dtype)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/librosa/core/audio.py", line 198, in __audioread_load
May 10 21:08:22 birdnet server.py[2211103]:     with audioread.audio_open(path) as input_file:
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/__init__.py", line 111, in audio_open
May 10 21:08:22 birdnet server.py[2211103]:     return BackendClass(path)
May 10 21:08:22 birdnet server.py[2211103]:   File "/home/ojala/BirdNET-Pi/birdnet/lib/python3.9/site-packages/audioread/rawread.py", line 62, in __init__
May 10 21:08:22 birdnet server.py[2211103]:     self._fh = open(filename, 'rb')
May 10 21:08:22 birdnet server.py[2211103]: FileNotFoundError: [Errno 2] No such file or directory: '/home/ojala/BirdSongs/May-2022/10-Tuesday/2022-05-10-birdnet-21:07:46.wav'
May 10 21:08:24 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41171.
May 10 21:08:24 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:24 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:24 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:27 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41172.
May 10 21:08:27 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:27 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:27 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:30 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41173.
May 10 21:08:30 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:30 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:30 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:31 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 19278.
May 10 21:08:31 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May 10 21:08:31 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May 10 21:08:31 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.
May 10 21:08:34 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41174.
May 10 21:08:34 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:34 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:34 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:37 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41175.
May 10 21:08:37 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:37 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:37 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:40 birdnet systemd[1]: pushed_notifications.service: Scheduled restart job, restart counter is at 41176.
May 10 21:08:40 birdnet systemd[1]: Stopped BirdNET-Pi Pushed.co Notifications.
May 10 21:08:40 birdnet systemd[1]: Started BirdNET-Pi Pushed.co Notifications.
May 10 21:08:40 birdnet systemd[1]: pushed_notifications.service: Succeeded.
May 10 21:08:41 birdnet systemd[1]: spectrogram_viewer.service: Scheduled restart job, restart counter is at 19279.
May 10 21:08:41 birdnet systemd[1]: Stopped BirdNET-Pi Spectrogram Viewer.
May 10 21:08:41 birdnet systemd[1]: Started BirdNET-Pi Spectrogram Viewer.
May 10 21:08:41 birdnet systemd[1]: spectrogram_viewer.service: Succeeded.

And so on.. so my quick fix removed the issue from the script that kept looping and now the same failure happens on server.py with more fatal results.

Restarted all the systemd daemons and now it's processing recordings from yesterday. The database will be fine for the recordings on the disk but the scripts won't update the detection charts etc. (which was my initial concern).

ehpersonal38 May 12, 2022
Collaborator

@petri-ojala-tieto Good news, this issue was fixed by Patrick yesterday! #266 (comment)

Go ahead and "Tools" > "System Controls" > "Update", and you should be on your way.

mcguirepr89 May 12, 2022
Maintainer

Hi, @petri-ojala-tieto

I'm sorry you're having trouble retroactively creating the daily charts.
I will address this for you if you would please follow the steps below:
I apologize that these steps are silly -- this is a totally new request for which there is no quick remedy in place. With that said, however, all of the data that can be viewed on the daily charts can be seen in greater detail using the "Species Stats" page and selecting the particular date-range of interest.

Go to "Tools" > "Web Terminal" and paste the following:

sudo -E chown -R $USER:$USER ~/Bird*
sudo -E chmod -R g+wr ~/Bird*

Next, go to "Tools" > "File Manager"
In the file manager, navigate to the daily_plot.py (Location: ~/BirdNET-Pi/scripts/daily_plot.py).
Select the file and then choose the "Advanced Editor"

Change line 16 from this:

cursor.execute('SELECT * FROM detections WHERE Date = DATE(\'now\', \'localtime\')')

to this:

cursor.execute('SELECT * FROM detections')

Then go back to "Tools" > "Web Terminal" and run this command, replacing the date with the date for which you wish to create new charts:
```
sed -i 's/\"%Y-%m-%d\"/\"2022-05-11\"/g' ~/BirdNET-Pi/scripts/daily_plot.py
```
Finally, call daily_plot.py in the terminal to create the charts for that date.

When you are finished, you will want to run these commands to get things back to default:

git -C ~/BirdNET-Pi checkout scripts/daily_plot.py
sed -si "1 i\\#\!$HOME/BirdNET-Pi/birdnet/bin/python3" ~/BirdNET-Pi/scripts/daily_plot.py

Please let me know if you need any help with those steps as they address your request.

My best regards,
Patrick

PS
Please be sure that your system is up to date via "Tools" > "System Controls" > "Update"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebuild Combo table of detections manually #254

{{title}}

Replies: 3 comments 7 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Rebuild Combo table of detections manually #254

petri-o-ojala May 8, 2022

Replies: 3 comments · 7 replies

CaiusX May 8, 2022 Collaborator

mcguirepr89 May 8, 2022 Maintainer

petri-o-ojala May 8, 2022 Author

CaiusX May 9, 2022 Collaborator

petri-o-ojala May 9, 2022 Author

mcguirepr89 May 9, 2022 Maintainer

petri-o-ojala May 10, 2022 Author

petri-o-ojala May 12, 2022 Author

ehpersonal38 May 12, 2022 Collaborator

mcguirepr89 May 12, 2022 Maintainer

petri-o-ojala
May 8, 2022

Replies: 3 comments 7 replies

CaiusX
May 8, 2022
Collaborator

mcguirepr89 May 8, 2022
Maintainer

petri-o-ojala
May 8, 2022
Author

CaiusX May 9, 2022
Collaborator

petri-o-ojala
May 9, 2022
Author

mcguirepr89 May 9, 2022
Maintainer

petri-o-ojala May 10, 2022
Author

petri-o-ojala May 12, 2022
Author

ehpersonal38 May 12, 2022
Collaborator

mcguirepr89 May 12, 2022
Maintainer