Backlog - How do I get rid of it?

[krummrey]

My instance is running on a Pi4 with 4GB of Ram. Should be plenty, and the load average is below 30%. Yet the database is always busy and I often have a backlog of 2-4 hours. Is there something I can do to speed it up? Create a database index to speed it up?

[Nachtzuster]

That seems odd indeed.

Yet the database is always busy

What exactly does this mean? how do you know?

[krummrey]

It says "Database is busy" when I open the page. After a while it loads, but has a backlog.
In the "Services" Tab it currently says "BirdNET Analysis (stalled - backlog of 399 files in ~/BirdSongs/StreamData/)"

[MueJosh]

I had the same problem in the beginning
My sd card was too slow to handle the writing/reading of birdsongs, so it always had some issues.

What kind of sd card are you using?
Any other applications on your raspberry?

Are you using BirdWeather and apprise?
Depending on your settings this can slow down the analyzing process, though on this fork it works much better.

Does Your pi have a heatsink and appropriate power supply?

[krummrey]

No SD-Card, running off a 16GB USB stick
Fullsize heatsink case (like this)
fresh install, updated, no pending updates, no other software running on it.
No notifications.

Memory: 2.14 GiB of 3.70 GiB used
Load Averages | 4.18 3.57 3.38 - 19%

No under voltage detected.

Could it be, that it is waiting for Birdweather?

09:39:36---[server][INFO] READING AUDIO DATA...
09:39:36---[server][INFO] READING DONE! READ 5 CHUNKS.
09:39:36---[server][INFO] ANALYZING AUDIO...
09:39:36---[birdnet_analysis][INFO] 2024-04-06;08:59:43;Chloris chloris;Grünfink;0.9577;53.6397;10.2162;0.66;14;1.25;0;Grünfink-96-2024-04-06-birdnet-08:59:43.mp3
09:39:37---[server][INFO] DONE! Time 1.28 SECONDS
09:39:37---[server][INFO] 0.0;3.0-('Chloris chloris_Grünfink', 0.43267784)
09:39:37---[server][INFO] 3.0;6.0-('Chloris chloris_Grünfink', 0.8889095)
09:39:37---[server][INFO] 6.0;9.0-('Chloris chloris_Grünfink', 0.46040592)
09:39:37---[server][INFO] 9.0;12.0-('Chloris chloris_Grünfink', 0.57832104)
09:39:37---[server][INFO] 12.0;15.0-('Columba palumbus_Ringeltaube', 0.8726949)
09:39:52---[birdnet_analysis][INFO] 2024-04-06;08:59:43;Chloris chloris;Grünfink;0.9174;53.6397;10.2162;0.66;14;1.25;0;Grünfink-92-2024-04-06-birdnet-08:59:43.mp3
09:39:58---[birdnet_analysis][INFO] 2024-04-06;08:59:43;Chloris chloris;Grünfink;0.7267;53.6397;10.2162;0.66;14;1.25;0;Grünfink-73-2024-04-06-birdnet-08:59:43.mp3
09:40:04---[birdnet_analysis][INFO] 2024-04-06;08:59:43;Chloris chloris;Grünfink;0.8846;53.6397;10.2162;0.66;14;1.25;0;Grünfink-88-2024-04-06-birdnet-08:59:43.mp3
09:40:22---[utils.reporting][INFO] Soundscape POST Response Status - 201
09:40:23---[utils.reporting][INFO] Detection POST Response Status - 201
09:40:44---[utils.reporting][ERROR] Cannot POST detection: HTTPSConnectionPool(host='app.birdweather.com', port=443): Read timed out. (read timeout=20)
09:41:04---[utils.reporting][ERROR] Cannot POST detection: HTTPSConnectionPool(host='app.birdweather.com', port=443): Read timed out. (read timeout=20)
09:41:24---[utils.reporting][ERROR] Cannot POST detection: HTTPSConnectionPool(host='app.birdweather.com', port=443): Read timed out. (read timeout=20)

[Nachtzuster]

Could it be, that it is waiting for Birdweather?

kind of: the connection to Birdweather is timing out, so that holds the analysis back for a minute or so.
Could be a flaky internet connection or Birdweather is having issues

[krummrey]

So posting to birdweather isn't a parallel task? Birdweather seems to have problems loading on my Mac also...

[Nachtzuster]

It kind of is, but we won't let the reporting thread get behind the analysis because other stuff happens in the reporting thread too.

Maybe external reporting should happen in yet another thread we do allow to get behind

[Nachtzuster]

@krummrey could you provide a screenshot of the View log window?
also screenshot of top would be usefull: could you ssh to the RPi and run top ?

[krummrey]

When no birds are detected it just blazes through, but when a bird is detected it takes ages to go along.
I've had 3000+ detections yesterday, >1000 this morning, too much to handle apparently.

[krummrey]

I'll try to get a small SSD, hopefully that helps. Thanks.
The "Database is busy" message is a bit misleading though.

[Nachtzuster]

The "Database is busy" message is a bit misleading though.

yes, it most certainly is. there is still a bit of cruft here and there, I'm afraid 😄

[Nachtzuster]

FYI looking at the code Database is busy might actually be correct and due to the slow storage, so a separate issue than the Birdweather induced delay

[krummrey]

I've installed it on a cheap 128GB SSD and it runs fine now. So it was my Imation 16GB USB 3.0 Stick that was too slow

[lloydbayley]

USB Drives (unless they're a specialist, good-quality USB3.0 drive) are not optimised for running applications like a high-class microSD card. Most of the time you'll find that a bog standard USB 2.0 drive will be enormously slower than even a basic microSD card. If you don't want to use SD cards, I suggest getting a nice SSD and booting the Pi from that and using it as storage.

[BAM-Team]

I can recommend SanDisk Extreme microSDXC UHS-I Memory Card (128 GB)
Never had any Problems with it on > 5 BirdnetPI Computers

[Frono]

Just lost a Pi with a SanDisk Extreme 64GB. However, I think it felt it was more due to power failures and corruption than the card itself, even though it would not reformat and had to be tossed. That said, it ran the birdnetpi is outside in the heat/humidity. Thunderstorms and resultant intermittent power outages are common. Even with all that the SanDisk Extreme ran a couple of years without issue. So agreed, I also recommend the SanDisk Extreme, and Extreme Pro. I just rebuilt the BirdNetPi (after the last failure) with a samsung Pro Endurance. So we will see how it compares. :)