RTSP + USB Audio - Simultaneous Sources?

[domenicdistefano]

Based on the text from the RTSP Stream box in the Advanced Settings page I don't think this is possible but thought I would ask anyway...

I have a second microphone in a different area of my yard I want to "merge" into the system. My idea is to place a second Pi with only a microphone in the secondary location and (TBD) figure out how to output it via an RTSP stream. If I could then ADD that source to BirdNET I would be in business.

Unfortunately again the text below seems clear that its an either or situation but it does say multiple RTSP streams can be added simultaneously so perhaps a couple of cheap Pi zeros as "audio servers" is the answer and simply not using the USB sound card on the BirdNET at all?

---

If you place an RTSP stream URL here, BirdNET-Pi will use that as its audio source.
Multiple streams are allowed but may have a impact on rPi performance.
Analyze ffmpeg CPU/Memory usage with top or htop if necessary.
To remove all and use the soundcard again, just delete the RTSP entries and click Save at the bottom.

[srd424]

I think you're right, the way the code is currently set-up I think it's either-or. Though in theory it shouldn't be too much re-architecting to allow both.

At the moment I use Pi Zero to push audio to my main general-purpose house server with the following gstreamer pipeline:

gst-launch-1.0 alsasrc provide-clock=false device=default:CARD=Microphone ! opusenc bitrate=48000 ! rtspclientsink location=rtsp://server.lan:8554/mic1

I chose gstreamer just because it seemed to pull in far fewer dependencies than ffmpeg, but it shouldn't be hard to do with ffmpeg either. I use mediamtx to push the RTSP to, though in theory the icecast instance that BNPI already creates could be used as well I think (although maybe not with RTSP.)

If you did want to mix local and remote audio, the quickest hack would probably be to present even the local audio as an RTSP/HTTP stream to birdnet. Probably adds a little overhead moving the data around unnecessarily, but hopefully not much.

[domenicdistefano]

This is a brilliant idea! I can just create RTSP Steam 1 from my existing BirdNET Pi and loop it back locally and then throw in a Pi zero with a second mic as RTSP 2. I am a little confused however in terms as how you are implementing it. So above you have you gstreamer command line you are using. So how does mediamtx fit in? Can you reply with your full recipe? Thanks!

[srd424]

You can grab mediamtx binaries from its github page - looks like there are multiple arm versions, so you'll need to pick the right one for your model of Pi. There's a default config file in the tarball, I haven't needed to change that, but I guess you might in theory want to set a password if there are untrusted devices/users on your network.

[srd424]

If you're "looping back" audio from the Pi the BirdNET-Pi is installed on, you could probably use ffmpeg instead of gstreamer. I'm about to sit down to supper, but I'll see if I can concoct an appropriate ffmpeg command line today or tomorrow :)

[domenicdistefano]

Manny thanks & take your time. So gstreamer (or ffmpeg) creates the audio then mediamtx takes that feed and creates the RTSP stream that would be pulled from the BirdNET pi? In my case my BirdNET pi is a RP 4 (where I am plaining on looping back the local audio as you suggested) and I found an older RP 3 I am going to connect the new mic thus producing the second RTSP feed to BirdNet.

[srd424]

Something like this might work if you wanted to use ffmpeg instead of gstreamer:

ffmpeg -nostdin -f alsa -i default -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://server.lan:8554/starlite

Untested on an actual Pi, but it seems to work on my laptop :)

[domenicdistefano]

So does the above ffmpeg command both encode the audio from the local USB sound card mic and produce the RTSP stream itself? I am confused how you are mediamtx fits in.

I got my old RP 3 going but jut need the microphone to arrive from Micbooster. I can probably test however in the meantime somewhat.

[domenicdistefano]

I think I have all of this figured out! Just to recap my goal is to utilize the local USB microphone installed on my BirdNET Pi (lets call it RP1) PLUS add a second remote USB microphone located on a secondary Pi, not running BirdNET (lets call it RP2). Net result is to have multiple microphones simultaneously feeding the single instance of BirdNET audio so detections can be made originating from either microphone which are located at separate locations. In a perfect world I would want to have multiple independent "nodes" or instances of BirdNET running at each location somehow consolidated into a "master" BirdNET Pi. This however does seem to work but I just got it working and I need to let it run for a few days in production to make sure the wheels don't fall off or at least the steering wheel doesn't shake too much.

So here is my final (pending tweaks) recipe. Many thanks to srd424 for his suggestions above.

Secondary USB microphone connected to secondary Pi (RP2)
Note - this Pi does NOT have BirdNET running and only has the minimum software installed in order to produce an RTSP feed.

Step 1 - Install the Raspberry Pi OS Lite (64-bit) as per the official BirdNET instructions but just do not install BirNET. Pro Tip - For reliability & speed purposes install the OS on an external SATA drive using a SATA to USB 3 adapter. A 512GB drive is only about $25-$30 on Amazon these days with the adapter cable running $8.

Step 2 - (still operating on RP2) Install ffmpeg & mediamtx

• First login as the regular user, update the OS as usual, set the root password, enable SSH login by Root, and then re-login as root.
• Next install ffmpeg apt-get install ffmpeg
• Install lsof apt-get install lsof
• Install mediamtx making sure you are in the /root directory
  wget -c https://github.com/bluenviron/mediamtx/releases/download/v1.0.0/mediamtx_v1.0.0_linux_arm64v8.tar.gz -O - | sudo tar -xz
• Test to see if you can create a RTSP feed with audio streaming from your USB microphone. In my case this is the particular code that worked:

./mediamtx &
ffmpeg -nostdin -f alsa -ac 1 -i hw:1 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp

• To see if your RTSP feed is working fire up VLC, go to Media, Open Network Stream, and paste in rtsp://hostnameofRP2:8554/birdmic and hit enter. You should connect and be able to hear your microphone.
• Still in the /root directory create a startmic.sh file nano startmic.sh
• The contents of the file is the same code used above to test.
• Make the file executable chmod +x startmic.sh
• Execute the crontab command crontab -e and select nano as your editor.
• Paste in @reboot /home/linuxconfig/startmic.sh then save and exit nano.
• Reboot the Pi and test again with VLC to make sure the RTSP stream is live.

Step 3 - Reconfigure your BirdNET Pi (RP1)

• Go into your BirdNET Tools, Settings, Advanced Settings and add two entries under the RTSP section. rtsp://localhost:8554/birdmic & rtsp://ipaddressofRP2:8554/birdmic

• Reboot your BirdNET Pi

• Again login as the regular user, update the OS as usual, set the root password, enable SSH login by Root, and then re-login as root.

• Install ffmpeg apt-get install ffmpeg

• Install lsof apt-get install lsof

• Install mediamtx making sure you are in the /root directory
  wget -c https://github.com/bluenviron/mediamtx/releases/download/v1.0.0/mediamtx_v1.0.0_linux_arm64v8.tar.gz -O - | sudo tar -xz

• You now need to adjust a port used by mediamtx on your BirdNET Pi (RP1) because it conflicts with the same port already in use.
  Fire up Nano and edit the mediamtx.yml nano /root/mediamtx.yml and search for the line that reads hlsAddress: :8888 and change 8888 to 7777 then save and exit.

• Test to see if you can create a RTSP feed with audio streaming from your USB microphone. In my case this is the particular code that worked:

./mediamtx &
ffmpeg -nostdin -f alsa -ac 1 -i hw:1 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp

• To see if your RTSP feed is working fire up VLC, go to Media, Open Network Stream, and paste in rtsp://hostnameofRP1:8554/birdmic and hit enter. You should connect and be able to hear your microphone.
• Still in the /root directory create a startmic.sh file nano startmic.sh
• The contents of the file is the same code used above to test.
• Make the file executable chmod +x startmic.sh
• Execute the crontab command crontab -e and select nano as your editor.
• Paste in @reboot /home/linuxconfig/startmic.sh then save and exit nano.
• Reboot the Pi and test again with VLC to make sure the RTSP stream is live.

Everything should now be working! To verify hit the Spectrogram on the main BirdNET menu select each each of your RTSP feeds which should be showing in a drop down menu. As long as you hear audio from each you are in business.

My secondary microphone is a couple of weeks out shipping from the UK but in the meantime I am going to monitor performance and make sure the microphone on RP1 performs just as well as when it was "natively" connected as now its looped back via RTSP.

[domenicdistefano]

Update - sometimes due to the nature of USB the hw: parameter can "float" randomly between 1&4 upon a reboot Fix is just to repeat the command four times as one will eventually be the one you want.

./mediamtx &
ffmpeg -nostdin -f alsa -ac 1 -i hw:4 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:3 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:2 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:1 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp

[bagrcoufal]

Thanks for good idea with microphone stream. But I have problem with script. Only command working perfectly:
./mediamtx & ffmpeg -nostdin -f alsa -i default -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdnet
The same command in execute in script. not working.

#!/bin/bash
/home/pi/BirdNET-Pi/mediamtx & /usr/bin/ffmpeg -nostdin -f alsa -i default -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdnet

error [alsa @ 0x55ab3f8630] cannot open audio device default (No such file or directory)
default: Input/output error
Any Idea? Thanks

[Peterkal2112]

Update - sometimes due to the nature of USB the hw: parameter can "float" randomly between 1&4 upon a reboot Fix is just to repeat the command four times as one will eventually be the one you want.

./mediamtx &
ffmpeg -nostdin -f alsa -ac 1 -i hw:4 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:3 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:2 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp
ffmpeg -nostdin -f alsa -ac 1 -i hw:1 -acodec libmp3lame -b:a 160k -ac 2 -content_type 'audio/mpeg' -f rtsp rtsp://localhost:8554/birdmic -rtsp_transport tcp

Brilliant...

Why not use "default" as Hardware... if the HW parameter float randomly, the default can be a solution...

do you have these steps also for docker ?

I do not want to install ffmpeg and mediamtx, easier and faster is to set up a docker compose file and fire them up....

I like BirdNet-Pi but it is a shame that it does not have a docker image, or the install script supports only Debian 11 and not 12...

[domenicdistefano]

Not sure if I ever tried "default"... Also when I was working with this that was before my "Docker awakening"...

I would LOVE a docker version of all of this.

[srd424]

My dockerized version mostly works, it just needs documenting properly! Unfortunately I got overtaken by health issues and other circumstances so everything rather got paused. In particularly I think I haven't documented the "multi site" support stuff, so what docs there are are a little out of date :(

My pi zero which does the sound recording is now actually running Balena OS, so I in theory I should be able to publish the sound recording bit as an app on Balena Hub or something, too.

[srd424]

FWIW on the docker front, if you haven't seen it, someone else seems to be working on this too: #211 (comment)

Meanwhile I'm still distracted by the usual pile of real-world stuff, sorry :(

[alexbelgium]

Hi, my docker version works but is heavily dependent on homeassistant - it could however easily be ported in a more general manner by adapting the dockerfile :) for example, mapping of folders, permissions... Is managed by HA so not included out of the box in my version. But if we add it in my dockerfile then it would achieve both aims with a single container :) or don't hesitate to take part of my code if useful to create another standalone container!

[alexbelgium]

That's really great! Is there an impact on sound quality to perform this additional encoding to rtsp ? thanks !

[srd424]

RTSP itself is just the transport, but whatever codec you choose to stream will lose a bit of quality (unless you use flac or plain wav.) I have a vague memory that when I originally set this up with gstreamer & opus as above, I may have tested to make sure I didn't see a significant drop in detections over the same length of time in similar weather conditions etc. At some point I intend to move to mp3 because it seems to consume significantly less cpu on a pi zero, I'll try to remember to do the same test again.

[lolorc]

that looks interesting, I'm also playing with mediamtx & go2rtc, I've just replaced pulseaudio/arecord/ffmpeg(livestream) by one ffmpeg instance doing the wav recordings directly from alsa and also the opus audio encoding to mediamtx. That makes it more efficient to stream video and audio to frigate for instance.

[srd424]

Ah, nice, that's something I'd been meaning to look at! I also think saved recordings get converted back to mp3 from the wav? So there might be another step there that could be saved ..

[lolorc]

converted to opus to be streamed, saved in wav, analyzed in wav, then converted to mp3.
could stream and save in opus, but that would require decoding it to analyze
for an external source it's decoded in wav, analyzed then encoded again, isn't it ?

[alexbelgium]

Thanks for the guide! I've adapted the code to use pcm :
ffmpeg -nostdin -f alsa -acodec pcm_s16le -ac 2 -ar 48000 -i hw:1,0 -f rtsp -acodec pcm_s16be rtsp://localhost:8554/birdmic -rtsp_transport tcp

[dumbledorethor]

Thanks for the guide! Here is the config I used for RPI Zero W2 with BOYA BY-LM40 microphone.

./mediamtx &
ffmpeg -nostdin -f alsa -acodec pcm_s24le -channels 1 -ar 48000 -i hw:1,0 -f rtsp rtsp://localhost:8554/birdnet -rtsp_transport tcp

[kongjudas]

Thanks! That worked for my BY-LM40 but with a huge delay. Do you know how to get it to work with libopus? I get an error no matter what I try. This is my latest attempt (Don't use it, not working):

./mediamtx &
ffmpeg -nostdin -f alsa -ac 1 -ar 48000 -i hw:1,0 -c:a libopus -b:a 32k -f rtsp rtsp://localhost:8554/birdnet -rtsp_transport tcp