10-30ms latency or 100-300ms?

[ugnich]

There are 1000ms in one second.
0.1s = 100ms
From your photo it looks like the latency is 100-300ms, not 10-30ms.

[RomanLut]

Epic fail , shame on me! I will retest with glasses and update the docs.

[RomanLut]

It's hard to shoot glasses image; the latency is 90-110ms and does not depend on 30 or 50 fps mode.

I am sure delay is on RPI2W side because esp32 simply does not have memory to buffer more then 30ms.

Honestly I did not work on latency measurement and improvements yet; should be the topic for the next release.

I am declaring it 100ms untill I measure it properly with oscilloscope.

[ugnich]

HDMI output-input could also add some latency. Try to measure it on a modern desktop or laptop computer to reduce the number of components affecting the latency. No RasperryPi, no goggles, just the PC monitor. You can use this timer: https://openipc.org/tools/high-resolution-timer

I ordered an esp32-cam board to do my own experiments but it did not arrive yet.

Unrelated, please check out this thread: gehee/FPVue_android#7
The author is building an Android app to view the real time FPV video feed. It works with OpenIPC and maybe you could work with him to add support for your project.

[RomanLut]

Thanks for your interest, review is appreciatted as there is always a place for such dumb mistakes with such big number of features in the project :)

With profiling tools built into esp firmware, I can tell that frame arrives at gs not later then 33+10ms from frame scanning start and 10ms from frame scanning end (33ms per frame in 30 fps mode). What happens on PRI is a mistery and is not studied yet.

Personally I think that testing latency on notebook is pointless because latency is not important for GS with big screen.
I have to study GS+glasses https://youtu.be/RdzTcvRw8LY?si=d7pBuUtSvvJIgzaI&t=758. RPI should be able to receive, decode and display a frame inside 17 ms period in 60 fps videomode, so (33+10)/2+10 + 17 = 35,5 + [hdmi delay] should be achievable. I dont think hdmi delay with glasses is more then 20ms because Walksnail Avatar has latency ~50ms.

Hdmi delay can probably be elliminated with Oculus Quest 2, custom glasses or custom firmware for HD zero glasses https://github.com/ef-end-y/hdzero-doom

[Ncerzzk]

I have some test on low resolution 320*240(esp32, ov2640, overclocking, the image out from camera is about 70 fps), and get latency(end to end, monitor to monitor) at about 30-50ms, remove the impact from both side monitor update rate, the TRUE latency should be about 40ms.

[ugnich]

@RomanLut Thank you for the details. Having a profiler must be very helpful.

Because of the open source nature of this system I'm sure that people will want to use all kinds of hardware for the GS. It would be good to have a benchmark GS with the lowest latency possible so people could see what ESP32 is capable of and to be able to compare their own hardware against the benchmark.

What do you think about using an Android phone inside a VR headset as a "custom glasses"? Like this one: https://www.aliexpress.com/item/1005006996500059.html
Cheap, easy to build and does not depend on one vendor.

@Ncerzzk Thank you for sharing your numbers!

[RomanLut]

What do you think about using an Android phone inside a VR headset as a "custom glasses"?

I am following this project since development of Android 8812au driver and plan to implement GS in future. May not happend in this year though. I also hope someome will be quicker then me :)

[geofrancis]

I will do some testing soon, been busy with some other projects.

[Consti10]

I think this project uses jpeg and cpu decoding, right ?
By decoding directly into the framebuffer on rpi (using cpu) you can probably
reduce latency by 30ms or more.