After having introduced an automated latency measurement method in the last blog post, we used it to test several combinations of monitors and FPV systems, both digital and analog.
Transmission systems tested:
Analog: standard 5.8 GHz system with a CCTV camera (XK-3089).
Analog: standard 5.8 GHz system with a GoPro camera, running at 720p and 100fps.
Digital: Amimon Prosight, in both HP (High Performance) and HQ (High Quality) modes.
Digital: fpv.blue, using the HDMI input (GoPro, 720p@50fps) and custom camera (at 720p and 480p).
Digital: Wifibroadcast with default settings, running on a Raspberry Pi 1 on the transmitter and a Raspberry Pi 2 on the receiver.
Display devices tested:
Analog: a simple FPV monitor (RMRC-LCD-12).
Analog and Digital: Headplay HD googles.
Digital: a simple HD monitor (part numbers LQ121K1LG52, M.NT68676.2A).
The previous blog post goes into the methodology with more detail, but we are using a photodiode, a Texas Instruments OPT101P, to measure video latency. The measurement starts from a random point in time by toggling a LED via an Arduino and checking how long the photodiode takes to detect a change in light (code here). This has several advantages over the standard stopwatch method, as already explained.
The raw results can be downloaded from here. The following is a graph visualising them that we invite you to take as a comparison point next time you are thinking about latency.
The display device matters. A lot. If you flown with an analog system and Headplay glasses you already experienced more latency than many digital FPV configurations.
Analog systems are still the only ones that can claim close to zero latency. Digital systems are, at best, two times slower.
Increasing GoPro framerate over 50 fps hoping to decrease analog output latency does not help. In fact, latency increased by a few milliseconds in our tests when going from 50 to 100 fps.
The analog video output of a GoPro at 50 or 100 fps is also slower than many digital FPV solutions.
HDMI display devices are not immune from display latency. The Headplay glasses introduced an average of 24 milliseconds latency when feed with a 720p stream and only 16 ms when feed with a 1280×800 stream. The additional 6 milliseconds are explained by the time the Headplay takes to resize the image from 720p to 1280×800, something that does not need to happen if the native resolution is correctly provided (well done, Wifibroadcast!).
A word of advice if you decide to replicate the test. Do it with battery powered devices only. Displays, oscilloscopes, etc., were all creating too much noise in the photodiode voltage sensor, making measurements tricky. If you need some help replicating those numbers don’t hesitate to send an email.
A couple of months ago, when we first announced the product specifications, we had a video latency figure of around 80 milliseconds. That number had a note attached to it: a promise to substantially lower it, to 50 milliseconds or less.
This kept us busy for much more than anticipated and required a hardware revision, but it finally happened. This post talks about how FPV latency is usually measured, what FPV latency actually is, how we are measuring this number (how would you measure sub-frame latencies?) and ends with some test results.
The wrong way of measuring FPV latency
The “industry standard” in measuring video latency is nothing short of terrible. It goes something like this: take a portable device, run some kind of stopwatch application on it, point the FPV camera to its screen, take a picture including both FPV screen and stopwatch, subtract the difference.
As you can see, there are several elements here that can influence (timing) resolution: the freeware stopwatch not being written for the application (simply simulating a live, running stopwatch, and not actually implementing one with live output), the digital camera frame exposure time if the two stopwatches are not horizontally aligned (rolling shutter), the portable device display refresh rate, the FPV video system framerate, etc. All together those errors can easily exceed the actual latency of an analog FPV system, where glass to glass latency is usually around 20 milliseconds, a number that is not possible to measure when using the stopwatch method.
A definition of latency
Before measuring it, we better define it. Sure, latency is the time it takes for something to propagate in a system, and glass to glass latency is the time it takes for something to go from the glass of a camera to the glass of a display. However, what is that something? If something is a random event, is it happening in all of the screen at the same time, or is restricted to a point in space? If it is happening in all of the camera’s lenses at the same time, do we consider latency the time it takes for the event to propagate in all of the receiving screen, or just a portion of it? The difference between the two might seem small, but it is actually huge.
Consider a 30 fps system, where every frame takes 33 milliseconds. Now consider the camera of that system. To put it simply, every single line of the vertical resolution (say, 720p) is read in sequence, one at the time. The first line is read and sent to the video processor. 16 milliseconds later the middle line (line 360) is read and sent to the processor. 33 milliseconds from start the same happens to the last line (line 720). What happens when an event occurs on the camera glasses at line 360 and the camera just finished reading it? That’s easy, it is going to take the camera the time of a full frame (33 milliseconds) to even notice something changed. That information will have to be sent to the processor and down the line up to the display, but even by supposing all that to be latency-free (it is not), it takes the time of a full frame, 33 milliseconds, to propagate a random event from a portion of the screen in a worst case scenario.
That is what happens to analog systems, where the interlaced 60 frames per second are converted to 30 progressive and are affected by this latency. There is no such a thing as zero latency. It’s just a marketing gimmick, sustained by the difficulty of actually measuring those numbers.
Measuring latency with sub-frame resolution
Or, measuring the propagation time of a random event from and to a small portion of the camera/display, because the other way is wrong (it only partially takes framerate into account).
A simple way of doing it is to use an Arduino, light up a LED and measure the time it takes for some kind of sensor to detect a difference in the display. The sensor needs to be fast, and the most oblivious choice for the job (a photoresistor) is too slow, with some manufacturers quoting as much as 100 milliseconds to detect a change in light. For this we need to use something more sophisticated, a photodiode, maybe with included amplifier. A Texas Instruments OPT101P was selected for the job. The diode is pretty fast in detecting light changes, try putting it below a LED table lamp and you will be able to see the LED switching on and off – something usually measured in microseconds. However measuring the time between two slightly different lights in a screen is going to take some tweaking and you might be forced to increase the feedback loop of the integrated op amp of something like 10M Ohms.
However, the end result is worth it: You will have a system capable of measuring FPV latency with milliseconds precision.
Now that we are done with the introductions let’s look at the actual numbers.
Over 1000 consecutive measurements, that you can download as a data dump from here, the average glass to glass latency for a random event in a restricted area of the screen is 49 milliseconds.
Minimum latency is 34 ms, maximum latency is 67 ms. Those numbers are compatible with what explained above: they depend on the current camera read position and display output position: for a a 60 fps system with 16 ms frame times this is latency average ± 16, or 33 and 65 ms.
So, there you have it, latency is now under 50 milliseconds, as promised.
Can it be lower?
Uhm. Not really. Maybe it could be possible to cut a total of 2-5 milliseconds with months of work, but such an optimisation is not planned at this stage.
However, that is for 720p. If you are ok with compromising on video quality and using speciality cameras that can output very high frame rates, then yes, it is possible to go even lower. In the following example the average latency is 38 milliseconds.
This was achieved with a frame rate of around 80 fps at both camera and display. Some cameras can go even faster and output VGA (not NTSC) at 120 fps, but this wasn’t tested.
However, for sub 30 milliseconds, that is probably the way to go.