Measuring latency in a test system using captured images

• US 9,514,664 B2
• Filed: 09/25/2012
• Issued: 12/06/2016
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Blanton, William Brendan, Allen, Robert Crenshaw, DuBois, Thomas Alfred
• Primary Examiner(s): Patel, Jay Au
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US13/626,144
• Publication Number:
  US 20140085459A1
• Time in Patent Office: 1,533 Days
• Field of Search: 348/135
• US Class Current: 1/1
• CPC Class Codes: G09G 2360/145 the light originating from the display screen
• CPC Class Codes: G09G 2380/12 Avionics applications
• CPC Class Codes: G09G 3/006 Electronic inspection or testing of displays and display drivers, e.g. of LED or LCD displays testing individual LED's G01R31/2635; testing lamps G01R31/44; testing of optical features of LCD displays G02F1/1309
• CPC Class Codes: H04N 17/04 for receivers

Abstract

A latency measurement system includes an event generation device that generates an initial event used to measure system latency. A component test system receives the event and in response outputs a test component output signal and a zero-latency indicator. An electronics system including a multifunction display unit receives the test component output signal and displays a visible element on the multifunction display unit. A camera generates a series of recorded images, where each recorded image contains an image of the zero-latency indicator and an image of the visible element. A processor then determines the system latency by determining a time difference in the series of recorded images between a representation of an occurrence of the event in the image of the zero-latency indicator and a representation of the occurrence of the event in the image of the visible element.

Claims

1. A latency measurement system, the system comprising:

an event generation device that generates an event used to measure latency;

a hardware test component for controlling a vehicle, wherein the test component is configured to receive the event and in response output a test component output signal;

a zero-latency indicator configured to output a visual indication of the event, wherein the zero-latency indicator and the test component receive the event from the event generation device at substantially the same time;

an electronics system including a multifunction display device, where the electronics system receives the test component output signal and, in response, displays an image on the multifunction display device;

a camera that generates a series of recorded images, where each recorded image contains the zero-latency indicator and the image displayed by the multifunction display device; and

a processor that determines a latency of the test component by determining a time difference in the series of recorded images between a representation of an occurrence of the event as indicated by the zero-latency indicator and a representation of the occurrence of the event as indicated by the image of the multifunction display device.

2. The latency measurement system of claim 1, wherein the event further comprises one of:

an event trigger signal;

rotational motion;

or

electromagnetic radiation.

3. The latency measurement system of claim 1, wherein the test component is one of an imaging device, an actuator, or an inertial navigation system and the zero-latency indicator comprises an electromagnetic radiation emitter.

4. The latency measurement system of claim 1, zero-latency indicator comprises an electromagnetic radiation emitter.

5. The latency measurement system of claim 4, wherein the electromagnetic radiation emitter further comprises:

an infrared illumination source;

a low-level visible light illumination source;

a visible light source;

or

a laser output of a laser level.

6. The latency measurement system of claim 1, wherein the electronics system further comprises at least one of:

an avionics control system;

a vehicle control system;

or

a ship control system.

7. The latency measurement system of claim 1, wherein the multifunction display device further comprises at least one of:

a graphical image display device that graphically displays the image based input received at the electronics system;

a warning/caution/advisory (WCA) indicator display that is activated based input received at the electronics system;

or

an artificial horizon display that corresponds to input received at the electronics system.

8. The latency measurement system of claim 1, where determining the time difference further comprises:

detecting a threshold within the representation of the occurrence of the event indicated by the zero-latency indicator and a threshold within the representation of the occurrence of the event indicated by the image;

creating a plurality of binary waveforms based on detecting the threshold within the representation of the occurrence of the event as indicated by the zero-latency indicator and the threshold within the representation of the occurrence of the event as indicated by the image;

detecting edges for each of the plurality of binary waveforms that measure one of rising or falling; and

measuring a delta time between the zero-latency indicator and the image in the multifunction display device based on the detected edges for each of the plurality of binary waveforms.

9. The latency measurement system of claim 1, wherein determining the time difference further comprises:

determining a difference in a rate of angular change between the representation of the occurrence of the event as indicated by the zero-latency indicator and a rate of angular change of the representation of the occurrence of the event as indicated by the image.

10. A method of measuring latency, the method comprising:

triggering an event that generates an indication of the event at both a hardware test component for controlling a vehicle and a zero-latency indicator at substantially the same time;

sending the indication of the event from the test component to a multi-function display device which displays an image in response to the indication of the event;

recording a series of images with a camera, where each image contains both the zero-latency indicator and the image displayed by the multi-function display device; and

processing, by a computing device, the series of images to compute a latency value of the test component based on determining a time difference between the indication of the event by the zero-latency indicator and the multi-function display device.

11. The method of measuring latency according to claim 10, wherein the triggering an event further comprises one of:

generating an event trigger signal;

generating rotational motion;

or

transmitting electromagnetic radiation.

12. The method of measuring latency according to claim 10, further comprising:

emitting an electromagnetic radiation emission from the zero-latency indicator based on receiving the indication of the event at the zero-latency indicator.

13. The method of measuring latency according to claim 12, wherein the emitting the electromagnetic radiation emission further comprises one of:

emitting an infrared illumination source;

emitting a low-level visible light illumination source;

emitting a visible light source;

or

emitting a laser output of a laser level.

14. The method of measuring latency according to claim 10, wherein the sending the indication of the event further comprises sending the indication through an electronics system to the multi-function display device.

15. The method of measuring latency according to claim 10, wherein the displaying image further comprises one of:

displaying the image on a graphical image display device that graphically outputs display images based on input received at a the electronics system;

displaying the image on a warning/caution/advisory (WCA) indicator that is activated based on input received at the electronics system;

or

displaying the image on an artificial horizon display that corresponds to input received at the electronics system.

16. The method of measuring latency according to claim 10, wherein the processing the series of images further comprises:

detecting a threshold corresponding to the zero-latency indicator and a threshold corresponding to the image displayed on the multi-function display device;

creating a plurality of binary waveforms based on detecting the threshold corresponding to the zero-latency indicator and a threshold-corresponding to the image displayed on the multi-function display device;

detecting edges for each of the plurality of binary waveforms that measure one of rising and falling; and

measuring a delta time between the zero-latency indicator and the image on the multi-function display device based on the detected edges for each of the plurality of binary waveforms.

17. The method of measuring latency according to claim 10, wherein determining the time difference further comprises:

determining a difference in a rate of angular change between the indication of the event by the zero-latency indicator and a rate of angular change indicated by the image on the multi-function display device.

18. A method for measuring latency in electronic equipment, comprising:

generating pulsed source signals at a sensor aperture and at a multi-functional display;

capturing a source illumination and multi-functional display illumination directly and through a sensor path using a high frame rate camera;

isolating a plurality of regions of interest for the source illumination and the multi-functional display illumination in video captured by the high frame rate camera;

detecting peak thresholds in the regions of interest;

creating a plurality of binary waveforms based on the detected peak thresholds;

detecting a plurality of edges of one of the plurality of binary waveforms by measuring one of rising or falling edges; and

measuring a delta time between the generated pulsed source signals and the plurality of edges providing a latency measurement.

19. The method of claim 18, further comprising:

generating the source illumination by emitting an electromagnetic radiation emission based on receiving the generated pulsed source signals; and

generating the multi-functional display illumination by receiving an event signal based on the generated pulsed source signals.