Delay compensation while controlling a remote sensor

• US 10,551,474 B2
• Filed: 01/14/2014
• Issued: 02/04/2020
• Est. Priority Date: 01/17/2013
• Status: Active Grant

• Current Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Original Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Inventors: Bar David, Sagiv, Cohen, Yossi
• Primary Examiner(s): Hansell, Jr., Richard A
• Application Number:
  US14/761,529
• Publication Number:
  US 20150378000A1
• Time in Patent Office: 2,212 Days
• Field of Search: 348169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

The presently disclosed subject matter includes a method, a system and a delay compensation unit configured for compensating a delay in communication between a sensing unit and a control unit. A succession of two or more captured images are received from the sensing unit; information indicative of command dynamics parameters are obtained from the control unit; based on the information and a respective transfer function which models the dynamics of the sensor module, the expected reaction of the sensor module to the command is determined; and information indicative of the reaction before the command is executed in the sensing unit can be provided.

Claims

1. A delay compensation unit:

the delay compensation unit is operatively connectable to a control unit;

the control unit is configured to communicate over a communication link with a sensing unit, wherein a delay in communication between the control unit and the delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the sensing unit comprises a sensor module configured to;

capture a plurality images of a scene, and transmit a succession of images to the control unit;

the delay compensation unit comprises at least one computer processor configured, responsive to a command issued in the control unit with respect to the succession of images, to;

obtain from the control unit information indicative of command dynamics parameters, characterizing the command issued at the control unit;

determine, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensing module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed at the sensing unit;

wherein the command includes one or more of the following commands;

i. a sensor-movement command, directed for pointing the sensor in a desired direction;

ii. a zoom command; and

iii. a focus command;

in case the command is a sensor-movement command the delay compensation unit is configured to;

determine, based on information indicative of the command dynamics parameters and the respective transfer function, a future location of the desired pointing direction of the sensor in a captured image following the execution of the command;

provide an indication to the control unit indicating the future location before the command is executed at the sensing unit;

in case the command is a zoom command;

the delay compensation unit is configured to;

determine, based on the information indicative of the command dynamics parameters and the respective transfer function, the effect of the zoom command on the two or more captured images;

generate a digital simulation of the effect of the zoom command;

provide the control unit with digital simulation before the command is executed in the sensing unit; and

in case the command is a focus command;

the delay compensation unit is configured to;

generate, based on the information indicative of the command dynamics parameters and the respective transfer function which models the dynamics of the sensor module, a focus curve indicative of the optimal focus of the sensor;

provide the control unit with data indicative of said focus curve;

indicate a position of a current focus with respect to the focus curve before the focus command is executed in the sensing unit, thereby providing the control unit with information indicative of the optimal focus.

2. The delay compensation unit according to claim 1 wherein the data indicative of the command dynamics parameters include data indicative of the angular velocity of a respective command operation.

3. An electro optic system, comprising:

a sensing unit comprising a sensor module, the sensing unit configured to communicate over a communication link with a control unit, to capture two or more images of a scene and to generate a succession of images and transmit the succession of images to the control unit;

a delay compensation unit operatively connected to the control unit, wherein a delay in communication between the control unit and the delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the delay compensation unit comprises at least one computer processor configured, responsive to a command issued in the control unit with respect to the succession of images, to;

obtain from the control unit information indicative of command dynamics parameters, characterizing the command issued at the control unit;

determine, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensing module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein said command includes one or more of the following commands;

i. a sensor-movement command, directed for pointing the sensor in a desired direction;

ii. a zoom command;

iii. a focus command;

in case the command is sensor-movement command the delay compensation unit is configured to;

determine, based on information indicative of the command dynamics parameters and the respective transfer function, a future location of the desired pointing direction of the sensor in a captured image, following the execution of the command;

provide an indication to the control unit indicating the future location before the command is executed in the sensing unit;

in case the command is a zoom command, the delay compensation unit is configured to;

determine, based on the information indicative of the command dynamics parameters and the respective transfer function, the effect of the zoom command on the two or more captured images;

generate a digital simulation of the effect of the zoom command;

provide the control unit with digital simulation before the command is executed in the sensing unit; and

in case the command is a focus command, the delay compensation unit is configured to;

generate, based on the information indicative of the command dynamics parameters and the respective transfer function which models the dynamics of the sensor module, a focus curve indicative of the optimal focus of the sensor;

provide the control unit with data indicative of said focus curve;

indicate a position of a current focus with respect to the focus curve before the focus command is executed in the sensing unit, thereby providing the control unit with information indicative of the optimal focus.

4. The system according to claim 3 wherein the information indicative of the command dynamics parameters includes data indicative of the angular velocity of a respective command operation.

5. The system according to claim 3 wherein said sensing unit is located in any one of a list including:

an airborne vehicle;

a ground vehicle;

a marine vehicle;

or a missile; and

wherein said control unit and said delay compensation unit are located in another remote location.

6. The system according to claim 3 wherein the command is a command issued with respect to a selected object of interest in the scene and wherein the sensing unit further comprises a video motion detection (VMD) module;

the sensing unit is configured to activate the VMD module to identify moving objects in a predefined area;

in case the VMD module identifies moving objects within the predefined area, the moving objects are designated as candidate objects for being the selected object.

7. A method of compensating a delay in communication between a sensing unit and a control unit, where a delay in communication between the control unit and a delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the method comprising;

using a computer processor in the delay compensation unit for performing at least the following;

receiving from said sensing unit a succession of two or more captured images captured by a sensor module located at the sensing unit;

obtaining from the control unit information indicative of command dynamics parameters, characterizing a command issued at the control unit for controlling the sensor module;

determining, based on the information and a respective transfer function which models the dynamics of a sensor module, an expected reaction of the sensor module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein said command includes one or more of the following commands;

i. a sensor-movement command;

ii. a zoom command;

iii. a focus command;

wherein in case the command is a sensor-movement command the method further comprising;

determining, based on the information indicative of the command dynamics parameters and the respective transfer function, a future location of a center of field of view of the sensor module following the execution of the command; and

providing data to the control unit for presentation the future location of the center of field of view before the command is executed in the sensing unit;

in case the command is a zoom command, the method further comprising;

determining, based on the information indicative of the command dynamics parameters and the respective transfer function, the effect of the zoom command on the captured images;

generating a digital simulation of the effect of the zoom command; and

providing the control unit with digital simulation before the command is executed in the sensing unit; and

in case the command is a focus command, the method further comprising;

generating, based on the information indicative of the command dynamics parameters and the respective transfer function, a focus curve indicative of the optimal focus of the sensor;

providing the control unit with data indicative of said focus curve; and

indicating a position of a current focus with respect to the focus curve before the focus command is executed in the sensing unit, thereby providing the control unit with information indicative of the optimal focus.

8. The method according to claim 7, wherein the command dynamics parameters include data indicative of the angular velocity of a respective command operation.

9. The method according to claim 7 wherein said providing data to the control unit for presentation of the future location of the center of field of view further comprises:

placing an indicator in the latest available image of a surveyed scene at a future location in the center of the field.

10. The method according to claim 7 wherein said determining of the future location of the center of field of view following the execution of the command is based also on a time delay between the sensing unit and control unit.

11. A non-transitory computer storage device readable by a computer device, tangibly embodying a program of instructions executable by the computer device to perform a method of compensating a delay in communication between a sensing unit and a control unit, the method comprising:

receiving from said sensing unit a succession of two or more captured images;

obtaining from the control unit information indicative of command dynamics parameters, characterizing a command issued at the control unit for controlling a sensor module in the sensing unit;

determining, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensor module to the command; and

providing data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein said command includes one or more of the following commands;

i a sensor-movement command;

ii a zoom command;

iii a focus command;

wherein in case the command is a sensor-movement command the method further comprising;

determining, based on the information indicative of the command dynamics

parameters and the respective transfer function, a future location of a center of field of view of the sensor module following the execution of the command; and

providing data to the control unit for presentation the future location of the center of field of view before the command is executed in the sensing unit;

in case the command is a zoom command, the method further comprising;

determining, based on the information indicative of the command dynamics

parameters and the respective transfer function, the effect of the zoom command on the captured images;

generating a digital simulation of the effect of the zoom command; and

providing the control unit with digital simulation before the command is executed in the sensing unit; and

in case the command is a focus command, the method further comprising;

generating, based on the information indicative of the command dynamics parameters and the respective transfer function, a focus curve indicative of the optimal focus of the sensor;

providing the control unit with data indicative of said focus curve; and

indicating a position of a current focus with respect to the focus curve before the focus command is executed in the sensing unit, thereby providing the control unit with information indicative of the optimal focus.

12. A delay compensation unit:

the delay compensation unit is operatively connectable to a control unit;

the control unit is configured to communicate over a communication link with a sensing unit, wherein a delay in communication between the control unit and the delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the sensing unit comprises a sensor module configured to;

capture two or more images of a scene, generate a succession of images and transmit the succession of images to the control unit;

the delay compensation unit comprises at least one computer processor configured, responsive to a command issued at the control unit with respect to the succession of images, to;

obtain from the control unit information indicative of command dynamics parameters, characterizing the command issued at the control unit;

determine, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensing module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed at the sensing unit;

wherein the command is a lock command, directed for locking the sensor on a selected object of interest in the scene,

the delay compensation unit is configured in case said object of interest is moving, to;

determine future location of said object at the time the command is executed in said sensing unit;

modify the lock command to include the future location; and

transmit the modified lock command to the sensing unit thereby providing the sensing unit with information indicative of predicted real-time location of the selected object.

13. The delay compensation unit according to claim 12 further configured to execute said lock command with respect to the selected object notwithstanding a time-delay between a time when the sensing unit acquires the image with the selected object, to a time when a corresponding command is received at the sensing unit with respect to the selected object.

14. A method of compensating a delay in communication between a sensing unit and a control unit, where a delay in communication between the control unit and a delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the method comprising;

using a computer processor in the delay compensation unit for performing at least the following;

receiving from said sensing unit a succession of two or more captured images captured by a sensor module located at the sensing unit;

obtaining from the control unit information indicative of command dynamics parameters, characterizing a command issued at the control unit for controlling the sensor module;

determining, based on the information and a respective transfer function which models the dynamics of a sensor module, an expected reaction of the sensor module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein the command is a lock command directed for locking the sensor onto a selected object of interest in a surveyed scene;

in case the object of interest is moving, the method further comprising;

determining future location of said object at the time the command is executed in said sensing unit;

modifying the lock command to include the future location, in case said object of interest is moving; and

transmitting the modified lock command to the sensing unit, thereby providing the sensing unit with information indicative of predicted real-time location of the selected object.

15. The method according to claim 14, wherein the command dynamics parameters include data indicative of the angular velocity of a respective command operation.

16. A non-transitory computer storage device readable by a computer device, tangibly embodying a program of instructions executable by the computer device to perform a method of compensating a delay in communication between a sensing unit and a control unit, the method comprising:

receiving from said sensing unit a succession of two or more captured images;

obtaining from the control unit information indicative of command dynamics parameters, characterizing a command issued at the control unit for controlling a sensor module in the sensing unit;

determining, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensor module to the command; and

providing data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein the command is a lock command directed for locking the sensor onto a selected object of interest in a surveyed scene and wherein the command dynamics parameters include data indicative of the angular velocity of a respective command operation;

in case the object of interest is moving, the method further comprising;

determining future location of said object at the time the command is executed in said sensing unit;

modifying the lock command to include the future location, in case said object of interest is moving; and

transmitting the modified lock command to the sensing unit, thereby providing the sensing unit with information indicative of predicted real-time location of the selected object.

17. An electro optic system, comprising:

a sensing unit comprising a sensor module, the sensing unit configured to communicate over a communication link with a control unit, to capture two or more images of a scene and to generate a succession of images and transmit the succession of images to the control unit;

a delay compensation unit operatively connected to the control unit, wherein a delay in communication between the control unit and the delay compensation unit is smaller than a delay in communication between the control unit and the sensing unit;

the delay compensation unit comprises at least one computer processor configured, responsive to a command issued in the control unit with respect to the succession of images, to;

obtain from the control unit information indicative of command dynamics parameters, characterizing the command issued at the control unit;

determine, based on the information and a respective transfer function which models the dynamics of the sensor module, an expected reaction of the sensing module to the command; and

provide data to the control unit for presentation of the expected reaction of the sensing unit to the command before the command is executed in the sensing unit;

wherein the command is a lock command, directed for locking the sensor on a selected object of interest in the scene,

the system is configured in case said object of interest is moving, to;

determine future location of said object at the time the command is executed in said sensing unit;

to modify the lock command to include the future location; and

transmit the modified lock command to the sensing unit thereby providing the sensing unit with information indicative of predicted real-time location of the selected object.

18. The system according to claim 17 wherein said sensing unit is configured, responsive to a lock command received from delay compensation unit:

to search for moving objects in a predefined area surrounding the predicted real-time location of the selected object in order to ensure that the correct moving object is selected.