Method for guiding a remote vehicle via lagged communication channel

• US 7,620,483 B2
• Filed: 12/23/2002
• Issued: 11/17/2009
• Est. Priority Date: 12/27/2001
• Status: Active Grant

• Current Assignee: Rafael - Armament Development Authority Ltd.
• Original Assignee: Rafael - Armament Development Authority Ltd.
• Inventors: Kalisman, Amir, Florentin, Itzhak, Berger, Liat
• Primary Examiner(s): Tran; Khoi
• Assistant Examiner(s): MARC, MCDIEUNEL
• Application Number:
  US10/499,799
• Publication Number:
  US 20050119801A1
• Time in Patent Office: 2,521 Days
• Field of Search: 701/2, 701/25, 701/26, 701/28, 701/301, 701/23, 700/245, 700/246, 700/247, 700/248, 700/258, 318/587, 348/114, 348/118, 348/119, 348/E7.001, 348/E7.046, 348/E7.088, 348/E7.092, 348/143, 348/581, 348/571, 348/E5.062, 348/E9.055, 348/630, 348/631, 348/663, 180/168, 901/1, 901/46, 901/47, 375/240.08, 375/240.12, 375/240.26, 386/46, 725/131, 726/26, 382/100, 382/107, 382/289, 455/421, 455/422, 455/427, 709/219
• US Class Current: 701/2
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0282 generated in a local control room

Abstract

The invention relates to a method for guiding from a remote control center a vehicle towards a target object, said remote control center communicating with the vehicle by means of a lagged communication channel, comprising:

• • At the vehicle: (a) Periodically capturing frame images by a camera, assigning to each of said captured frames an associated unique time stamp, and saving within a storage at the vehicle full frame data or partial frame data of captured frames and their associated time stamps; (b) For a plurality of saved frames, sending to the control center via the lagged communication channel full frame data, partial frame data or a combination thereof with the corresponding associated time stamp for each sent frame so that approximate or exact version of the sent frames can be reconstructed and displayed at the control center; At the control center: (c) Receiving said frame data and associated time stamps, sequentially reconstructing frame images from each said sent full and/or partial frame data, and displaying the reconstructed images on a display; (d) Upon marking by an operator at the control center a point on a specific displayed frame, sending coordinates indication relating to said marked point as appearing on said specific frame or on a reference frame available at the control center, and the time stamp associated with said specific or reference frame, as is the case, to the vehicle; At the vehicle: (e) Receiving said coordinates indication as marked and the sent frame time stamp; (f) Given the coordinates indication and frame time stamp as received, fast forward tracing said point or object coordinates from the said frame towards the most recently available captured frame, thereby finding the coordinates of the same point or object as appearing in the most recently available captured frame; and (g) Providing the coordinates of the target point or object within the most recently available captured frame, as found, to an inner guidance sub-system of the vehicle, for enabling it to track said object.

Claims

1. A method for guiding from a remote control center a vehicle (10) towards a target object, said remote control center communicating with the vehicle by a lagged communication channel comprising:

Periodically capturing frame images by a camera on the vehicle, assigning to each of said captured frames an associated unique time stamp, and saving full frame data or partial frame data of captured frames and their associated time stamps within a storage at the vehicle;

For a plurality of frames saved at the vehicle, sending to the control center via the lagged communication channel full frame data, partial frame data or a combination thereof with the corresponding associated time stamp for each sent frame so that an approximate version or an exact version of the sent frames can be reconstructed and displayed at the control center;

Receiving said frame data and associated time stamps at the remote control center, sequentially reconstructing frame images from each said sent fill and/or partial frame data, and displaying the reconstructed images on a display;

Upon marking by an operator at the control center a point on a specific displayed frame, sending coordinates indication relating to said marked point as appearing on said specific frame or on a reference frame available at the control center, and the time stamp associated with said specific frame or said reference frame to the vehicle;

Receiving said coordinates indication as marked and the sent frame time stamp;

Given the coordinates indication and frame time stamp as received, locating said point or object coordinates on a previously saved corresponding frame having a same time stamp, and using an image processing procedure, tracing at a rate higher than said periodical frame capturing said point or object coordinates from said frame towards the most recently available captured frame to find the coordinates of the same point or object as appearing in the most recently available captured frame; and

Providing the coordinates of the target point or object within the most recently available captured frame, as found, to an inner guidance sub-system of the vehicle, for enabling the vehicle to track said object.

2. A method according to claim 1, wherein each partial frame data represents an intermediate frame and each full frame data represents a reference frame.

3. A method according to claim 2 wherein at least one intermediate frame is produced between any two reference frames.

4. A method according to claim 2, wherein each partial frame data comprises a set of Global Motion Parameters indicating the global change of a given frame with respect to a previous reference frame, as calculated using a Global Motion Function.

5. A method according to claim 4 wherein a same Global Motion Function is used at the vehicle and at the control center.

6. A method according to claim 5 wherein each reference frame is displayed after being decompressed at the control center.

7. A method according to claim 5 wherein each intermediate frame is displayed at the control center after activating a Global Motion Function, using the Global Motion Parameters of same frame, as received, and data relating to a previous reference frame.

8. A method according to claim 7, wherein, when the operator marks a target on a displayed intermediate frame, the method further comprising backward activating a Global Motion Function to convert the marked coordinates to coordinates on a previous reference frame, before performing the tracing at the vehicle.

9. A method according to claim 8, wherein the activation of the backward Global Motion Function is carried out at the vehicle.

10. A method according to claim 8, wherein the activation of the backward Global Motion Function is carried out at the control center.

11. A method according to claim 8, wherein the tracing of the object coordinates from the reference frame towards the most recently available captured frame is carried out by cascade activating a Global Motion Function at the vehicle, staring from the object coordinates on a reference frame and applying the function on all newer saved frame Global Motion Parameters until determining said target coordinates to target coordinates on the most recently captured frame.

12. A method according to claim 4, for guiding the vehicle towards a static target, wherein only Global Motion Parameters of intermediate frames are saved at the vehicle.

13. A method according to claim 4, wherein the vehicle is guided towards a dynamic target, and wherein full data frames are saved at the vehicle.

14. A method according to claim 13, wherein the tracing of the object coordinates from the reference frame towards the most recently available captured frame is carried out by activating at a rate higher than said periodic frame capturing an object tracing utility at the vehicle, starting from the target coordinates on a reference frame and applying the utility on all newer saved frames until determining the coordinates of the target on the most recently captured frame.

15. A method according to claim 14, wherein the object tracing utility at the vehicle first identifies the marked object region on a reference frame by the coordinates as provided from the control center, and then uses correlation for determining the region movement in all newer saved frames until determining the coordinates of the target, being within said region, on the most recently captured frame.

16. A method according to claim 2, wherein reference frames are compressed at the vehicle before being sent to the control center, and decompressed at the control center.

17. A method according to claim 16 wherein at least one set of Global Motion Parameters is sent to the control center between any two compressed reference frames.

18. A method according to claim 1 wherein the inner guidance sub-system is a system capable of tracking an object to a point to a certain direction, or a combination thereof by a navigation sub-system, a tracker sub-system or a combination thereof.

19. A method according to claim 1 wherein the full frame data representing a reference frame comprises pixel representation data of the frame.

20. A method according to claim 19 wherein the full frame data further comprises at least one set of Global Motion Parameters indicating global change of the reference frame with respect to a previous reference frame.

21. A method according to claim 20 wherein the reference frame is reconstructed and displayed at the control center by either using said pixel representation data or by substituting one of said set of Global Motion Parameters indicating the global change of the reference frame with respect to a previous reference frame in a Global Motion Function.

22. A method according to claim 19 wherein the reference frame is reconstructed and displayed at the control center by using said pixel representation data.

23. A method according to claim 1 wherein the operator is a computerized system.

24. A method according to claim 1, wherein a Global Motion Parameters are used to construct frames as soon as the Global Motion Parameters arrive at the control center, to produce frames with minimum delay.

25. A method according to claim 1, wherein a reconstruction of a frame is based on more than one reference frame.