Light-directed ranging system implementing single camera system for telerobotics applications

• US 5,673,082 A
• Filed: 04/10/1995
• Issued: 09/30/1997
• Est. Priority Date: 04/10/1995
• Status: Expired due to Fees

• Current Assignee: National Aeronautics and Space Administration (Government of the United States of America)
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Cox, Brian J., Li, Larry C., Wells, Dennis L.
• Primary Examiner(s): Au, Amelia
• Application Number:
  US08/425,003
• Time in Patent Office: 904 Days
• Field of Search: 348/113, 348/114, 348/115, 348/118, 348/119, 348/158, 348/148, 348/135, 348/139, 348/142, 348/116, 348/348, 348/123, 348/136, 33/284, 33/227, 33/277, 33/280, 356/306, 356/307, 354/403, 382/103, 382/106, 382/153, 250/559.23, 345/8
• US Class Current: 348/139
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 7/493 Extracting wanted echo signals
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A laser-directed ranging system has utility for use in various fields, such as telerobotics applications and other applications involving physically handicapped individuals. The ranging system includes a single video camera and a directional light source such as a laser mounted on a camera platform, and a remotely positioned operator. In one embodiment, the position of the camera platform is controlled by three servo motors to orient the roll axis, pitch axis and yaw axis of the video cameras, based upon an operator input such as head motion. The laser is offset vertically and horizontally from the camera, and the laser/camera platform is directed by the user to point the laser and the camera toward a target device. The image produced by the video camera is processed to eliminate all background images except for the spot created by the laser. This processing is performed by creating a digital image of the target prior to illumination by the laser, and then eliminating common pixels from the subsequent digital image which includes the laser spot. A reference point is defined at a point in the video frame, which may be located outside of the image area of the camera. The disparity between the digital image of the laser spot and the reference point is calculated for use in a ranging analysis to determine range to the target.

Claims

1. A ranging system for telerobotics applications, comprising:

a single video camera having a lens;

a directional light source offset from said video camera, said directional light source selectively illuminating a portion of a selected target within the field of view of said video camera, said lens of said single video camera comprising means for forming an image of said illuminated portion of said selected target in the focal plane of said video camera;

a frame grabber for capturing a video frame generated by said video camera and containing said image of said illuminated portion of said selected target;

a robotic device; and

a processor electrically coupled to said video camera and said robotic device, said processor including a ranging algorithm for determining the range to said target based upon a comparison between the position of the image of said illuminated target portion formed by said lens of said video camera in the focal plane of said video camera and contained in said video frame, and the position of a reference point located in said focal plane.

2. A system as in claim 1, wherein said light source comprises a laser, and said video camera and said laser are mounted on a camera platform.

3. A system as in claim 2, wherein said camera platform is electrically coupled to said processor, and said processor positions said camera platform based upon electrical signals generated by a position and orientation sensor.

4. A system as in claim 3, wherein said position and orientation sensor generates signals indicative of the movement of an operator'"'"'s head.

5. A system as in claim 4, wherein the operator is positioned remotely from said video camera.

6. A system as in claim 2, further comprising a voice recognition system coupled to said processor, wherein said laser is activated by a voice command of an operator located remotely from said laser.

7. A system as in claim 6, wherein said laser is activated by a physically handicapped person.

8. A system as in claim 2, wherein said illuminated image includes a spot in said focal plane corresponding to said portion of the selected target.

9. A system as in claim 8, wherein said frame grabber generates a first image of said target with said spot, and said processor determines the disparity in said plane between the spot in said first image and the reference point in said plane.

10. A system as in claim 9, wherein said reference point is the center of said video frame.

11. A system as in claim 9, wherein said reference point is outside the center of said video frame.

12. A system as in claim 9, wherein said reference point is outside said video frame.

13. A system as in claim 5, further comprising a remote display for said operator, said display receiving images from said video camera through said processor.

14. A system as in claim 2, wherein said video camera and said laser are aimed in the same horizontal plane with respect to each other, for determining horizontal disparity.

15. An apparatus for determining range to a target, comprising:

a directional light source operable to selectively illuminate a portion of a target to selectively create an illuminated spot on said target;

a video camera for generating an image of said illuminated spot on a video frame; and

a computer operably associated with said video camera for determining the range to said target by comparing said image of said spot to a reference point located within the plane of said video frame to determine disparity.

16. An apparatus as in claim 15, wherein said light source is offset vertically and horizontally from said video camera.

17. An apparatus as in claim 16, wherein said computer obtains the image of said light source by eliminating common pixels between an image of said target without said illuminated spot and an image of said target with said spot.

18. An apparatus as in claim 17, wherein said computer turns off and on said directional light source.

19. An apparatus as in claim 18, further comprising a platform for directing the directional light source toward said target.

20. An apparatus as in claim 19, wherein said video camera is mounted on said platform, and said platform is positioned by an operator to direct said light source.

21. An apparatus as in claim 20, wherein the platform is positioned in response to the head movement of the operator.

22. An apparatus as in claim 20, wherein said platform is positioned in response to voice commands of operator.

23. An apparatus as in claim 21, further including an operator display receiving images from said video camera.

24. An apparatus as in claim 23, wherein said display is provided in a head-mounted unit.

25. An apparatus as in claim 24, further comprising a robotic device connected to said computer, wherein said operator activates said robotic device and said computer provides an electrical signal indicative of target location to said robotic device.

26. An apparatus as in claim 25, wherein said robotic device is activated by a voice command from the operator.

27. An apparatus as in claim 26, further comprising a voice recognition system connected to said computer, said voice recognition system converting said voice command to an electrical output signal.

28. A method for determining range to a target using a single video camera, comprising the steps of:

(a) capturing a first image of a target;

(b) activating a directional light source to form an illuminated spot on a portion of said target;

(c) capturing a second image of the target while the light source is directed onto the portion of said target;

(d) generating a third image of the target by eliminating common elements of the first and second set of images; and

(e) determining range to the target based upon remaining elements in the third image.

29. A method as in claim 28, wherein the range to the target is determined by calculating the disparity between the remaining elements of the third image with a reference point in a video frame of said video camera.

30. A method as in claim 29, wherein said first and said second images are digitized for generating the third image.

31. A method as in claim 28, wherein the step of determining the range to the target includes the step of determining the z component of the range value r_z.

32. A method as in claim 31, wherein two r_z values (r_z.sbsb.x and r_z.sbsb.y) are determined.

33. A method as in claim 32, wherein the r_z value is determined by averaging r_z.sbsb.x and r_z.sbsb.y.

34. A method for remotely controlling a robotic device to manipulate a target device, comprising the steps of:

(a) remotely viewing the target device;

(b) capturing a video image of said target device;

(c) activating a laser to generate a spot on said target device;

(d) capturing a video image of said target device with said spot thereon;

(e) eliminating common pixels between said video image captured in step (b) and said video image captured in step (d) to obtain a video image of said spot in a focal plane of the video camera;

(f) determining disparity between said video image obtained in step (e) and a reference point in said focal plane;

(g) calculating range to said target device based upon said disparity determination;

(h) positioning said robotic device to contact said target device based upon said range calculation.

35. A method as in claim 34, wherein the step of remotely viewing the target device includes positioning a video camera to view the target device.

36. A method as in claim 35, wherein the positioning of said camera is performed by moving the head of an operator.

37. A method as in claim 34, wherein the video images are captured by a frame grabber circuit.

38. A method as in claim 34, wherein said laser is positioned by an operator and activated by said operator.

39. A method as in claim 34, wherein said laser is activated by a computer after said target device is selected by an operator.

40. A method as in claim 34, wherein the reference point represents the coordinates in the focal plane of the camera for an illuminated spot at infinite range.

41. A method as in claim 34, wherein the disparity includes a horizontal and vertical component.