Position detection apparatus, position detection method and position detection program

• US 7,239,339 B2
• Filed: 05/26/2001
• Issued: 07/03/2007
• Est. Priority Date: 05/26/2000
• Status: Active Grant

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Sakagami, Yoshiaki, Nagai, Takaaki, Matsunaga, Shinichi
• Primary Examiner(s): Philippe; Gims
• Assistant Examiner(s): REKSTAD, ERICK J
• Application Number:
  US09/866,151
• Publication Number:
  US 20010055063A1
• Time in Patent Office: 2,229 Days
• Field of Search: 701/28, 701/44, 701/207, 382/153, 382/106, 348/135, 348/139, 348/114, 348/116, 348/118
• US Class Current: 348/116
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves

Abstract

A position detection apparatus that is able to easily detect self-position during autonomous movement by a humanoid robot that moves by the use of legs or automobile. The position detection apparatus detects the position of a moving object, said position detection apparatus being provided with a brightness image acquisition device that acquires a brightness image of the forward field of view of the moving object, a distance image acquisition device having the same field of view as the brightness image acquisition device that acquires a distance image simultaneous to acquisition of a brightness image by the brightness image acquisition device, a characteristic point extraction device that extracts respective characteristic points from the brightness images of at least two consecutive frames, and a reference characteristic point selection device that calculates the amount of displacement of a position between two frames of a characteristic point extracted by the characteristic point extraction device based on a distance image, and selects a reference characteristic point for calculating self-position according to said amount of displacement.

Claims

1. A position detection apparatus for detecting a position of a moving robot, said position detection apparatus comprising:

a local image acquisition device for acquiring an image of a forward field of view of said moving robot;

a distance image acquisition device, having the same field of view as said local image acquisition device, said distance image acquisition device operable to acquire a distance image simultaneous to acquisition of an image by said local image acquisition device;

a characteristic point extraction device that extracts respective characteristic points from the images by a specific method, wherein the characteristic points are created using two consecutively incorporated images, and

a reference characteristic point selection device that selects a reference characteristic point for calculating the position of the moving robot, based on the characteristic points and the distance image.

2. A position detection apparatus according to claim 1, wherein the characteristic point extraction device extracts respective characteristic points from the images of at least two consecutive frames, and

wherein the reference characteristic point selection device calculates the amount of displacement of a position between two frames of a characteristic point extracted by said characteristic point extraction device based on said distance image, and selects a reference characteristic point for calculating position according to said amount of displacement.

3. A position detection apparatus according to claim 1, wherein

the characteristic point extraction device includes a position detection device that detects position by substituting self-movement control and the observed amount of said reference point into an extended Kalman filter, and

the reference characteristic point selection device includes a reference point determination device that determines a reference characteristic point to serve as a reference during movement of said moving robot based on an image obtained from said local image acquisition device.

4. A position detection apparatus according to claim 1, wherein the reference characteristic point selection device compares pre-stored object information with extracted characteristic points, and considers those characteristic points having a high correlation to be known characteristic points that are used as reference characteristic points for calculating position.

5. The position detection apparatus according to claim 4, wherein said reference characteristic point selection device updates said object information by determining the relative relationship between unknown characteristic points and known characteristic points in an image in which characteristic points considered to be known are present, and storing said unknown characteristic points as known characteristic points.

6. The position detection apparatus according to claim 1, wherein said characteristic point extraction device comprises:

a characteristic point group extraction device that extracts a characteristic point group in said image, and

a position detection device that calculates position by correlating and storing multiple characteristic point groups in an image pre-obtained with said image acquisition device with positions at which said characteristic point groups are obtained, and calculating the correlation between a characteristic point group of a newly obtained image and pre-stored characteristic point groups.

19. The position detection apparatus of claim 1, wherein said position detection apparatus further comprises two cameras, and wherein

the image acquired by the local image acquisition device comprises a brightness image in which pixels in the image acquired by the local image acquisition device represent relative brightness, and

the distance image acquired by the distance imaging device is formed by determining corresponding points of the pixels of each image from two brightness images obtained with the cameras such that pixels in the distance image represent a distance from the cameras.

7. A position detection method for detecting a position of a moving robot, said position detection method comprising:

a local image acquisition step of acquiring an image of a forward field of view of said moving robot;

a distance image acquisition step that, in the same field of view as said local image acquisition step, acquires a distance image simultaneous to acquisition of an image by said local image acquisition step;

a characteristic point extraction step that extracts respective characteristic points from the images by a specific method, wherein the characteristic points are created using two consecutively incorporated brightness images, and

a reference characteristic point selection step that selects a reference characteristic point for calculating the position of the moving robot, based on the characteristic points and the distance image.

8. The position detection method according to claim 7, wherein the characteristic point extraction step extracts respective characteristic points from the images of at least two consecutive frames, and

wherein the reference characteristic point selection step calculates the amount of displacement of a position between two frames of a characteristic point extracted by said characteristic point extraction step based on said distance image, and selects a reference characteristic point for calculating position according to said amount of displacement.

9. The position detection method according to claim 7, wherein the characteristic point extraction step comprises a position detection step that detects position by substituting self-movement control and the observed amount of said reference point into an extended Kalman filter, and

the reference characteristic point selection step comprises a reference point determination step that determines a reference characteristic point to serve as a reference during movement of said moving robot based on an image obtained from said local image acquisition step.

10. The position detection method according to claim 7, wherein the reference characteristic point selection step compares pre-stored object information with extracted characteristic points, and considers those characteristic points having a high correlation to be known characteristic points that are used as reference characteristic points for calculating position.

11. The position detection method according to claim 10, wherein said reference characteristic point selection step updates said object information by determining the relative relationship between unknown characteristic points and known characteristic points in an image in which characteristic points considered to be known are present, and storing said unknown characteristic points as known characteristic points.

12. The position detection method according to claim 7, wherein said characteristic point extraction step comprises:

a characteristic point group extraction step that extracts a characteristic point group in said image, and

a position detection step that calculates position by correlating and storing multiple characteristic point groups in an image pre-obtained with said image acquisition step with positions at which said characteristic point groups are obtained, and calculating the correlation between a characteristic point group of a newly obtained image and pre-stored characteristic point groups.

20. The position detection method according to claim 7, wherein said position detection apparatus further comprises two cameras, and wherein

the image acquired by the local image acquisition device comprises a brightness image in which pixels in the image acquired by the local image acquisition device represent relative brightness, and

the distance image acquired by the distance imaging device is formed by determining corresponding points of the pixels of each image from two brightness images obtained with the cameras such that pixels in the distance image represent a distance from the cameras.

13. A position detection program for detecting position of a moving robot, said position detection program comprising performing by computer:

a local image acquisition processing of acquiring an image of a forward field of view of said robot;

a distance image acquisition processing that, in the same field of view as said local image acquisition processing, acquires a distance image simultaneous to acquisition of an image by said local image acquisition processing;

a characteristic point extraction processing that extracts respective characteristic points from the images by a specific program, wherein the characteristic points are created using two consecutively incorporated brightness images, and

a reference characteristic point selection processing that selects a reference characteristic point for calculating the position of the moving robot, based on the characteristic points and the distance image.

14. The position detection program according to claim 13, wherein the characteristic point extraction processing extracts respective characteristic points from images of at least two consecutive frames, and

wherein the reference characteristic point selection processing calculates the amount of displacement of a position between two frames of a characteristic point extracted by said characteristic point extraction processing based on said distance image, and selects a reference characteristic point for calculating position according to said amount of displacement.

15. The position detection program according to claim 13, wherein the characteristic point extraction processing comprises a position detection processing that detects position by substituting self-movement control and the observed amount of said reference point into an extended Kalman filter, and

the reference characteristic point selection processing comprises a reference point determination processing that determines a reference characteristic point to serve as a reference during movement of said moving robot based on an image obtained from said local image acquisition processing.

16. The position detection program according to claim 13, wherein the reference characteristic point selection processing updates said object information by determining the relative relationship between unknown characteristic points and known characteristic points in an image in which characteristic points considered to be known are present, and storing said unknown characteristic points as known characteristic points.

17. The position detection program according to claim 16, wherein said reference characteristic point selection processing updates said object information by determining the relative relationship between unknown characteristic points and known characteristic points in an image in which characteristic points considered to be known are present, and storing said unknown characteristic points as known characteristic points.

18. The position detection program according to claim 13, wherein said characteristic point extraction processing comprises:

a characteristic point group extraction processing that extracts a characteristic point group in said image, and

a position detection processing that calculates position by correlating and storing multiple characteristic point groups in an image pre-obtained with said image acquisition processing with positions at which said characteristic point groups are obtained, and calculating the correlation between a characteristic point group of a newly obtained image and pre-stored characteristic point groups.

21. The position detection program according to claim 13, wherein said position detection apparatus further comprises two cameras, and wherein

the image acquired by the local image acquisition device comprises a brightness image in which pixels in the image acquired by the local image acquisition device represent relative brightness, and

the distance image acquired by the distance imaging device is formed by determining corresponding points of the pixels of each image from two brightness images obtained with the cameras such that pixels in the distance image represent a distance from the cameras.

22. A position detection apparatus for detecting a position of a moving robot, said position detection apparatus comprising:

a local image acquisition device for acquiring an image of a forward field of view of said moving robot;

a distance image acquisition device, having the same field of view as said local image acquisition device, said distance image acquisition device operable to acquire a distance image simultaneous to acquisition of an image by said local image acquistion device;

a characteristic point extraction device that extracts respective characteristic points from the images by a specific method, wherein the characteristic point extraction device extracts respective characteristic points from the images of at least two consecutive frames, and

a reference characteristic point selection device that selects a reference characteristic point for calculating the position of the moving robot, based on the characteristic points and the distance image,

wherein the reference characteristic point selection device calculates the amount of displacement of a position between two frames of a characteristic point extracted by said characteristic point extraction device based on said distance image, and selects a reference characteristic point for calculating position according to said amount of displacement.