Position recognizing system of autonomous running vehicle

• US 6,025,790 A
• Filed: 08/03/1998
• Issued: 02/15/2000
• Est. Priority Date: 08/04/1997
• Status: Expired due to Term

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Saneyoshi, Keiji
• Primary Examiner(s): Wu, Daniel J.
• Assistant Examiner(s): Pham, Toan
• Application Number:
  US09/127,843
• Time in Patent Office: 561 Days
• Field of Search: 340/937, 340/946, 244/189, 244/190, 244/17.11, 348/117, 348/144
• US Class Current: 340/946
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

A camera assembly 10 images a distant landscape and a downward landscape every a predetermined period of time. A position recognizing unit 30 of a moving object (autonomous running vehicle) detects the movement of the moving object on the basis of the movement between the imaged pictures of the distant and downward landscapes, which are imaged at different imaging times. That is, the movements of the distant-view and down-view images are converted to the moving amounts in the actual space on the basis of the distance images thereof, respectively, and the component of rotational speed based on the movement of the distant-view image is removed from the component of velocity based on the movement of the down-view image to derive a pure component of translation speed. Then, the pure component of translation speed is converted to a component of translation viewed from the ranging starting point to be accumulated to derive a navigation locus in a three-dimensional space to recognize the moving-object'"'"'s own position. Thus, it is possible to analyze the surrounding environment in a three-dimensional space to cause the analyzed surrounding environment to be reflected on a navigation information, so that it is possible to obtain a precise and minute navigation information.

Claims

1. A position recognizing system of an autonomous running vehicle, comprising:

imaging means for imaging a distant landscape and a downward landscape every a predetermined period of time by an imaging system mounted on said autonomous running vehicle;

first capturing means for extracting a plurality of characteristic pattern regions serving as first blocks from a first frame of an imaged picture of each of said distant landscape and said downward landscape;

second capturing means for searching a second frame of an imaged picture of each of said distant landscape and said downward landscape for regions serving as second blocks having a same pattern as that of said first blocks, said second frame being imaged at the next timing to said first frame;

rotational processing means for deriving a component of rotational speed of said autonomous running vehicle between said first and second frames on the basis of a movement of said imaged picture of said distant landscape from said first blocks to said second blocks and on the basis of an elapsed time between said first and second frames;

translation processing means for deriving a component of velocity of said autonomous running vehicle between said first and second frames on the basis of a movement of said imaged picture of said downward landscape from said first blocks to said second blocks and on the basis of an elapsed time between said first and second frames and for removing said component of rotational speed from said component of velocity to derive a component of translation speed of said autonomous running vehicle between said first and second frames; and

navigation processing means for converting said component of translation speed between said first and second frames to a component of translation speed viewed from a ranging starting point and for accumulating the converted component of translation speed to derive a navigation locus in a three-dimensional space so as to accurately perform an autonomous navigation by processing a huge amount of image data at a fast speed without drifting.

2. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said imaging system is a stereo imaging system, and each of said rotational processing means and said translation processing means derives said movement of said imaged picture as a moving amount in actual space coordinates, which are obtained by converting imaging surface coordinates to actual distances on the basis of distance image corresponding to each of said distant landscape and said downward landscape by said stereo imaging system.

3. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said first capturing means extracts a small region in said imaged picture of said distant landscape as a proposed first block, which has a small in distance image value in a region corresponding to said distance image and which is sufficiently far from a ranging point.

4. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said first capturing means extracts a small plane region in said imaged picture of said downward landscape as a proposed first block, which has a small dispersion in distance image value in a region corresponding to said distance image.

5. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said first capturing means extracts said first blocks, evaluating a picture quality of an objective region by a histogram prepared using a distance image value of a corresponding region of said distance image.

6. The position recognizing system of an autonomous running vehicle according to claim 5, wherein said first capturing means prepares said histogram by counting a frequency larger than or equal to a predetermined threshold every extracting pixel.

7. The position recognizing system of an autonomous running vehicle according to claim 5, wherein said first capturing means evaluates said picture quality of said objective region, by determining whether a proportion of a total frequency of histogram values in a predetermined range to a distance image value recording the maximum histogram is higher than or equal to a predetermined proportion.

8. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said second capturing means extracts a region having a minimum city block distance between a predetermined search area surrounding each of said first blocks and each of said first blocks, as a proposed second block.

9. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said second capturing means extracts a region having a minimum city block distance between a predetermined search area surrounding predicted imaging surface coordinates for each of said second blocks and each of said first blocks, as a proposed second block, and said predicted imaging surface coordinates are predicted as imaging surface coordinates of a new position of each of said first blocks on the next frame on the basis of the last component of rotational speed or translation speed.

10. The position recognizing system of an autonomous running vehicle according to claim 2, wherein said stereo imaging system comprises a distant landscape imaging stereo camera and a downward landscape imaging stereo camera, and imaging surfaces of said stereo cameras are perpendicular to each other.

11. The position recognizing system of an autonomous running vehicle according to claim 10, wherein said distant landscape imaging stereo camera and said downward landscape imaging stereo camera have a reference camera, respectively, and said reference cameras of said stereo cameras are arranged so that said reference cameras are adjacent to each other.

12. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said navigation processing means converts said component of translation speed between said first and second frames to said component of translation speed viewed from said ranging starting point, correcting said component of translation speed between said first and second frames by a component of rotational speed representative of the current attitude of said autonomous running vehicle viewed from said ranging starting point.

13. The position recognizing system of an autonomous running vehicle according to claim 12, wherein said navigation processing means represents said current attitude of said autonomous running vehicle viewed from said ranging starting point as a matrix, which is derived by multiplying a matrix representative of the attitude of said autonomous running vehicle viewed from said ranging starting point until the last processing by a matrix representative of a component of rotational speed between the said first and second frames at the current processing, after a matrix representative of a component of rotational speed of said autonomous running vehicle at said ranging starting point is converted to a unit matrix to be initialized.

14. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said first capturing means extracts said first blocks from a plurality of search areas, which are obtained by diving said imaged picture by a predetermined range, so that a maximum of one of said first blocks is extracted every search area.

15. The position recognizing system of an autonomus running vehicle according to claim 1, wherein said first capturing means extracts m first blocks from n regions, which are obtained by dividing said imaged picture, so that the number of said first blocks extracted in each of said n regions is m/n at the maximum.

16. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said first capturing means extracts said first blocks from said imaged picture of said downward landscape within a scanning range which has a search starting line offset from a center line of said imaged picture by a predetermined amount in a traveling direction, and said search starting line is variable by said component of translation speed.

17. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said second capturing means extracts each of said second blocks from a predetermined search area surrounding each of said first blocks.

18. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said second capturing means extracts each of said second blocks from a predetermined search area surrounding predicted imaging surface coordinates for each of said second blocks, and said predicted imaging surface coordinates are predicted as imaging surface coordinates of a new position of each of said first blocks on the next frame on the basis of the last component of rotational speed or translation speed.

19. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said rotational processing means updates components of rotational speeds in the last processing as components of rotational speeds in the current processing, when said first capturing means and second capturing means do not capture predetermined pairs of said first and second blocks.

20. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said translation processing means updates components of translation speeds in the last processing as components of translation speeds in the current processing, when said first capturing means and second capturing means do not capture predetermined pairs of said first and second blocks.

21. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said first capturing means examines a difference between an arrangement pattern of said first blocks and an arrangement pattern of said second blocks to remove blocks having different arrangement.

22. The position recognizing system of an autonomous running vehicle according to claim 21, wherein said first capturing means examines said difference between said arrangement patterns by a distance between the respective blocks and an area of a triangle formed by the respective blocks.

23. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said second capturing means examines a difference between an arrangement pattern of said first blocks and an arrangement pattern of said second blocks to remove blocks having different arrangement.

24. The position recognizing system of an autonomous running vehicle according to claim 23, wherein said second capturing means examines said difference between said arrangement patterns by a distance between the respective blocks and an area of a triangle formed by the respective blocks.

25. The position recognizing system of an autonomous running vehicle according to claim 1, wherein said navigation processing means initializes said ranging starting point by an information obtained by a positioning system or a known map information, and recognizes an absolute position of said autonomous running vehicle by said navigation locus.