Method and system for detecting object on a road

• US 8,923,605 B2
• Filed: 11/09/2012
• Issued: 12/30/2014
• Est. Priority Date: 11/18/2011
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: You, Ganmei, Lu, Yaojie, Zheng, Jichuan
• Primary Examiner(s): Fitzpatrick, Atiba O
• Assistant Examiner(s): JAMES, THOMAS AIDEN
• Application Number:
  US13/673,199
• Publication Number:
  US 20130128001A1
• Time in Patent Office: 781 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Disclosed are a method and a system for detecting an object on a road. The method comprises a step of simultaneously capturing two depth maps, and then calculating a disparity map; a step of obtaining, based on the disparity map, a V-disparity image by adopting a V-disparity algorithm; a step of detecting an oblique line in the V-disparity image, and then removing points in the disparity map, corresponding to the oblique line so as to acquire a sub-disparity map excluding the road; a step of detecting plural vertical lines in the V-disparity image, and then extracting, for each of the plural vertical lines, points corresponding to this vertical line from the sub-disparity map as an object sub-disparity map corresponding to this vertical line; and a step of merging any two rectangular areas of the object sub-disparity maps approaching each other, into a rectangular object area.

Claims

1. A method of detecting at least one object on a road, comprising:

simultaneously capturing two depth maps of a current scene by adopting two depth cameras, and then calculating a disparity map of the current scene based on the two depth maps;

obtaining, based on the disparity map, a V-disparity image of the current scene by adopting a V-disparity algorithm;

detecting an oblique line in the V-disparity image, corresponding to the road in the disparity map, and then removing all of the points in the disparity map, corresponding to the oblique line detected in the V-disparity image, so as to acquire a sub-disparity map excluding the road;

detecting plural vertical lines in the V-disparity image, and then extracting, for each of the plural vertical lines, all of the points corresponding to this the vertical line from the sub-disparity map excluding the road, to serve as an object sub-disparity map corresponding to the vertical line; and

merging any two rectangular areas of the object sub-disparity maps, whose disparity distance is less than a distance value and whose overlap rate is greater than a threshold value, into a rectangular object area.

2. The method according to claim 1, wherein:

a Hough transform is utilized for detecting the oblique line in the V-disparity image, corresponding to the road in the disparity map.

3. The method according to claim 1, wherein:

detecting each of the plural vertical lines in the V-disparity image is conducted by seeking continuous points having a same gray level not equal to zero, in the V-disparity image.

4. The method according to claim 1, wherein, detecting the oblique line in the V-disparity image, corresponding to the road in the disparity map, and then removing all of the points in the disparity map, corresponding to the oblique line detected in the V-disparity image includes:

detecting the oblique line in the V-disparity image by adopting a Hough transform;

extending the oblique line so as to let it intersect with horizontal and vertical coordinate axes of the V-disparity image, respectively;

dividing, based on horizontal coordinate values of the oblique line, the oblique line into plural sub-line segments; and

removing, for each of the plural sub-line segments, all of the points in the disparity map, corresponding to the sub-line segment, wherein, all of the points are within a corresponding disparity range.

5. The method according to claim 1, further comprising:

performing, before carrying out the merging step, a point clustering process with regard to each of the object sub-disparity maps.

6. The method according to claim 5, further comprising:

before carrying out the point clustering process, for each of the object sub-disparity maps, conducting a smoothing process with regard to all of the points forming the object sub-disparity map, and then removing connection points in the object sub-disparity map.

11. The method of claim 1, wherein the merged rectangular object area is based on minimum coordinate values and maximum width and height values of the two rectangular areas.

12. The method of claim 1, wherein the removing all of the points in the disparity map is based on a disparity range associated with the points.

13. The method of claim 12, wherein the disparity range is associated with a plurality of disparity values, the disparity values being based on a focal length of one of the two depth cameras and a distance between the two depth cameras.

14. The method of claim 6, wherein the smoothing process is a median filtering method.

7. A system for detecting at least one object on a road, comprising:

two depth cameras configured to simultaneously capture two depth maps of a current scene so as to calculate a disparity map of the current scene based on the two depth maps;

a V-disparity map calculator configured to obtain, based on the disparity map, a V-disparity image of the current scene by adopting a V-disparity algorithm;

a road extractor configured to detect an oblique line in the V-disparity image, corresponding to the road in the disparity map, and then to remove all of the points in the disparity map, corresponding to the oblique line detected in the V-disparity image, so as to acquire a sub-disparity map excluding the road;

an object area detector configured to detect plural vertical lines in the V-disparity image, and then to extract, for each of the plural vertical lines, all of the points corresponding to the vertical line from the sub-disparity map excluding the road, to serve as an object sub-disparity map corresponding to the vertical line; and

an object area merging unit configured to merge any two rectangular areas of the object sub-disparity maps, whose disparity distance is less than a distance value and whose overlap rate is greater than a threshold value, into a rectangular object area.

8. The system according to claim 7, wherein:

the object area detector is configured to detect each of the plural vertical lines in the V-disparity image by seeking continuous points having a same gray level not equal to 0, in the V-disparity image.

9. The system according to claim 7, wherein, the road extractor is configured to detect the oblique line in the V-disparity image, corresponding to the road in the disparity map, and then remove all of the points in the disparity map, corresponding to the oblique line detected in the V-disparity image by:

detecting the oblique line in the V-disparity image by adopting a Hough transform;

extending the oblique line so as to let it intersect with horizontal and vertical coordinate axes of the V-disparity image, respectively;

dividing, based on horizontal coordinate values of the oblique line, the oblique line into plural sub-line segments; and

removing, for each of the plural sub-line segments, all of the points in the disparity map, corresponding to the sub-line segment, wherein, all of the points are within a corresponding disparity range.

10. The system according to claim 7, further comprising:

a noise processor configured to, for each of the object sub-disparity maps, conduct a smoothing process with regard to all of the points forming the object sub-disparity map, and then to remove connection points in the object sub-disparity map; and

a point clustering processor configured to perform a point clustering process with regard to each of the object sub-disparity maps after processed by the noise processor.

15. The system of claim 7, wherein the merged rectangular object area is based on minimum coordinate values and maximum width and height values of the two rectangular areas.

16. The system of claim 7, wherein the removing all of the points in the disparity map is based on a disparity range associated with the points.

17. The system of claim 16, wherein the disparity range is associated with a plurality of disparity values, the disparity values being based on a focal length of one of the two depth cameras and a distance between the two depth cameras.

18. The system of claim 10, wherein the smoothing process is a median filtering method.