Object detection apparatus, vehicle-mounted device control system and storage medium of program of object detection

• US 9,230,165 B2
• Filed: 02/28/2014
• Issued: 01/05/2016
• Est. Priority Date: 03/06/2013
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Guan, Haike
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): Delaney, Molly
• Application Number:
  US14/193,249
• Publication Number:
  US 20140254872A1
• Time in Patent Office: 676 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An object detection apparatus, using at least one processing circuit, for detecting an object in an image capturing area based on parallax information generated from a plurality of images captured by a plurality of image capturing units, includes a parallax histogram information generator to generate vertical-direction parallax histogram information indicating a frequency profile of parallax values in each of vertical row areas in a captured image based on the parallax information; and an object image area extraction unit to extract, among parallax values having frequency exceeding a given frequency threshold, a group of pixels having parallax values existing within proximity of a given parallax value and having a pixel-to-pixel interval in an image left-to-right direction within a given range as an object image area displaying an object based on the vertical-direction parallax histogram information.

Claims

1. An object detection apparatus, using at least one processing circuit, for detecting an object in an image capturing area based on parallax information generated from a plurality of images captured by a plurality of cameras, the object detection apparatus comprising:

a processor configured to,

generate vertical-direction parallax histogram information based on the parallax information, the vertical-direction parallax histogram information indicating a frequency profile of parallax values in each of vertical row areas in a captured image, the vertical row area preparable by dividing the captured image in a plurality of areas in a left-right direction,

extract, among parallax values having frequency exceeding a given frequency threshold, a group of pixels having parallax values existing within proximity of a given parallax value and having a pixel-to-pixel interval in an image left-to-right direction within a given range, as an object image area that includes an object for display, based on the vertical-direction parallax histogram information,

calculate a width and a height of the object based on the extracted object image area,

compute data corresponding to an actual horizontal size and vertical size of the object based on the parallax information at the object image area and the calculated width and height of the object, and

determine a type of the object based on the computed data by,

determining whether the actual horizontal size and vertical size are within a threshold width and a threshold height for one of a plurality of types of objects,

computing, for each of a plurality of blocks, a weighted feature value between the object image area of a respective one of the plurality of blocks and a luminance image, and

computing an evaluation value based on the weighted feature values for each of the plurality of blocks.

2. The object detection apparatus of claim 1, wherein the processor,

conducts Hough conversion to pixels having parallax values having frequency exceeding the given frequency threshold on a two dimensional coordinate system, defined by parallax values prepared from the vertical-direction parallax histogram information and image left-to-right direction positions, to extract a straight line extending in the image left-to-right direction, and

extracts the group of pixels having parallax values existing within proximity of a given parallax value and having a pixel-to-pixel interval in the image left-to-right direction within a given range using the extracted straight line as a reference.

3. The object detection apparatus of claim 1, wherein the processor extracts the group of pixels as an object image area displaying an object on a parallax image prepared based on the parallax information.

4. A vehicle-device control system comprising:

a plurality of cameras to capture an area outside a vehicle as an image capturing area;

the processor of claim 1 to detect an object in the image capturing area based on a plurality of captured images; and

a controller to control one or more devices equipped to the vehicle based on a detection result of the object detection apparatus.

5. A method of detecting an object in an image capturing area based on parallax information, the method comprising the steps of:

generating parallax information based on a plurality of captured images captured by a plurality of cameras;

generating vertical-direction parallax histogram information indicating a frequency profile of parallax values in each of vertical row areas in a captured image based on the parallax information, the vertical row area preparable by dividing the captured image in a plurality of areas in a left-right direction;

extracting, among parallax values having frequency exceeding a given frequency threshold, a group of pixels having parallax values existing within proximity of a given parallax value and having a pixel-to-pixel interval in an image left-to-right direction within a given range as an object image area displaying an object, based on the vertical-direction parallax histogram information calculating a width and a height of the object based on the extracted object image area;

computing data corresponding to an actual horizontal size and vertical size of the object based on the parallax information at the object image area and the calculated width and height of the object; and

determining a type of the object based on the computed data by,

determining whether the actual horizontal size and vertical size are within a threshold width and a threshold height for one of a plurality of types of objects,

computing, for each of a plurality of blocks, a weighted feature value between the object image area of a respective one of the plurality of blocks and a luminance image, and

computing an evaluation value based on the weighted feature values for each of the plurality of blocks.

6. A non-transitory computer-readable storage medium storing an object detection program that, when executed by a computer having a processing circuit, causes the computer to execute a method of detecting an object in an image capturing area based on parallax information, the method comprising the steps of:

generating parallax information based on a plurality of captured images captured by a plurality of cameras;

generating vertical-direction parallax histogram information indicating a frequency profile of parallax values in each of vertical row areas in a captured image based on the parallax information, the vertical row area preparable by dividing the captured image in a plurality of areas in a left-right direction;

extracting, among parallax values having frequency exceeding a given frequency threshold, a group of pixels having parallax values existing within proximity of a given parallax value and having a pixel-to-pixel interval in an image left-to-right direction within a given range as an object image area displaying an object, based on the vertical-direction parallax histogram information

calculating a width and a height of the object based on the extracted object image area;

computing data corresponding to an actual horizontal size and vertical size of the object based on the parallax information at the object image area and the calculated width and height of the object; and

determining a type of the object based on the computed data by,

determining whether the actual horizontal size and vertical size are within a threshold width and a threshold height for one of a plurality of types of objects,

computing, for each of a plurality of blocks, a weighted feature value between the object image area of a respective one of the plurality of blocks and a luminance image, and

computing an evaluation value based on the weighted feature values for each of the plurality of blocks.