INFORMATION PROCESSING DEVICE, IMAGING DEVICE, APPARATUS CONTROL SYSTEM, INFORMATION PROCESSING METHOD, AND COMPUTER PROGRAM PRODUCT

• US 20190279386A1
• Filed: 09/13/2017
• Published: 09/12/2019
• Est. Priority Date: 11/30/2016
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: MOTOHASHI, Naoki, TAKAHASHI, Sadao
• Granted Patent: US 10,762,656 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

An information processing device includes a disparity information input unit configured to acquire disparity image data indicating a disparity image including a disparity; a V map generating unit configured to generate a V map indicating a frequency of the disparity for each of coordinates of the disparity image by voting the disparity at a position corresponding to the coordinate of each of pixels of the disparity image based on the disparity image data; a shape estimating unit configured to estimate a road surface shape based on the V map; a height table generating unit configured to generate a height table indicating a relationship between the disparity and a height of a road surface based on the estimated road surface shape; and a correcting unit configured to correct, based on rolling information on rolling of a stereo camera that acquires the disparity, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling.

Claims

1. An information processing device comprising:

an acquiring unit configured to acquire distance image data indicating a distance image including distance information;

a first generating unit configured to generate frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

an estimating unit configured to estimate a road surface shape based on the frequency information;

a second generating unit configured to generate height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

a correcting unit configured to correct, based on rolling information on rolling of a distance information acquiring unit that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling.

2. The information processing device according to claim 1, wherein the correcting unit corrects the height information based on the rolling information.

3. The information processing device according to claim 2, wherein the correcting unit adds or subtracts a rolling correction amount, which is calculated so as to cancel out a deviation that occurs between the estimated road surface shape and an actual road surface caused by the rolling, to or from the height included in the height information.

4. The information processing device according to claim 3, wherein the rolling correction amount includes a first correction amount to rotate the estimated road surface shape such that a slope of the estimated road surface shape matches a slope of an actual road surface, and a second correction amount to translate the estimated road surface shape such that coordinates of the estimated road surface match coordinates of the actual road surface.

5. The information processing device according to claim 3, wherein the correcting unit adds or subtracts the rolling correction amount when a predetermined height is extracted and referred to from the height information.

6. The information processing device according to claim 1, wherein the correcting unit corrects the frequency information based on the rolling information.

7. The information processing device according to claim 6, wherein the correcting unit corrects a voting position of the distance information which is voted from the distance image to the frequency information.

8. The information processing device according to claim 1, wherein

the distance information acquiring device is a stereo camera,

the distance information is a disparity,

the distance image is a disparity image, and

the distance image data is disparity image data indicating the disparity image.

9. An imaging device comprising:

a stereo camera configured to generate the distance image data; and

the information processing device according to claim 1.

10. An apparatus control system comprising:

the information processing device according to claim 1; and

an apparatus control unit configured to control a behavior of an apparatus based on information indicating the road surface shape.

11. An information processing method comprising:

acquiring distance image data indicating a distance image including distance information;

generating frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

estimating a road surface shape based on the frequency information;

generating height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

correcting, based on rolling information on rolling of a distance information acquiring unit that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling.

12. A computer program product comprising a non-transitory computer-readable recording medium having stored therein a program that causes a computer to execute:

acquiring distance image data indicating a distance image including distance information;

generating frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

estimating a road surface shape based on the frequency information;

generating height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

correcting, based on rolling information on rolling of a distance information acquiring unit that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling.