Information processing device, imaging device, apparatus control system, information processing method, and computer program product

• US 10,762,656 B2
• Filed: 09/13/2017
• Issued: 09/01/2020
• Est. Priority Date: 11/30/2016
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Motohashi, Naoki, Takahashi, Sadao
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US16/347,959
• Publication Number:
  US 20190279386A1
• Time in Patent Office: 1,084 Days
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

An information processing device includes a disparity information input unit configured to acquire disparity image data indicating a disparity image including a disparity; a V map generating unit configured to generate a V map indicating a frequency of the disparity for each of coordinates of the disparity image by voting the disparity at a position corresponding to the coordinate of each of pixels of the disparity image based on the disparity image data; a shape estimating unit configured to estimate a road surface shape based on the V map; a height table generating unit configured to generate a height table indicating a relationship between the disparity and a height of a road surface based on the estimated road surface shape; and a correcting unit configured to correct, based on rolling information on rolling of a stereo camera that acquires the disparity, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling.

Claims

1. An information processing device comprising:

circuitry configured to

acquire distance image data indicating a distance image including distance information;

generate frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

estimate a road surface shape based on the frequency information;

generate height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

correct, based on rolling information on rolling of a distance information acquiring device that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling,

wherein the circuitry corrects the height information based on the rolling information, and

the circuitry adds or subtracts a rolling correction amount, which is calculated so as to cancel out a deviation that occurs between the estimated road surface shape and an actual road surface caused by the rolling, to or from the height included in the height information.

2. The information processing device according to claim 1, wherein the rolling correction amount includes a first correction amount to rotate the estimated road surface shape such that a slope of the estimated road surface shape matches a slope of an actual road surface, and a second correction amount to translate the estimated road surface shape such that coordinates of the estimated road surface match coordinates of the actual road surface.

3. The information processing device according to claim 1, wherein the circuitry adds or subtracts the rolling correction amount when a predetermined height is extracted and referred to from the height information.

4. The information processing device according to claim 1, wherein the circuitry corrects the frequency information based on the rolling information.

5. The information processing device according to claim 4, wherein the circuitry corrects a voting position of the distance information which is voted from the distance image to the frequency information.

6. The information processing device according to claim 1, wherein

the distance information acquiring device is a stereo camera,

the distance information is a disparity,

the distance image is a disparity image, and

the distance image data is disparity image data indicating the disparity image.

7. An imaging device comprising:

a stereo camera configured to generate the distance image data; and

the information processing device according to claim 1.

8. An apparatus control system comprising:

the information processing device according to claim 1; and

circuitry configured to control a behavior of an apparatus based on information indicating the road surface shape.

9. An information processing method comprising:

acquiring distance image data indicating a distance image including distance information;

generating frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

estimating a road surface shape based on the frequency information;

generating height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

correcting, based on rolling information on rolling of a distance information acquiring device that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling,

wherein the method includes correcting the height information based on the rolling information, and

adding or subtracting a rolling correction amount, which is calculated so as to cancel out a deviation that occurs between the estimated road surface shape and an actual road surface caused by the rolling, to or from the height included in the height information.

10. A computer program product comprising a non-transitory computer-readable recording medium having stored therein a program that causes a computer to execute a method comprising:

acquiring distance image data indicating a distance image including distance information;

generating frequency information indicating a frequency of the distance information for each of coordinates in the distance image by voting the distance information at a position corresponding to a coordinate of each of pixels of the distance image based on the distance image data;

estimating a road surface shape based on the frequency information;

generating height information indicating a relationship between the distance information and a height of a road surface based on the estimated road surface shape; and

correcting, based on rolling information on rolling of a distance information acquiring device that acquires the distance information, information used to generate information indicating the road surface shape so as to cancel out an influence of the rolling,

wherein the method includes correcting the height information based on the rolling information, and

adding or subtracting a rolling correction amount, which is calculated so as to cancel out a deviation that occurs between the estimated road surface shape and an actual road surface caused by the rolling, to or from the height included in the height information.