Obstacle recognition apparatus and method, obstacle recognition program, and mobile robot apparatus

• US 7,386,163 B2
• Filed: 03/13/2003
• Issued: 06/10/2008
• Est. Priority Date: 03/15/2002
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Okubo, Atsushi, Gutmann, Steffen, Sabe, Kohtaro, Kawamoto, Kenta, Ohashi, Takeshi, Fukuchi, Masaki
• Primary Examiner(s): Mehta, Bravesh M.
• Assistant Examiner(s): Seth, Manav
• Application Number:
  US10/387,647
• Publication Number:
  US 20040013295A1
• Time in Patent Office: 1,916 Days
• Field of Search: 382103-108, 382153-154, 348 42- 51, 348113-124, 348135-137, 348139-143, 348152-155, 700/245, 700/251, 700258-259, 701/23, 701/28, 701300-301
• US Class Current: 382/153
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An obstacle recognition apparatus is provided which can recognize an obstacle by accurately extracting a floor surface. It includes a distance image generator (222) to produce a distance image using a disparity image and homogeneous transform matrix, a plane detector (223) to detect plane parameters on the basis of the distance image from the distance image generator (222), a coordinate transformer (224) to transform the homogeneous transform matrix into a coordinate of a ground-contact plane of a robot apparatus (1), and a floor surface detector (225) to detect a floor surface using the plane parameters from the plane detector (223) and result of coordinate transformation from the coordinate transformer (224) and supply the plane parameters to an obstacle recognition block (226). The obstacle recognition block (226) selects one of points on the floor surface using the plane parameters of the floor surface detected by the floor surface detector (225) and recognizes an obstacle on the basis of the selected point.

Claims

1. An obstacle recognition apparatus comprising:

a distance image producing means for producing a distance image on the basis of a disparity image calculated based on image data supplied from a plurality of imaging means provided in a robot apparatus and sensor data outputs from a plurality of sensing means provided in the robot apparatus and using a homogeneous transform matrix corresponding to the disparity image at locations of the plurality of imaging means;

a plane detecting means for detecting plane parameters on the basis of the distance image produced by the distance image producing means;

a coordinate transforming means for transforming a coordinate system at the locations of the plurality of imaging means into a one on the floor surface;

a floor surface detecting means for detecting the floor surface on the basis of the plane parameters detected by the plane detecting means; and

an obstacle recognizing means for selecting a point on the floor surface using the plane parameters supplied from the plane detecting means to recognize an obstacle on the basis of the selected point, wherein the obstacle recognizing means uses the plane parameters of the floor surface detected by the floor surface detecting means to recognize an obstacle.

2. The apparatus as set forth in claim 1, wherein the floor surface detected by the floor surface detecting means and the obstacle recognized by the obstacle recognizing means are transformed into a plane coordinate whose height is zero to make a distinction between a floor surface and obstacle in a two-dimensional position in the plane.

3. An obstacle recognition method comprising steps of:

producing a distance image on the basis of a disparity image calculated based on image data supplied from a plurality of imaging means provided in a robot apparatus and sensor data outputs from a plurality of sensing means provided in the robot apparatus and using a homogeneous transform matrix corresponding to the disparity image at locations of the plurality of distance image producing means;

detecting plane parameters on the basis of the distance image produced in the distance image producing step;

transforming a coordinate system at the locations of the plurality of imaging means into a one on the floor surface;

detecting the floor surface on the basis of the plane parameters detected in the plane detecting step;

selecting a point on the floor surface using the plane parameters supplied from the detecting step to recognize an obstacle on the basis of the selected point; and

recognizing an obstacle based on the plane parameters of the floor surface.

4. A computer-readable medium adapted to store an obstacle recognition program, executed by a robot apparatus to recognize an obstacle, the program comprising steps of:

producing a distance image on the basis of a disparity image calculated based on image data and sensor data outputs;

using a homogeneous transform matrix corresponding to the disparity image;

detecting plane parameters on the basis of the distance image produced in the distance image producing step;

transforming a coordinate system into a one on the floor surface;

detecting the floor surface on the basis of the plane parameters detected in the plane detecting step;

selecting a point on the floor surface using the plane parameters supplied from the detecting step to recognize an obstacle on the basis of the selected point; and

utilizing the plane parameters of the floor surface to recognize an obstacle.

5. A mobile robot apparatus comprising:

a head unit having a plurality of imaging means and sensing means;

at least one moving leg unit having a sensing means;

a body unit having an information processing means and sensing, and that moves on a floor surface by the use of the moving leg unit while recognizing an obstacle on the floor surface,

wherein the body unit comprises;

a distance image producing means for producing a distance image on the basis of a disparity image calculated based on image data supplied from the plurality of imaging means provided in the head unit and sensor data outputs from the plurality of sensing means provided in the head, moving leg and body units, respectively,

wherein a homogeneous transform matrix corresponding to the disparity image at locations of the plurality of imaging means;

a plane detecting means for a detecting plane parameters on the basis of the distance image produced by the distance image producing means;

a coordinate transforming means for transforming a coordinate system at the locations of the plurality of imaging means into a one on the floor surface;

a floor surface detecting means for detecting the floor surface on the basis of the plane parameters detected by the plane detecting means; and

an obstacle recognizing means for selecting a point on the floor surface using the plane parameters supplied from the plane detecting means to recognize an obstacle on the basis of the selected point, wherein the obstacle recognizing means uses the plane parameters of the floor surface detected by the floor surface detecting means to recognize an obstacle.

6. The apparatus as set forth in claim 5, wherein the floor surface detected by the floor surface detecting means and the obstacle recognized by the obstacle recognizing means are transformed into a plane coordinate whose height is zero to make a distinction between a floor surface and obstacle in a two-dimensional position in the plane.