Environment recognizing device, environment recognizing method, route planning device, route planning method and robot

• US 20050131581A1
• Filed: 09/16/2004
• Published: 06/16/2005
• Est. Priority Date: 09/19/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sabe, Kohtaro, Gutmann, Steffen
• Granted Patent: US 7,865,267 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 2201/0217 Anthropomorphic or bipedal robot
• CPC Class Codes: G06V 20/36 Indoor scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

An environment recognizing device and an environment recognizing method can draw an environment map for judging if it is possible to move a region where one or more than one steps are found above or below a floor, a route planning device and a route planning method that can appropriately plan a moving route, using such an environment map and a robot equipped with such an environment recognizing device and a route planning device. The robot comprises an environment recognizing section including a plurality of plane extracting section 401 adapted to compute plane parameters from a parallax image or a distance image and extract a plurality of planes including the floor surface, an obstacle recognizing section 402 adapted to recognize obstacles on the plurality of planes including the floor surface and an environment map updating section 403 adapted to draw an environment map (obstacle map) for each of the planes on the basis of the result of recognition of the obstacle recognizing section 402 and update the existing environment maps and a route planning section 404 adapted to plan a route on the basis of the environment maps. The route planning section 404 selects a plane as route coordinate when an obstacle is found on it in the environment map of the floor surface but not found in the environment map of the plane.

Claims

1. An environment recognizing device to be used by a traveling type robot in an environment containing one or more than one obstacles in order to recognize the environment, the device comprising:

plane detecting means for computing a plane parameter on the basis of input images and detecting a plane on the basis of the plane parameter;

obstacle recognizing means for recognizing an obstacle, if any, on the plane detected by the plane detecting means; and

environment map updating means for drawing an environment map and/or update the existing environment map on the basis of information on the obstacle recognized by the obstacle recognizing means;

the plane detecting means being adapted to detect a group of planes including a reference plane to be used as reference and one or more than one other planes substantially parallel to the reference plane;

the environment map updating means being adapted to draw an environment map and/or update the existing environment map for the plane carrying an obstacle as recognized by the obstacle recognizing means out of the group of planes.

2. The device according to claim 1, wherein the reference plane is a floor surface.

3. The device according to claim 1, wherein the reference plane is the plane on which the robot is moving.

4. The device according to claim 1, wherein each of the planes has a surface area that allows the robot to move thereon.

5. The device according to claim 1, wherein the planes other than the reference plane are planes showing a height difference not greater than a predetermined value from the reference plane.

6. The device according to claim 1, wherein the planes other than the reference plane include the plane that shows the smallest height difference from the reference plane within the scope of recognition of the robot.

7. The device according to claim 1, wherein the environment maps include obstacle regions where an obstacle is found, free space regions where the robot can move and unsurveyed regions.

8. The device according to claim 1, wherein the plane detecting means includes plane computing means for sampling three-dimensional data on three or more than three points out of the group of three-dimensional data obtained by converting the parallax image or the distance image input as the input image and plane determining means for polling the plurality of plane parameters obtained from the plane computing means for a polling space and determining the group of planes on the basis of the result of the polling.

9. The device according to claim 1, further comprising:

distance image generating means for generating a distance image by using the parallax image computationally obtained on the basis of the image data obtained from a plurality of imaging means arranged in the robot and a homogeneous transformation matrix corresponding to the parallax image at the positions where the plurality of imaging means are arranged; and

the plane detecting means being adapted to compute the plane parameters on the basis of the distance image.

10. The device according to claim 3, wherein the obstacle recognizing means is adapted to recognize an obstacle at least on the plane where the robot is moving out of the group of planes and the environment map updating means is adapted to draw an environment map of the plane where the robot is moving and/or update the existing environment map of the plane.

11. An environment recognizing method to be used by a traveling type robot in an environment containing one or more than one obstacles in order to recognize the environment, the method comprising:

a plane detecting step of computing a plane parameter on the basis of input images and detecting a plane on the basis of the plane parameter;

an obstacle recognizing step of recognizing an obstacle, if any, on the plane detected in the plane detecting step; and

an environment map updating step of drawing an environment map and/or update the existing environment map on the basis of information on the obstacle recognized in the obstacle recognizing step;

the plane detecting step being adapted to detect a group of planes including a reference plane to be used as reference and one or more than one other planes substantially parallel to the reference plane;

the environment map updating step being adapted to draw an environment map and/or update the existing environment map for the plane carrying an obstacle as recognized in the obstacle recognizing step out of the group of planes.

12. The method according to claim 11, wherein the reference plane is a floor surface.

13. The method according to claim 11, wherein the reference plane is the plane on which the robot is moving.

14. The method according to claim 11, wherein each of the planes has a surface area that allows the robot to move thereon.

15. The method according to claim 11, wherein the planes other than the reference plane are planes showing a height difference not greater than a predetermined value from the reference plane.

16. The method according to claim 11, wherein the planes other than the reference plane include the plane that shows the smallest height difference from the reference plane within the scope of recognition of the robot.

17. A route planning device to be used by a traveling type robot in an environment containing one or more than one obstacles in order to plan a traveling route to be followed to move from the current position to a target position, the device comprising:

plane detecting means for computing a plane parameter on the basis of input images and detecting a reference plane to be used as reference on the basis of the plane parameter and a group of planes including one or more than one planes other than the reference plane running substantially in parallel with the reference plane;

obstacle recognizing means for recognizing an obstacle, if any, on each of the group of planes detected by the plane detecting means;

environment map updating means for drawing an environment map and/or update the existing environment map for each of the planes of the group of planes on the basis of information on the obstacle(s) recognized by the obstacle recognizing means; and

route planning means for planning a moving route for the robot on the basis of the environment maps drawn and/or updated by the environment map updating means.

18. The device according to claim 17, wherein the route planning means is adapted to divide each environment map into regions in a predetermined manner, compute the cost necessary for moving in each of the regions to be passed through when moving from the current position of the robot to a target position and plan a moving route on the basis of the computed cost.

19. The device according to claim 17, wherein

the environment maps include obstacle regions containing one or more than one obstacles, free space regions allowing free movements and unsurveyed regions and;

if a first environment map corresponding to the currently traveling plane contains one or more than one obstacle regions, the route planning means refers to a second environment map corresponding to a plane different from the currently traveling plane and, if the obstacle region is a free space region in the second environment map, the obstacle region is regarded as movable region.

20. A route planning method to be used by a traveling type robot in an environment containing one or more than one obstacles in order to plan a traveling route to be followed to move from the current position to a target position, the method comprising:

a plane detecting step of computing a plane parameter on the basis of input images and detecting a reference plane to be used as reference on the basis of the plane parameter and a group of planes including one or more than one planes other than the reference plane running substantially in parallel with the reference plane;

an obstacle recognizing step of frecognizing an obstacle, if any, on each of the group of planes detected in the plane detecting step;

an environment map updating step of drawing an environment map and/or update the existing environment map for each of the planes of the group of planes on the basis of information on the obstacle(s) recognized in the obstacle recognizing step; and

a route planning step of planning a moving route for the robot on the basis of the environment maps drawn and/or updated in the environment map updating step.

21. The method according to claim 20, wherein the route planning step is adapted to divide each environment map into regions in a predetermined manner, compute the cost necessary for moving in each of the regions to be passed through when moving from the current position of the robot to a target position and plan a moving route on the basis of the computed cost.

22. The method according to claim 20, wherein

the environment maps include obstacle regions containing one or more than one obstacles, free space regions allowing free movements and unsurveyed regions and;

if a first environment map corresponding to the currently traveling plane contains one or more than one obstacle regions, a second environment map corresponding to a plane different from the currently traveling plane is referred to in the route planning step and, if the obstacle region is a free space region in the second environment map, the obstacle region is regarded as movable region.

23. A robot adapted to move autonomously in an environment containing one or more than one obstacles and having an environment recognizing device for recognizing the environment thereof;

the environment recognizing device comprising;

plane detecting means for computing a plane parameter on the basis of input images and detecting a plane on the basis of the plane parameter;

obstacle recognizing means for recognizing an obstacle, if any, on the plane detected by the plane detecting means; and

environment map updating means for drawing an environment map and/or update the existing environment map on the basis of information on the obstacle recognized by the obstacle recognizing means;

the plane detecting means being adapted to detect a group of planes including a reference plane to be used as reference and one or more than one other planes substantially parallel to the reference plane;

the environment map updating means being adapted to draw an environment map and/or update the existing environment map for the plane carrying an obstacle as recognized by the obstacle recognizing means out of the group of planes.

24. A robot adapted to autonomously plan a traveling route to be followed to move from the current position to a target position in an environment containing one or more than one obstacles and having a route planning device to be used to recognize the environment and plan a traveling route, the route planning device comprising:

plane detecting means for computing a plane parameter on the basis of input images and detecting a reference plane to be used as reference on the basis of the plane parameter and a group of planes including one or more than one planes other than the reference plane running substantially in parallel with the reference plane;

obstacle recognizing means for recognizing an obstacle, if any, on each of the group of planes detected by the plane detecting means;

environment map updating means for drawing an environment map and/or update the existing environment map for each of the planes of the group of planes on the basis of information on the obstacle(s) recognized by the obstacle recognizing means; and

route planning means for planning a moving route for the robot on the basis of the environment maps drawn and/or updated by the environment map updating means.