Remote-controlled robot and robot self-position identification method

• US 7,120,519 B2
• Filed: 11/02/2004
• Issued: 10/10/2006
• Est. Priority Date: 05/31/2002
• Status: Expired due to Term

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shimizu, Katsutoshi, Okabayashi, Miwa, Okabayashi, Keiju, Murase, Yuichi, Iwashita, Sumihisa, Sawasaki, Naoyuki
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): Marc, McDieunel
• Application Number:
  US10/978,439
• Publication Number:
  US 20050071047A1
• Time in Patent Office: 707 Days
• Field of Search: 700/245, 700/258, 318/658.1, 318/568.11, 701/23
• US Class Current: 700/254
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0229 in combination with fixed guiding means
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A remotely controlled robot comprises a unit storing the layout plan of a building, a unit receiving a position remotely designated in the layout plan from a remote location and a unit controlling the travel of the robot to the designated position. A self-position identification method is implemented by a robot with a camera whose shooting direction can be changed. The robot takes in advance a panoramic picture of a room where the robot may travel, generates a reference picture by extracting a plurality of block pictures from the panoramic picture and identifies a room where the robot is located, by applying correlation and DP matching, using both a picture taken in the room where the robot is located and the reference picture.

Claims

1. A robot that can be remotely controlled, comprising:

a layout storage unit for storing a layout plan of a building;

a communication unit for communicating with a remote terminal; and

a travel control unit for controlling travel of a robot to a designated position, wherein;

the robot identifies its own position, the robot receives a viewable picture from the remote terminal;

a size of the layout picture stored in the memory is converted into a size of the viewable picture received from the remote terminal, and stored;

the robot transmits the layout picture containing the identified own position to the remote terminal;

the robot receives coordinates of a destination designated from the remote terminal;

the robot calculates destination coordinates on a real scale from both a converted size and coordinates of the destination designated by the remote terminal;

the robot produces a route to the destination; and

the robot travels towards the destination.

2. The remotely controlled robot according to claim 1, further comprising

a step getting over unit for getting over steps in a building, based on the result of step detection by an obstacle detecting sensor.

3. The remotely controlled robot according to claim 1, further comprising

a self-position identification unit for identifying the current position and direction of the robot in the building, wherein

said travel control unit controls the travel of the robot to the designated position, based on the result of the identification.

4. A robot that can be remotely controlled, comprising:

a layout storage unit for storing a layout plan of a building;

a communication unit for communicating with a remote terminal; and

a travel control unit for controlling the travel of a robot to the designated room, wherein;

the robot identifies its own position;

the robot received a viewable picture from the remote terminal;

the robot transmits the layout picture containing the identified its own position to the remote terminal;

the robot receives data of name or number of the designated room from the remote terminal;

the robot calculates the coordinates of a destination from the designated room using a correspondence table storing in advance the coordinates of a target, such as an entrance for each room;

the robot produces a route to the destination; and

the robot travels towards the destination.

5. A robot that can be remotely controlled, comprising:

a picture taking unit for taking an inside picture of a building;

a communication unit for transmitting the picture taken by the picture taking unit when the robot regularly/irregularly patrols inside a building, to a computer with memory accessible from the outside through a network;

a step getting over unit for getting over steps in a building, based on a result of step detection by an obstacle detecting sensor;

a rotation unit for changing a shooting direction of the picture taking unit; and

an infrared ray emission/reception unit for emitting/receiving an infrared ray in parallel to the shooting direction of the picture taking unit.

6. The remotely controlled robot according to claim 5, further comprising

a step getting over unit for getting over steps in a building, based on the result of step detection by an obstacle detecting sensor.

7. The remotely controlled robot according to claim 5, further comprising

a rotation unit for changing the shooting direction of the picture taking unit;

and

an infrared ray emission/reception unit for emitting/receiving an infrared ray in parallel to the shooting direction of the picture taking unit.

8. The remotely controlled robot according to claim 5, wherein

the computer to which said communication unit transmits the picture is a Web server connected to the Internet.

9. The remotely controlled robot, according to claim 5, wherein said communication unit starts the computer when a ringer signal is transmitted from the outside through the Internet a prescribed number of times.

10. The robot that can be remotely controlled, according to claim 5, further comprising:

a script storage unit for storing the programs of scripts for one or more operations stored in the script storage unit, wherein

said communication unit received the command to execute one of the programs stored in the script storage unit.

11. A robot that can be remotely controlled, comprising:

a command receiving unit for receiving an emergency command from a person in a building;

a communication unit for issuing the emergency command to a predetermined remote terminal according to the command;

a picture taking unit for taking an inside picture of a building;

a rotation unit for changing a shooting direction of the picture taking unit; and

a communication unit for transmitting a picture taken by the picture taking unit to the outside, according to a command on a shooting direction from the outside.

12. A self-position identification method of a robot with a camera whose shooting direction can be changed, comprising of:

taking in advance a panoramic picture of each room to which the robot may travel;

generating a reference picture by extracting a plurality of block pictures from the panoramic picture; and

identifying a room where the robot is located, by applying correlation calculation and DP matching, using both a picture of a same size as the block picture in the panoramic picture taken in a position where the robot is located, and the reference picture, wherein

designating the picture of a landmark, taken in advance, of all the landmarks in the identified room, as a reference template, applying a correlation calculation between the reference template and a picture taken in a position where the robot is located, and calculating a position of the landmark in the taken picture; and

stereoscopically measuring a distance between the robot and each of a plurality of landmarks and identifying the own robot'"'"'s position and direction in the identified room.

13. The robot self-position identification method according to claim 12, wherein

extracting a vertical line in the taken picture of the landmark;

calculating cost corresponding to a difference between an angle that is formed by each of two shooting directions of two cameras and the direction of the vertical line, and a resulted angle of calculation that is formed by each of the two shooting directions of the two cameras and the direction of the landmark presuming the position and direction of the robot; and

changing the presumed position and direction of the robot and identifying a presumed position and direction of the robot, where the cost becomes the lowest, to be the position and direction of the robot.

14. A robot that can be remotely controlled, comprising:

layout storage means for storing a layout plan of a building;

communication means for receiving designation of a position designated in the layout plan from a remote terminal; and

travel control means for controlling travel of a robot to the designated position, wherein;

the robot identifies its own position;

the robot receives a viewable picture from a remote terminal;

a size of the layout picture stored in the memory is converted into a size of the viewable picture received from the remote terminal, and stored;

the robot transmits the layout picture containing the identified own position to the remote terminal;

the robot receives coordinates of a destination designated from the remote terminal;

the robot calculates destination coordinates on a real scale from both a converted size and coordinates of the destination designated by the remote terminal;

the robot produces a route to the destination; and

the robot travels towards the destination.

15. A robot that can be remotely controlled, comprising:

picture taking means for taking an inside picture of a building;

communication means for transmitting the picture taken by the picture taking means when the robot regularly/irregularly patrols inside a building, to a computer with memory accessible from the outside through a network;

a step getting over unit for getting over steps in a building, based on a result of step detection by an obstacle detecting sensor;

a rotation unit for changing a shooting direction of the picture taking unit; and

an infrared ray emission/reception unit for emitting/receiving an infrared ray in parallel to the shooting direction of the picture taking unit.