Article handling system and method and article management system and method

• US 7,191,035 B2
• Filed: 01/11/2006
• Issued: 03/13/2007
• Est. Priority Date: 06/02/2003
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Matsukawa, Yoshihiko, Sato, Satoshi, Yamada, Osamu, Okamoto, Shusaku, Naruoka, Tomonobu
• Primary Examiner(s): Black; Thomas
• Assistant Examiner(s): Marc; McDieunel
• Application Number:
  US11/330,373
• Publication Number:
  US 20060184279A1
• Time in Patent Office: 426 Days
• Field of Search: 700/245
• US Class Current: 700/245
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 2203/041 Camera
• CPC Class Codes: G05B 2219/39387 Reflex control, follow movement, track face, work, hand, visual servoing
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

A camera or other sensing unit senses the conditions of articles and mobile entities, including humans within a living space. An article management/operation server manages, within an article database, attribute information about the articles, including operators, according to the information from the sensing unit. The server receives a user'"'"'s instruction, input through a console unit, and refers to the article database to convert this instruction into a control command, which is then transmitted to a life-support robot.

Claims

1. An article handling system, comprising:

a robot for handling an article which exists in a predetermined life space;

a plurality of image capturers installed in the life space for capturing images of the life space;

a sensor for detecting a state of an article existing in the life space, the state of the article including at least a current location of the article;

a console unit which includes a display section for displaying a console window presented to a user and an input section which serves as an interface for the user; and

a display controller for generating the console window,

wherein the console window includes

a virtual space rendered in the form of an image seen from a virtual view point by transforming and synthesizing data of the images captured by the image capturers,

an article icon corresponding to an article existing in the life space, and

window mask data having coordinate values corresponding to the virtual space for specifying a position of the article icon in the virtual space, and

the window mask data is generated based on a current location of the article seen from the virtual view point which is determined by transformation from a current location of the article detected by the sensor,

the user uses the input section to switch the virtual view point and designate a task which is assigned to the robot in the virtual space, and

the robot handles an article in the life space based on the designated task.

2. The article handling system of claim 1, wherein the task carried out by the robot is a transfer of an article in the life space.

3. The article handling system of claim 2, wherein:

the console window includes an article icon placed in the virtual space such that the article icon corresponds to an article existing in the life space; and

the transfer of the article is designated using the article icon.

4. The article handling system of claim 3, wherein the transfer of the article is designated by drag-and-dropping the article icon onto a desired transfer destination in the virtual space.

5. The article handling system of claim 1, wherein:

the console window includes an article icon placed in the virtual space such that the article icon corresponds to an article existing in the life space and a pointer controlled by the user through the input section; and

when the pointer points at an article icon corresponding to an article handleable by the robot on the console window, the article icon is highlighted.

6. The article handling system of claim 1, wherein:

the console window includes an article icon placed in the virtual space such that the article icon corresponds to an article existing in the life space and a pointer controlled by the user through the input section; and

when the pointer points at an article icon corresponding to an article handleable by the robot on the console window, information about the article is displayed.

7. The article handling system of claim 1, wherein:

the console window includes an article icon placed in the virtual space such that the article icon corresponds to an article existing in the life space; and

when an article is selected as a task subject of the robot by designating a corresponding article icon on the console window, the article icon is highlighted.

8. A robot controller unit for designating a task which is assigned to a robot for handling an article existing in a predetermined life space, comprising:

a receiver installed in the life space for receiving information from a plurality of image capturers which capture images of the life space and a sensor which detects a state of an article existing in the life space, the state of the article including at least a current location of the article;

a display section for displaying a console window presented to a user;

an input section which serves as an interface for the user; and

a display controller for generating the console window,

wherein the console window includes

a virtual space rendered in the form of an image seen from a virtual view point by transforming and synthesizing data of the images captured by the image capturers,

an article icon corresponding to an article existing in the life space, and

window mask data having coordinate values corresponding to the virtual space for specifying a position of the article icon in the virtual space, and

the window mask data is generated based on a current location of the article seen from the virtual view point which is determined by transformation from a current location of the article detected by the sensor, and

the user uses the input section to switch the virtual view point and designate a task which is assigned to the robot in the virtual space.