User-machine interface system for enhanced interaction

• US 6,347,261 B1
• Filed: 08/03/2000
• Issued: 02/12/2002
• Est. Priority Date: 08/04/1999
• Status: Expired due to Fees

• Current Assignee: Yamaha Hatsudoki Kabushiki Kaisha (Yamaha Corporation)
• Original Assignee: Yamaha Hatsudoki Kabushiki Kaisha (Yamaha Corporation)
• Inventors: Yoshida, Makoto, Kamiya, Tsuyoshi, Sakaue, Masaya, Arimune, Nobuyasu
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US09/630,577
• Time in Patent Office: 558 Days
• Field of Search: 700/245, 700/258, 700/257, 700/260, 700/264, 700/17, 700/11, 700/56, 700/83, 700/86, 345/156, 345/157, 345/163, 345/184, 345/781, 345/161, 345/168, 345/705, 345/663, 345/775, 382/118, 382/209, 701/29
• US Class Current: 700/245
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 19/00 Programme-control systems
• CPC Class Codes: G05B 2219/33051 BBC behavior based control, stand alone module, cognitive, independent agent
• CPC Class Codes: G05B 2219/33055 Holon, agent executes task and cooperates with other, distributed control
• CPC Class Codes: G05B 2219/33056 Reinforcement learning, agent acts, receives reward, emotion, action selective
• CPC Class Codes: G05B 2219/33061 Behaviour fusion, each layer can influence other by suppression or amplification
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G05B 2219/39254 Behaviour controller, robot have feelings, learns behaviour
• CPC Class Codes: G05B 2219/40413 Robot has multisensors surrounding operator, to understand intention of operator
• CPC Class Codes: G05B 2219/40414 Man robot interface, exchange of information between operator and robot
• CPC Class Codes: G05B 2219/40625 Tactile sensor
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A user information input structure that inputs user information, a memory structure that stores user'"'"'s bearing information as robot'"'"'s bearing pattern, and a regenerating structure that regenerates the pattern based on the information input to the user information input structure. A detection structure for detecting user information and environment information, a feelings generation structure for generating artificial feelings for a robot based on the detecting user and environment information, and a behavior determination structure for the behavior determination structure controls the actuator so that a resistance acted on the robot is changed.

Claims

1. A method of enhancing interaction between a user and a machine, said machine comprising (i) a sensor unit for detecting a user, (ii) a data processing unit for extracting features of the user under predetermined rules, (iii) a memory, (iv) a second sensor unit for detecting a user'"'"'s action, (v) a behavior decision unit including a behavior decision algorithm programmed to select the machine'"'"'s action when receiving designated signals, and (vi) an actuating unit for actuating the selected action, said method comprising the steps of:

(a) detecting a user by the sensor unit, extracting features of the user by the data processing unit, and recording the features of the user in the memory;

(b) interacting with the user, detecting the user'"'"'s action by the second sensor unit, and saving the user'"'"'s action in the memory in relation to the features of the user, wherein the machine behaves using the behavior decision unit and the actuating unit;

(c) determining whether any user having features identical or similar to those of the user is of record in the memory, and if any, updating the features of the user of record by statistical processing, and recording the updated features of the user in the memory in relation to the user'"'"'s action; and

(d) modifying the behavior decision algorithm in accordance with the significance of probability of identifying the user based on the statistically processed features of the user, whereby outcome of the machine actuated by the actuating unit changes based on interaction between the user and the machine.

2. A method of enhancing interaction between a user and a machine, said machine comprising (i) a sensor unit for detecting a user'"'"'s gestures associated with or without voice, (ii) a behavior library storing behavioral patterns in relation to causative signals, (iii) a memory, (iv) a behavior decision unit including a behavior decision algorithm programmed to select the machine'"'"'s action from the behavior library when receiving designated signals, and (vi) an actuating unit for actuating the selected action, said method comprising the steps of:

(a) sensing the user'"'"'s gestures associated with or without voice by the sensing unit, and recording the sensed gestures in the memory in relation to designated signals causative of the gestures;

(b) updating the behavior library if the gestures are not of record in the library, and accordingly modifying the behavior decision algorithm;

(c) when receiving signals indicative of gestures, selecting gestures by the behavior decision unit using the behavior library; and

(d) actuating the machine to perform the gestures by the actuating unit.

3. A method of enhancing interaction between a user and a machine, said machine comprising (i) a sensor unit for detecting a user'"'"'s gestures associated with or without voice, (ii) a behavior library storing behavioral patterns in relation to causative signals, (iii) a memory, (iv) a behavior decision unit including a behavior decision algorithm programmed to select the machine'"'"'s action from the behavior library when receiving designated signals, and (vi) an actuating unit for actuating the selected action, said method comprising the steps of:

(a) sensing the user'"'"'s gestures associated with or without voice by the sensing unit, and recording the sensed gestures in the memory in relation to designated signals causative of the gestures;

(b) updating the behavior library if the gestures are not of record in the library, and accordingly modifying the behavior decision algorithm;

(c) when receiving signals indicative of gestures, selecting gestures by the behavior decision unit using the behavior library; and

(d) actuating the machine to perform the gestures by the actuating unit.

4. A method of enhancing interaction between a user and a machine, said machine comprising (i) a tactile sensor unit for detecting tactile signals when a user touches the machine, (ii) a second sensor unit for sensing signals causative of the tactile signals, (iii) a memory, (iv) a pseudo-emotion generation unit including an emotion generation algorithm programmed to generate pseudo-emotions when receiving designated signals, (v) a behavior decision unit including a behavior decision algorithm programmed to select the machine'"'"'s action when receiving designated signals including outcome of the pseudo-emotion generation unit, and (vi) an actuating unit for actuating the selected action, said method comprising the steps of:

(a) sensing the user'"'"'s tactile input by the sensing unit;

(b) generating pseudo-emotions based on the tactile input and designated signals causative of the tactile input;

(c) selecting an action of the machine by the behavior decision unit based on the tactile input, the generated pseudo-emotions, and the signals causative of the tactile input; and

(d) actuating the machine to perform the selected action by the actuating unit.

5. A method of enhancing interaction between a user and a machine, said machine comprising (i) a tactile sensor unit for detecting tactile signals when a user touches the machine, (ii) a second sensor unit for sensing signals causative of the tactile signals, (iii) a memory, (iv) a pseudo-emotion generation unit including an emotion generation algorithm programmed to generate pseudo-emotions when receiving designated signals, (v) a behavior decision unit including a behavior decision algorithm programmed to select the machine'"'"'s action when receiving designated signals including outcome of the pseudo-emotion generation unit, and (vi) an actuating unit for actuating the selected action, said method comprising the steps of:

(a) sensing the user'"'"'s tactile input by the sensing unit;

(b) generating pseudo-emotions based on the tactile input and designated signals causative of the tactile input;

(c) selecting an action of the machine by the behavior decision unit based on the tactile input, the generated pseudo-emotions, and the signals causative of the tactile input; and

(d) actuating the machine to perform the selected action by the actuating unit.