Man-machine interface unit control method, robot apparatus, and its action control method

• US 6,862,497 B2
• Filed: 06/03/2002
• Issued: 03/01/2005
• Est. Priority Date: 06/01/2001
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Limited (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Limited (Sony Group Corp.)
• Inventors: Horinaka, Rika, Minamino, Katsuki, Kompe, Ralf, Kemp, Thomas, Fujita, Masahiro, Kawamoto, Kenta, Tato, Raquel
• Primary Examiner(s): Chin, Gary
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US10/343,525
• Publication Number:
  US 20040039483A1
• Time in Patent Office: 1,002 Days
• Field of Search: 700/17, 700/83, 700/86, 700245-246, 700257-259, 341 20- 22, 341/173, 341/176, 342/45, 342/473, 345157-158, 345/837, 345/957, 318/563
• US Class Current: 700/264
• CPC Class Codes: B62D 57/02 with ground-engaging propulsion means, e.g. walking members
• CPC Class Codes: B62D 57/032 with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid B62D57/024 takes precedence
• CPC Class Codes: G05B 13/0265 the criterion being a learning criterion
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence

Abstract

There is proposed a method that may be universally used for controlling a man-machine interface unit. A learning sample is used in order at least to derive and/or initialize a target action (t) to be carried out and to lead the user from an optional current status (e_c) to an optional desired target status (e_t) as the final status (e_f). This learning sample (l) is formed by a data triple made up by an initial status (e_i) before an optional action (a) carried out by the user, a final status (e_f) after the action taken place (a).

Claims

1. A method for controlling a man-machine-interface unit, wherein

with respect to at least one user an initial status (e_i) before and a final status (e_f) after an action (a) taken place are essentially detected on said man-machine-interface, and wherein

data triples essentially representing said initial status (e_i), said final status (e_f) and said action (a) taken place are collected as learning samples (1), and wherein

said learning samples (1) are used to at least derive and initialize a target action to be carried out so as to carry in each case said user from a given current status (e_c) to a given desired target status (e_t) on said man-machine-interface.

2. A method according to claim 1 wherein in each case current and initial status data (CSD, ISD) are received and generated essentially corresponding to and representing said current status (e_c) and the initial status (e_i) of said user, respectively.

3. A method according to claim 1 wherein in each case final and target status data (FSD, TSD) are received and generated essentially corresponding to and representing said final status (ef) and said target status (et) of said user, respectively.

4. A method according to claim 1 wherein action data (AD) and target action data (TAD) are received and generated essentially corresponding to and representing said action (a) taken place and said target action (t) derived.

5. A method according to claim 1 wherein said current, initial, final and target statuses (ec, ei, ef, et) and in particular the corresponding and/or representing data (CSD, ISD, FSD, TSD) thereof are in each case represented by and modeled in an embedding topological manifold (EM, DM), in particular of two dimensions.

6. A method according to claim 5 wherein separate manifolds are derived and/or used for distinct users and/or for classes of users and/or wherein common or global manifolds for all users are derived and/or used.

7. A method according to claim 1 wherein said action (a) taken place, said target action (t) derived said representing and corresponding data (AD, TAD) thereof are in each case represented by and modeled in an embedding topological manifold (AM, ADM).

8. A method according to claim 1 wherein emotional statuses are used said current, initial, final, target statuses (ec, ei, ef, et).

9. A method according to claim 1 wherein for each user based on the set (L) of learning samples (1) a transmotion mapping (T)

E⊕

A→

E

<

ei, a>

E⊕

A→

ef;

=T(ei, a)=Ta(ei) E

is derived and wherein E denotes an emotion manifold and is one of the topological manifolds (EM, EDM) for the user statuses and for the status data thereof, A denotes an action manifold and is one the topological manifolds (AM, ADM) for the action and for the action data and ei, ef and a denote respective elements of the manifolds E and A.

10. A method according to claim 1 wherein for given desired initial and target statuses (ei, et) for a given user an action (a) from a learning sample (1) with a given current status (ec), final status (ef) and an action (a) taken place already sampled is chosen as a target action (t) for which the initial and current statuses (ei, ec) and the final or the target statuses (ef, et) and the data (ISD, CSD, FSD, TSD) thereof are in a next topological neighborhood.

11. A method according to claim 10 wherein a distance measure, a metric, continuity, direction, vector properties of said statuses (e_i, e_e, e_T, e_t) and actions (a, t) are involved to derive and describe said neighborhood.

12. A method according to claim 1 wherein a sequence of actions (a) to be carried out is derived and initialized to be executed as said target action (t) so as to lead a given user to a target status (et) to fit best to a given desired final status (ef) for said user.

13. A method according to claim 1 wherein a modification is included into and performed on a derived target action (t) and into the data (TAD) thereof.

14. A method according to claim 13 wherein said modification is introduced randomly, by means of a probabilistic measure, on the basis of topological, distance, metric, continuity, direction, vector properties.

15. A method according to claim 1 wherein obtained learning samples (1)—

in particular to derive said topological manifolds (EM, EDM, AM, ADM)—

are generalized.

16. A method according to claim 15 wherein the step of generalizing said learning samples (1) at least comprises a step of averaging and weighting learning samples (1) from different users, in particular to derive common and/or global topological manifolds, wherein in particular said different users are emotionally clustered depending on their typical status/and or behavior.

17. A method according to claim 15 wherein the step of generalizing said learning samples (1) at least comprises at least a step of reducing the dimension of the action manifold (AN, ADM), in particular by means of a projection.

18. A method according to claim 5 wherein the step of generalizing said learning samples (1) comprises at least a step of enlarging and reducing the extension of neighborhood of a given learning sample (1) within which all triples of initial statuses (ei), final statuses (ef) and target actions (t) are classified as being equivalent to said given learning sample (1), in particular until the point where

a final status (ef) and a target action (t) are valid for all initial statuses (ei) and until the point where the final status (ef) is irrelevant and only a direction of a change of the statuses is considered.

19. A system for controlling a man-machine-interface unit which is capable of performing and realizing the method for controlling a man-machine-interface unit according to claim 1 and the steps thereof.

20. A computer program product comprising computer program means adapted to perform and realize the method for controlling a man-machine-interface unit according to claim 1 and the steps thereof, when it is executed on a computer, a digital signal processing means.

21. A robot apparatus acting responsive to inner and outer statuses comprising:

status detection means for detecting the status of a user of the robot apparatus;

target status setting means for setting a desired target status of the user;

action determining means for determining an action to be carried out to lead the user'"'"'s status detected by said status detection means to said target status; and

controlling means for carrying out the determined action.

22. The robot apparatus according lo claim 21 further comprising:

learning means;

wherein

the action to be carried out to lead the user'"'"'s status as detected by said status detection means to said target status is updated by said learning means.

23. The robot apparatus according to claim 22 wherein said learning means collects a data triple representing the initial status before and a final status after an action taken place, and the action taken place, as learning sample, in connection with said user;

and wherein

said action determining means decides on said action to be carried out based on said learning sample.

24. The robot apparatus according to claim 21 wherein said user'"'"'s status and the target status are the status of the user'"'"'s feeling.

25. The robot apparatus according to claim 24 wherein said target status setting means uses the feeling status of said robot apparatus as said target status.

26. The robot apparatus according to claim 24 further comprising:

speech recognition means and picture recognition means;

said status detection means detecting the feeling status as said user'"'"'s status from the speech and a face image of said user.

27. The robot apparatus according to claim 24 wherein

said feeling status may be represented as a coordinate in topological space of a multiple dimensions;

and wherein

the user'"'"'s status and the target status are given as coordinates in said topological space.

28. The robot apparatus according to claim 24 wherein

said feeling status may be represented as a coordinate in a multi-dimensional topological space;

and wherein

the user'"'"'s status and the target status are given as coordinates obtained by projecting the coordinate in said topological space on a topological plane comprising two of said multiple dimensions.

29. A method for controlling the behavior of a robot apparatus acting autonmously responsive to inner and outer statuses, comprising:

a status detection step of detecting the status of a user of the robot apparatus;

a target status setting step of setting a desired target status of the user;

an action determining step of determining an action to be carried out to lead the user'"'"'s status as detected by said status detection step to said target status; and

a controlling step of carrying out the action determined.

30. The method according to claim 29 further comprising:

a learning step;

wherein

the action to be carried out to lead the user'"'"'s status as detected by said status detection step to said target status is updated by said learning step.

31. The method according to claim 30 wherein

in said learning step, a data triple representing the initial status before and a final status after an action taken place, and the action taken place, as learning sample, in connection with said user, is collected;

and wherein

said action determining step decides on said action to be carried out based on said learning sample.

32. The method according to claim 29 wherein

the user'"'"'s status and the target status are the user'"'"'s feeling status.

33. The method according to claim 32 wherein, in said target status setting step, the feeling status of the robot apparatus is used as said target status.

34. The method according to claim 32 further comprising:

a speech recognition step and an image recognition step;

said status detection step detects the feeling status, as said user'"'"'s status, from the speech and the face image of the user.

35. The method according to claim 32 wherein said feeling status may be represented as a coordinate in topological space of a multiple dimensions;

and wherein

the user'"'"'s status and the target status are given as coordinates in said topological space.

36. The method according to claim 32 wherein

said feeling status may be represented as a coordinate in a multi-dimensional topological space;

and wherein

the user'"'"'s status and the target status are given as coordinates obtained by projecting the coordinate in said topological space on a topological plane comprising two of said multiple dimensions.