Information processing apparatus, system, and method thereof

• US 8,797,331 B2
• Filed: 08/04/2008
• Issued: 08/05/2014
• Est. Priority Date: 08/06/2007
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sano, Akane, Asukai, Masamichi, Ito, Taiji, Sako, Yoichiro
• Primary Examiner(s): Welch, David T
• Application Number:
  US12/221,526
• Publication Number:
  US 20090040231A1
• Time in Patent Office: 2,192 Days
• Field of Search: 345/474, 345/475, 345/950, 463/31, 463/32, 463/36
• US Class Current: 345/474
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/0006 ECG or EEG signals
• CPC Class Codes: A61B 5/0008 Temperature signals
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/026 Measuring blood flow A61B3/1233, A61B3/1241 take precedence
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/389 Electromyography [EMG]
• CPC Class Codes: A61B 5/4266 sweat secretion
• CPC Class Codes: A61B 5/742 : using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/744 : Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: G06F 3/015 : Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06T 13/40 : of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders
• CPC Class Codes: Y10S 345/95 : Sprite processing

Abstract

An information processing apparatus includes a bio-information obtaining unit configured to obtain bio-information of a subject; a kinetic-information obtaining unit configured to obtain kinetic information of the subject; and a control unit configured to determine an expression or movement of an avatar on the basis of the bio-information obtained by the bio-information obtaining unit and the kinetic information obtained by the kinetic-information obtaining unit and to perform a control operation so that the avatar with the determined expression or movement is displayed.

Claims

1. An information processing apparatus comprising:

a bio-information obtaining unit configured to obtain bio-information of a subject;

a kinetic-information obtaining unit configured to obtain kinetic information of the subject;

a storage unit storing a plurality of avatars reflecting a plurality of emotions; and

a control unit configured to determine an emotion of the subject on the basis of the bio-information obtained by the bio-information obtaining unit and the kinetic information obtained by the kinetic-information obtaining unit and to perform a control operation to select from the storage unit an avatar having an emotion matching the determined emotion of the subject and to display the avatar on a display together with an avatar representing a person other than the subject,

wherein the bio-information includes information representing at least one of an electromyogram, breathing, perspiration, galvanic skin response, blood pressure, oxygen saturation, skin surface temperature, brain wave, blood flow change, or temperature, and

wherein the bio-information obtaining unit, kinetic-information obtaining unit, display and control unit in combination form at least part of a mobile device;

the information processing apparatus further comprising a position-information obtaining unit configured to obtain position information; and

a map-image obtaining unit configured to obtain map image data,

wherein the control unit is configured to control, on the basis of the position information obtained by the position-information obtaining unit, the map-image obtaining unit to obtain map image data to display a map image, and performs the control operation so that the avatar having the emotion matching the determined emotion of the subject is displayed at a position in accordance with the position information on the map image.

2. The information processing apparatus according to claim 1, wherein the bio-information obtaining unit is a bio-information detecting unit configured to detect bio-information, and

wherein the kinetic-information obtaining unit is a kinetic-information detecting unit configured to detect kinetic information.

3. The information processing apparatus according to claim 1, wherein the bio-information obtaining unit and the kinetic-information obtaining unit are communication units configured to receive bio-information and kinetic information, respectively, from an external apparatus.

4. The information processing apparatus according to claim 1, further comprising a saving unit configured to save information,

wherein, when the bio-information obtaining unit and the kinetic-information obtaining unit obtain bio-information and kinetic information, respectively, the control unit causes the saving unit to save either the obtained bio-information and kinetic information or an avatar-related information, the avatar-related information indicating the determined emotion of the subject.

5. The information processing apparatus according to claim 4, wherein the control unit is configured to read either the bio-information and the kinetic information or the avatar-related information from the saving unit under a predetermined condition and to perform a display control operation so that the avatar having the emotion matching the determined emotion of the subject is displayed using either the read bio-information and kinetic information or the read avatar-related information.

6. The information processing apparatus according to claim 1, wherein the kinetic information is information indicating at least one of a still state, a walking state, a running state, an exercising state, or movement of a body element.

7. The information processing apparatus according to claim 1, wherein the control unit is further configured to determine a movement of the avatar of the subject on the basis of the bio-information obtained by the bio-information obtaining unit and the kinetic information obtained by the kinetic-information obtaining unit, and to perform the control operation so that the avatar of the subject is further displayed with the determined movement.

8. An information processing system comprising:

a transmitting apparatus; and

an information processing apparatus,

wherein the transmitting apparatus includes

a bio-information detecting unit configured to detect bio-information of a subject,

a kinetic-information detecting unit configured to detect kinetic information of the subject, and

a transmitting unit configured to transmit the bio-information detected by the bio-information detecting unit and the kinetic information detected by the kinetic-information detecting unit, and

wherein the information processing apparatus includes

a communication unit configured to communicate information, and

a control unit configured to cause the communication unit to receive the bio-information and the kinetic information, determine an emotion of the subject on the basis of the bio-information and kinetic information received by the communication unit, and perform a control operation to select, from a storage unit storing a plurality of avatars reflecting a plurality of emotions, an avatar having an emotion matching the determined emotion of the subject and to display the avatar on a display together with an avatar representing a person other than the subject,

wherein the bio-information includes information representing at least one of an electromyogram, breathing, perspiration, galvanic skin response, blood pressure, oxygen saturation, skin surface temperature, brain wave, blood flow change, or temperature, and

wherein the transmitting apparatus, display, and the processing apparatus in combination form a mobile device; and

wherein the control unit is further configured to cause display, on a map, of the avatar having the emotion matching the determined emotion of the subject and the avatar representing a person other than the subject.

9. The information processing system according to claim 8, further comprising a server apparatus including the storage unit configured to store information,

wherein the transmitting unit of the transmitting apparatus is configured to transmit the bio-information and the kinetic information to the server apparatus,

wherein the server apparatus is configured to store, in the storage unit, the bio-information transmitted by the transmitting unit and the kinetic information transmitted by the transmitting unit, and

wherein the communication unit of the information processing apparatus is configured to receive the bio-information stored in the storage unit of the server apparatus and the kinetic information stored in the storage unit of the server apparatus.

10. The information processing system according to claim 8, further comprising a server apparatus including the storage unit configured to store bio-information and kinetic information,

wherein the transmitting unit of the transmitting apparatus is configured to transmit the bio-information and the kinetic information to the server apparatus,

wherein the server apparatus is configured to store the transmitted bio-information and kinetic information in the storage unit, and

wherein the communication unit of the information processing apparatus is configured to receive avatar-related information indicating the emotion of the subject.

11. The information processing system according to claim 8, wherein the control unit is further configured to determine a movement of the subject on the basis of the bio-information and kinetic information received by the communication unit, and to perform the control operation so that the avatar of the subject is further displayed with the determined movement.

12. An information processing method comprising:

obtaining bio-information of a subject using a bio-information obtaining unit of a mobile device;

obtaining kinetic information of the subject using a kinetic information obtaining unit of the mobile device;

determining an emotion of the subject on the basis of the obtained bio-information and kinetic information; and

performing a control operation using a control unit of the mobile device to select, from a storage unit storing a plurality of avatars reflecting a plurality of emotions, an avatar having an emotion matching the determined emotion of the subject and to display the avatar on a display of the mobile device together with an avatar representing a person other than the subject,

wherein the bio-information includes information representing at least one of an electromyogram, breathing, perspiration, galvanic skin response, blood pressure, oxygen saturation, skin surface temperature, brain wave, blood flow change, or temperature,

wherein performing the control operation further comprises causing display, on a map, of the avatar having the emotion matching the determined emotion of the subject and the avatar representing a person other than the subject.

13. The information processing method according to claim 12, further comprising determining a movement of the subject on the basis of the obtained bio-information and kinetic information;

and

performing the control operation so that the avatar of the subject is further displayed with the determined movement.

14. An information processing apparatus comprising:

a mobile device controller, of a subject'"'"'s mobile device, configured to determine an emotion of the subject on the basis of bio-information and kinetic information of the subject and to perform a control operation to select, from a storage unit storing a plurality of avatars reflecting a plurality of emotions, an avatar having an emotion matching the determined emotion and to display the avatar on a display of the mobile device together with an avatar representing a person other than the subject,

wherein the bio-information includes information representing at least one of an electromyogram, breathing, perspiration, galvanic skin response, blood pressure, oxygen saturation, skin surface temperature, brain wave, blood flow change, or temperature, and

wherein the mobile device controller is further configured to cause display, on a map, of the avatar having the emotion matching the determined emotion and the avatar representing a person other than the subject.

15. An information processing method comprising:

determining, with at least one processor of a subject'"'"'s mobile device, an emotion of the subject on the basis of bio-information and kinetic information of the subject; and

performing a control operation using the at least one processor of the mobile device to select, from a storage unit storing a plurality of avatars reflecting a plurality of emotions, an avatar having an emotion matching the emotion of the subject and to display the avatar on a display of the mobile device together with an avatar representing a person other than the subject,

wherein the bio-information includes information representing at least one of an electromyogram, breathing, perspiration, galvanic skin response, blood pressure, oxygen saturation, skin surface temperature, brain wave, blood flow change, or temperature, and

wherein performing the control operation further comprises causing display, on a map, of the avatar having the emotion matching the emotion of the subject and the avatar representing a person other than the subject.