Biometric interface for a handheld device

• US 9,557,814 B2
• Filed: 12/08/2010
• Issued: 01/31/2017
• Est. Priority Date: 04/22/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Weising, George
• Primary Examiner(s): SHERWIN, RYAN W
• Application Number:
  US12/963,594
• Publication Number:
  US 20110260830A1
• Time in Patent Office: 2,246 Days
• Field of Search: 713/186, 463 1- 47, 380/251, 379/93.13, 340/5.52, 340 582- 584, 340/4.3, 340/12.1, 340/13.1, 340/13.2, 340/13.37
• US Class Current: 1/1
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/60 Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G07F 17/3204 Player-machine interfaces
• CPC Class Codes: G07F 17/3206 Player sensing means, e.g. presence detection, biometrics

Abstract

Methods and systems for applying biometric data to an interactive program executed by a portable device are provided. According to embodiments of the invention, raw bio-signal data is captured and filtered so as to determine the bio-signal of the user of the interactive program. The bio-signal is analyzed so as to determine biometrics of the user, which are applied as input to the interactive program. A setting or state of the interactive program is modified based on the biometrics. An updated state of the interactive program is rendered to the user, reflecting the modification of the setting or state of the interactive program.

Claims

1. A method for processing biometric data to affect interactivity during execution of an interactive program, comprising,

(a) displaying an interactive program on a handheld device, the handheld device including one or more selectable inputs, each of the selectable inputs being for a specific action to be executed by the interactive program;

(b) detecting the presence of human skin of a user by at least a first sensor and a second sensor of the handheld device;

(c) reading biometric data obtained from the human skin of the user using the first and second sensors;

(d) processing the biometric data to determine a bio-input characteristic; and

(e) analyzing the bio-input characteristic during execution of the interactive program, wherein analyzing the bio-input characteristic includes using a current state of interactivity to predict a future value of the bio-input characteristic; and

(f) modifying a state of the interactive program using the bio-input characteristic, wherein modifying the state of the interactive program includes adjusting one or more settings including a setting defining an accuracy of an action for at least one of the selectable inputs of the handheld device, the adjusting of the setting defining the accuracy of the action using the predicted future value of the bio-input characteristic;

wherein biometric data is obtained from bio-signal data that is selected from a group consisting of galvanic skin response signal data, electromuscular signal data, or electrocardio signal data.

2. The method of claim 1, wherein the reading of the biometric data includes,

receiving raw signal data from the first and second sensors;

filtering the raw signal data to determine bio-signal data; and

analyzing the bio-signal data to determine the biometric data.

3. The method of claim 1, wherein method operations (c), (d) and (e) are continuously repeated;

and

wherein the bio-input characteristic changes continuously over time, during execution of the interactive program.

4. The method of claim 1, wherein the one or more settings further include one or more of the following:

audio attributes, visual attributes, an attribute of a character in the interactive program, an attribute of an action performed by a character in the interactive program, and a level of difficulty;

wherein the audio attributes include volume;

wherein the visual attributes include brightness, zoom, steadiness, and color;

wherein the attribute of the character includes appearance, color, countenance, and respiration intensity and rate; and

wherein the attribute of the action performed by the character includes force of an attack action, accuracy of an attack action, and speed of movement.

5. The method of claim 1, wherein the selectable inputs define mechanisms for receiving intentional input from the user, the selectable inputs being selected from the group consisting of button input, touchscreen input, gesture input, and motion input.

6. The method of claim 1, wherein the modifying the state of the interactive program further includes,

determining a current stage of the interactive program;

and

associating a modification parameter with the current stage, the modification parameter determining the applicability of the bio-input characteristic to a current state of the interactive program.

7. The method of claim 6, wherein the modification parameter determines when the bio-input characteristic is applied to modify the state of the interactive program.

8. The method of claim 1, further comprising:

storing the biometric data and analyzing the stored biometric data to generate a model for predicting future values of the bio-input characteristic, the current state of interactivity being applied to the model to predict the future value of the bio-input characteristic.

9. The method of claim 1, wherein modifying the state of the interactive program includes adjusting a functionality setting for the at least one of the selectable inputs of the handheld device.

10. A method for affecting interactivity between a user and an interactive application, comprising,

initiating interactivity between a user and an interactive application displayed on a handheld device;

monitoring a physiological attribute of the user during the interactivity, the monitoring includes detecting data which describes the physiological attribute from one or more sensors coupled to the handheld device, the sensors configured to obtain bio-signal data that is selected from a group consisting of galvanic skin response signal data, electromuscular signal data, or electrocardio signal data;

adjusting a state of the interactive application based on the monitored physiological attribute of the user, wherein adjusting the state of the interactive application includes,

predicting a future value of the physiological attribute, and

adjusting the state of the interactive application using the predicted future value of the physiological attribute; and

wherein adjusting the state of the interactive application includes adjusting a respiration intensity or a respiration rate graphically depicted by a character defined by the interactive application.

11. The method of claim 10, wherein monitoring the physiological attribute includes,

determining a value of the physiological attribute.

12. The method of claim 11, wherein monitoring the physiological attribute further includes,

recording the value of the physiological attribute;

and

tracking changes in the value of the physiological attribute.

13. The method of claim 10, wherein adjusting a state of the interactive application includes,

processing the physiological attribute and the state of the interactive application to determine a relationship between the value of the physiological attribute and the state of the interactive application;

and

adjusting the state of the interactive application according to the value of the physiological attribute based on the relationship between the value of the physiological attribute and the state of the interactive application.

14. The method of claim 13, wherein adjusting the state of the interactive application further includes,

determining a stage of the interactivity between the user and the interactive application;

and

adjusting the state of the interactive application based on the stage of the interactivity between the user and the interactive application.

15. The method of claim 10, wherein adjusting the state of the interactive application provokes a predetermined change in the physiological attribute of the user.

16. The method of claim 10, wherein adjusting the respiration rate graphically depicted by the character includes synchronizing respiration graphically depicted by the character with a respiration of the user.

17. A handheld device for applying biometric input to an interactive application module, comprising,

a processor;

at least one input mechanism for receiving intentional input from a user;

at least one biometric sensor for detecting biometric data of the user;

a biometric processing module executed by the processor, the biometric processing module receiving the biometric data from the at least one biometric sensor and processing the biometric data to determine a bio-input characteristic, wherein the bio-input characteristic defines a predicted future value of a physiological attribute of the user, the at least one biometric sensor configured to obtain bio-signal data that is selected from a group consisting of galvanic skin response signal data, electromuscular signal data, or electrocardio signal data;

an interactive application module executed by the processor, the interactive application module receiving the bio-input characteristic and adjusting a state of the interactive application module based on the bio-input characteristic that defines the predicted future value of the physiological attribute, wherein adjusting the state of the interactive application module includes adjusting a color and a correlated ability of a character defined by the interactive application module, wherein adjusting the ability of the character defines a changed functionality of the at least one input mechanism, wherein the interactive application module processes the bio-input characteristic and the state of the interactive application module to determine a relationship between the bio-input characteristic and the state of the interactive application module, and adjusts the state of the interactive application module according to the relationship between the bio-input characteristic and the state of the interactive application module; and

a display for displaying the interactive application module to a user.

18. The handheld device of claim 17, wherein the at least one input mechanism includes one or more of a button, joystick, directional pad, touch-sensitive device, trackball, and a motion-sensing device.

19. A system for applying biometric input to an interactive application module, comprising,

a computer, the computer configured to execute an interactive application module and provide output to a display;

a controller, the controller configured to communicate with the computer, the controller including at least one input mechanism for receiving intentional input from a user,

and at least one biometric sensor for detecting biometric data of the user;

the computer further including a biometric processing module, the biometric processing module receiving the biometric data from the at least one biometric sensor and processing the biometric data to determine a bio-input characteristic when the biometric data exceeds a predefined threshold, the at least one biometric sensor configured to obtain bio-signal data that is selected from a group consisting of galvanic skin response signal data, electromuscular signal data, or electrocardio signal data; and

the interactive application module receiving the bio-input characteristic and adjusting a state of the interactive application module based on the bio-input characteristic, wherein the interactive application module is configured to use a current state of interactivity to predict a future value of the bio-input characteristic, the adjusting of the state of the interactive application module using the predicted future value of the bio-input characteristic.

20. The system of claim 19, wherein the interactive application module processes the bio-input characteristic and the state of the interactive application module to determine a relationship between the bio-input characteristic and the state of the interactive application module, and adjusts the state of the interactive application module according to the relationship between the bio-input characteristic and the state of the interactive application module.

21. The system of claim 19, wherein the at least one input mechanism includes one or more of a button, joystick, directional pad, touch-sensitive device, trackball, and a motion-sensing device.

22. The method of claim 19,

wherein biometric processing module is configured to store the biometric data and analyze the stored biometric data to generate a model for predicting future values of the bio-input characteristic, the current state of interactivity being applied to the model to predict the future value of the bio-input characteristic.