Biometric sensing and processing apparatus for mobile gaming, education, and wellness applications

• US 9,311,825 B2
• Filed: 12/21/2012
• Issued: 04/12/2016
• Est. Priority Date: 12/22/2011
• Status: Active Grant

• Current Assignee: Senstream, Inc.
• Original Assignee: Senstream, Inc.
• Inventors: Lusted, Hugh S., Knapp, Ben, Roy, Jashojit
• Primary Examiner(s): Voorhees, Catherine
• Application Number:
  US13/725,777
• Publication Number:
  US 20130183646A1
• Time in Patent Office: 1,208 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/164 Lie detection
• CPC Class Codes: A61B 5/6801 specially adapted to be attached to or worn on the body surface
• CPC Class Codes: A61B 5/6826 Finger
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

An apparatus for interoperably utilizing multiple biosensor data obtained from a finger of the user. A biometric sensor board is retained in a housing adapted for retaining a finger of the user and maintaining multiple sensors in contact with the skin thereof. Implementations are described for fingertip and ring mounted sensor boards. In one implementation, these sensors can be sensors electrodermal response (EDR), or photoplethysmograph (PPG) signals, or temperature, or acceleration in three axes, and combinations thereof. The biometric sensor board registers and processes the signals from the sensors and communicates them to a mobile device which interoperably utilizes multiple sensor information to determine aspects of user emotional state within an application to generate results which are displayed on the mobile device.

Claims

1. An apparatus for estimating emotional state of a user for display on a mobile device, said apparatus comprising:

(a) a housing configured for retention of a finger of a user;

(b) a sensor circuit retained in said housing;

(c) said sensor circuit comprising a plurality of sensors configured for skin contact with a portion of the finger of the user;

(d) said plurality of sensors comprising electrodermal response (EDR), photoplethysmograph (PPG), temperature, and acceleration sensors;

(e) said sensor circuit comprising a communications interface configured for communicating data from said sensors to a mobile device;

(f) said sensor circuit comprising a processor and programming executable on said processor for receiving input data from said sensors, processing said input data, and outputting processed data to the mobile device through said communications interface;

(g) wherein the mobile device is configured for hosting an application program for communicating with said processor, post-processing said processed data, and displaying said post-processed data as biofeedback data; and

(h) wherein said application program performs motion artifact removal on EDR and PPG signals in response to acceleration sensed by said acceleration sensor.

2. The apparatus recited in claim 1, wherein said acceleration sensor comprises a three axis acceleration sensor configured for sensing acceleration in the X, Y and Z directions.

3. The apparatus recited in claim 1, wherein said EDR sensor generates a tonic signal and a phasic signal as an indication of arousal and mood.

4. The apparatus recited in claim 1, wherein said PPG sensor measures user cardiac pulse from which heart rate (HR) and heart rate variability (HRV) can be determined.

5. The apparatus recited in claim 1, wherein said communications interface comprises an audio frequency electronic signal interface configured for connection to a microphone input of the mobile device.

6. The apparatus recited in claim 5, wherein said audio frequency electronic signal interface utilizes amplitude modulation and frequency division multiplexing of multiple signals from said sensors.

7. The apparatus recited in claim 1, wherein said communications interface comprises a wireless signal interface.

8. The apparatus recited in claim 7, wherein said wireless signal interface comprises a Bluetooth interface.

9. The apparatus recited in claim 1;

wherein said housing is configured for retention on a fingertip of a user with said EDR, PPG and temperature sensors held against a pad of the fingertip;

or

wherein said housing is configured as a ring for retention between knuckles of a finger of a user with said EDR, PPG and temperature sensors held against the pad of the finger between the knuckles.

10. The apparatus as recited in claim 1, wherein said application program performs motion artifact removal on EDR and PPG signals in response to acceleration sensed by said acceleration sensor, by rejecting EDR and PPG data in response to sensing a sufficient acceleration from said acceleration sensor.

22. The system recited in claim 10, wherein said application program is configured to detect user emotional state as a basis for results and decisions in game scenarios.

11. A system for estimating emotional state of a user for display on a mobile device, said apparatus comprising:

(a) a housing configured for retention of a finger of a user;

(b) a sensor circuit retained in said housing;

(c) said sensor circuit comprising a plurality of sensors configured for skin contact with a portion of the finger of the user;

(d) said plurality of sensors comprising electrodermal response (EDR), photoplethysmograph (PPG), temperature, and acceleration sensors;

(e) said sensor circuit comprising a communications interface configured for communicating data from said sensors to a mobile device;

(f) said sensor circuit comprising a processor and programming executable on the processor for receiving input data from said sensors, processing said input data, and outputting processed data to the mobile device through said communications interface; and

(g) an application program configured for execution on a processor of a mobile device having a display and user inputs and which is configured to receive said processed data;

(h) said application program configured for communicating with said sensor circuit processor, post-processing said processed data, and displaying said post-processed data on said display as biofeedback data; and

(i) wherein said application program further performs motion artifact removal on EDR and PPG signals in response to acceleration sensed as triaxial motion by said acceleration sensors in said sensor circuit configured for retention on the finger of the user for mediating motion effects which interfere with physiological signal measurements.

12. The system recited in claim 11, wherein said application program performs said motion artifact removal on EDR and PPG signals in response to acceleration sensed by said acceleration sensor, while also being configured for enabling a pointing function with appropriate finger motion, or gesture, as a user input.

13. The system recited in claim 11, wherein said sensor array further comprises a temperature sensor.

14. The system recited in claim 11, wherein said application program is further configured for performing a weighted average of said EDR and PPG sensors to create an overall emotional state estimation.

15. The system recited in claim 11, wherein said EDR sensor generates a tonic signal and a phasic signal as an indication of arousal and mood.

16. The system recited in claim 11, wherein said PPG sensor measures user cardiac pulse from which heart rate (HR) and heart rate variability (HRV) can be determined.

17. The system recited in claim 11, wherein said acceleration sensor comprises a three axis acceleration sensor configured for sensing acceleration in the X, Y and Z directions.

18. The system recited in claim 11, wherein said communications interface comprises an audio frequency electronic signal interface from said apparatus configured for connection to a microphone input of a mobile device.

19. The system recited in claim 11, wherein said communications interface comprises a wireless signal interface.

20. The system recited in claim 11, wherein said application program is configured to detect good stress versus bad stress for the user.

21. The system recited in claim 11, wherein said application program is configured to detect user short term and long term stress.

23. The system recited in claim 11, wherein said application program is configured to detect user emotional state in response to specific stimuli.

24. The system as recited in claim 11, wherein said application program performs motion artifact removal on EDR and PPG signals in response to acceleration sensed by said acceleration sensor, by rejecting EDR and PPG data in response to sensing a sufficient acceleration from said acceleration sensor.

25. A system for estimating emotional state of a user for display on a mobile device, said apparatus comprising:

(a) a mobile device having a processor, a display, and a user interface;

(b) a sensor array configured for communication with said mobile device;

(c) a housing of said sensor array configured for retention of a finger of a user;

(d) a sensor circuit retained in said housing;

(e) said sensor circuit comprising a plurality of sensors configured for skin contact with a portion of the finger of the user;

(f) said plurality of sensors comprising electrodermal response (EDR), photoplethysmograph (PPG), temperature, and acceleration sensors, and wherein motion effects which interfere with physiological signal measurements of EDR and PPG captured at this finger sensing location of a user are mediated in response to utilizing acceleration data to compensate for any movement which arises at the finger location;

(g) said sensor circuit comprising a communications interface configured for communicating data from said sensors to the mobile device;

(h) said sensor circuit comprising a processor and programming executable on said processor for receiving input data from said sensors, processing said input data, and outputting processed data to said mobile device through said communications interface;

(i) said mobile device having a communications interface configured for communication of data with said sensor circuit; and

(j) an application program configured for execution on the processor of said mobile device;

(k) said application program configured for communicating with said sensor circuit processor, post-processing said processed data, and displaying said post-processed data on said display as biofeedback data.

26. A system for displaying good stress versus bad stress on a mobile device in response to biofeedback signals received from an sensor array, said system comprising:

(a) a sensor array;

(b) a housing for said sensor array which is configured for insertion and retention of a finger of a user;

(c) a sensor circuit retained in said housing;

(d) said sensor circuit comprising a plurality of sensors configured for skin contact with a portion of the finger of the user;

(e) said plurality of sensors comprising electrodermal response (EDR), photoplethysmograph (PPG), and acceleration sensors;

(f) said sensor circuit comprising a communications interface configured for communicating said data from said sensors to a mobile device;

(g) said sensor circuit comprising a processor and programming executable on the processor for receiving input data from said sensors, processing said input data, and outputting processed data on to the mobile device through said communications interface; and

(h) an application program configured for execution on a processor of a mobile device having a display and user inputs configured to receive said processed data;

(i) said application program configured for communicating with said sensor circuit processor, post-processing said processed data, performing motion artifact removal on EDR and PPG signals in response to accelerations sensed by said acceleration sensor, and combining EDR and PPG signals using a weighted average to create an overall emotional state estimation, and determining and displaying good stress versus bad stress for the user; and

(j) wherein said application program is configured for processing said EDR and PPG signals in response to acceleration sensed as triaxial motion by said acceleration sensors in said sensor circuit configured for retention on the finger of the user for mediating motion effects which interfere with physiological signal measurements.