Smart wearable devices and methods with attention level and workload sensing

• US 10,234,936 B2
• Filed: 08/14/2016
• Issued: 03/19/2019
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Corporation Of America (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Corporation Of America (Sony Group Corp.)
• Inventors: Tanaka, Nobuo, Elgort, Vladimir, Danielson, Jacelyn, Kalachev, Anton, Wong, John, DaCosta, Behram, Bhat, Udupi Ramanath, Copere, Ludovic, Kataoka, Masaki
• Primary Examiner(s): Davis, Tony O
• Application Number:
  US15/236,450
• Publication Number:
  US 20170010667A1
• Time in Patent Office: 947 Days
• Field of Search: 345156
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 7/00 Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 63/0869 for achieving mutual authentication cryptographic mechanisms or cryptographic arrangements for mutual authentication H04L9/3273
• CPC Class Codes: H04W 12/065 Continuous authentication
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A wearable sensor device, system and method are provided for monitoring the attention, workload or other physical status of the wearer of the wearable sensor device. At least one biometric index value is formulated for a characteristic status from the processed sensor data of the sensors of the device and the biometric index value may also incorporate prior test data and questionnaire response information in the calculated value. The system may include any number of wearable devices that have sensors, a processor and a communications link as well as a remote computer and a non-wearable device with an interface. The status of the wearer of a wearable sensor device can be monitored with the simple display of the biometric index value as well as the data from the individual sensors.

Claims

1. A wearable biometric sensor apparatus, comprising:

(a) a computer processor with memory;

(b) a plurality of sensors operably coupled to the processor, wherein said sensors are configured to collect biological and environmental data;

(c) a display;

(d) programming in a non-transitory computer readable medium and executable on the computer processor for performing steps comprising;

(i) acquiring sensor data from said plurality of sensors configured for being worn by a wearer;

(ii) processing the acquired sensor data from each sensor and evaluating levels that are above or below pre-defined levels;

(iii) formulating at least one biometric index value from a composite of the processed sensor data to indicate the present condition of the wearer as a number on a scale; and

(iv) displaying the biometric index value on the display.

2. The apparatus of claim 1, wherein said biometric index value formulation further incorporates prior biological test data of the wearer.

3. The apparatus of claim 1, wherein said biometric index value formulation further incorporates contemporaneous wearer submitted status information.

4. The apparatus of claim 1, wherein said plurality of sensors comprise:

at least one biological sensor configured to measure an internal physical condition of a wearer; and

at least one non-biological sensor configured to measure an external condition of a wearer.

5. The apparatus of claim 1, said wearable apparatus further comprising:

(a) a communications link operably coupled to the processor, the link having a transmitter and receiver; and

(b) programming in a non-transitory computer readable medium and executable on the computer processor for performing steps comprising;

(i) communicating acquired sensor data to a remote computer; and

(ii) executing program commands received from the remote computer.

6. A wearable biometric sensor apparatus and monitoring system, comprising:

(a) at least one wearable sensor device, comprising;

(i) a computer processor with memory;

(ii) a plurality of sensors operably coupled to the processor, wherein said sensors are configured to collect biological and environmental data;

(iii) a communications link; and

(iv) programming in a non-transitory computer readable medium and executable on the computer processor for performing steps comprising;

1. acquiring sensor data from said plurality of sensors configured for being worn by a wearer;

2. communicating with a monitor and display device through the communications link; and

(b) a monitor and display device, comprising;

(i) a communications link;

(ii) a computer processor with memory;

(iii) a display; and

(iv) programming in a non-transitory computer readable medium and executable on the computer processor for estimating attention level and workload level based on performing steps comprising;

1. sending and receiving communications from each wearable sensor device;

2. processing sensor data received from each wearable sensor device and evaluating sensor data levels that are above or below predefined thresholds;

3. formulating at least one biometric index value for attention level and workload level of the wearer based on processing evaluated sensor data levels that are above or below pre-defined levels to indicate the present condition of the wearer as a number on a scale; and

4. displaying the biometric index value on the display.

7. The system of claim 6, wherein said biometric index value formulation further incorporates prior biological test data of the wearer.

8. The system of claim 6, wherein said biometric index value formulation further incorporates contemporaneous wearer submitted status information.

9. The system of claim 6, said system further comprising:

a non-wearable device with a communications link, computer processor with memory, display and user interface.

10. The system of claim 9, wherein said non-wearable device of said system further comprises an alarm configured to actuate if at least one biometric index value exceeds a designated range.

11. The system of claim 9, wherein contemporaneous wearer submitted status information is entered through the user interface and communicated to the wearable device or to the display device through the communications link.

12. The system of claim 9, wherein said user interface of the non-wearable device further comprises a programming interface configured to control the sensors and computer processor of the wearable device over the communications link.

13. The wearable biometric sensor apparatus of claim 6, wherein said instructions when executed by the computer processor further perform steps comprising comparing said biometric index value with a predetermined threshold value and generating a notification and/or alarm on the display.

14. A computer implemented method for biometric sensing and monitoring a wearer of a wearable device, the method comprising:

(a) acquiring sensor data from a plurality of sensors, including biological and environmental sensing, of a wearable device worn by a user;

(b) processing the acquired sensor data from each sensor and evaluating levels that are outside of pre-defined levels; and

(c) formulating at least one biometric index value from a composite of the processed sensor data to indicate the present condition of the wearer as a number on a scale from the sensors of the wearable device;

(d) displaying the biometric index value on a display;

(e) wherein said method is performed by executing programming on at least one computer processor, said programming residing on a non-transitory medium readable by the computer processor.

15. The method of claim 14, further comprising:

downloading outside prior biological test data and prior sensor data of the user; and

using the prior data with the acquired sensor data in the formulation of said biometric index values.

16. The method of claim 14, further comprising:

providing a questionnaire of inquiries pertaining to the current status of the wearer; and

using the responses to the questionnaire inquiries with the acquired sensor data in the formulation of said biometric index values.

17. The method of claim 14, further comprising:

transferring acquired sensor data from the wearable device to a remote computer;

processing the transferred sensor data;

formulating biometric index values from the processed data; and

displaying the biometric index values for each wearable device on a display.

18. The method of claim 17, further comprising:

transmitting biometric index values from the remote computer to a smart telephone when the index values exceed a set threshold value.

19. The method of claim 14, wherein said sensors comprise:

at least one biological sensor configured to measure an internal physical condition of a wearer; and

at least one non-biological sensor configured to measure an external condition of a wearer.

20. The method of claim 14, wherein said sensors comprise an accelerometer, a heart monitor and a clock;

and wherein said formulated biometric index value comprises a level of stress formulated from processed sensor data of a subject during REM and non-REM sleep.