Head-mounted display device with a camera imaging eye microsaccades

• US 20170146801A1
• Filed: 02/02/2017
• Published: 05/25/2017
• Est. Priority Date: 07/15/2013
• Status: Abandoned Application

• Current Assignee: Advanced Insurance Products & Services Incorporated
• Original Assignee: Advanced Insurance Products & Services Incorporated
• Inventors: Stempora, Jeffrey
• Application Number:
  US15/422,826
• Publication Number:
  US 20170146801A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification

Abstract

In one embodiment, a head-mounted display device comprises an augmented reality display, a camera positioned to capture images of at least one eye of the user at different times, an ambient light sensor, a speaker, and one or more processors that process the images to determine a change in size of at least one pupil of the user independent of a change in output from the ambient light sensor, determine at least one of an eye microsaccade amplitude, an eye microsaccade frequency, an eye gaze direction, and vergence, and provide a form of audio alert signal or a form of display alert signal when the change in size of at least one pupil indicates constriction or dilation and the eye microsaccade amplitude passes an eye microsaccade amplitude threshold, the eye microsaccade frequency passes an eye microsaccade frequency threshold, the eye gaze direction changes, or the vergence changes.

Claims

1. A head-mounted display device comprising

an augmented reality display configured to overlay information on a view of an environment of a user of the head-mounted display device;

a camera positioned to capture a plurality of images of at least one eye of the user at a first time period and a second time period after the first time period while the user is wearing the head-mounted display device;

an ambient light sensor;

a speaker;

one or more processors, and

one or more non-transitory computer-readable storage media operatively connected to the one or more processors and collectively comprising instructions, the instructions direct the one or more processors to;

process the plurality of images at the first time period and the second time period to determine a change in size of at least one pupil of the user independent of a change in output from the ambient light sensor;

process the plurality of images at the second time period to determine at least one of an eye microsaccade amplitude, an eye microsaccade frequency, an eye gaze direction, and vergence; and

provide in substantially real-time or subsequent to the second time period a form alert, the form of alert including a form of audio alert signal to the speaker or a form of display alert visible to the user on the augmented reality display when the change in size of at least one pupil indicates constriction or dilation and the eye microsaccade amplitude passes an eye microsaccade amplitude threshold, the eye microsaccade frequency passes an eye microsaccade frequency threshold, the eye gaze direction changes, or the vergence changes.

2. The head mounted display device of claim 1 wherein the instructions further direct the one or more processors to process the plurality of images of at least one eye to extract an iris image or a retina image at the first time period and the second time period to verify the identity of the user of the head-mounted display device and restrict usage of the head-mounted display device if the iris image or the retina image at the second time period does not match the iris image or retina image at the first time period.

3. The head-mounted display device of claim 1 further comprising at least one accelerometer and at least one gyroscope configured to measure an acceleration and an orientation of the head-mounted display device simultaneously or sequentially, wherein the form of display alert visible on the augmented reality display is based in part on the orientation or the acceleration.

4. The head-mounted display device of claim 1 further comprising one or more electrodes configured to monitor electrical brain activity of the user of the head-mounted display device wherein the form of display alert visible on the augmented reality display is based in part on the level of brain activity of the user of the head-mounted display device.