Gaze assisted field of view control

• US 10,686,972 B2
• Filed: 03/30/2018
• Issued: 06/16/2020
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Ronngren, Denny
• Primary Examiner(s): Yang, Nan-Ying
• Application Number:
  US15/941,624
• Publication Number:
  US 20180227470A1
• Time in Patent Office: 809 Days
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04R 1/1041 : Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 1/32 : for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 2201/40 : Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
• CPC Class Codes: H04R 2460/07 : Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
• CPC Class Codes: H04R 2499/11 : Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's

Abstract

A method for rotating a field of view represented by a displayed image is disclosed. The method may include displaying a first image representing a first field of view. The method may also include determining a gaze direction of a user toward the first image. The method may further include identifying a subject in the first image at which the gaze direction is directed, wherein the subject is in a first direction from a center of the first image. The method may further include receiving a directional input in a second direction. The method may additionally include, based at least in part on the second direction being substantially the same as the first direction, displaying a second image representing a second field of view, wherein the subject is centered in the second image.

Claims

1. A method for rotating a field of view represented by a displayed image, wherein the method comprises:

displaying, with a display device, a first image representing a first field of view;

determining, with an eye tracking device, a gaze direction of a user toward the first image;

identifying, with a processing device, a subject in the first image at which the gaze direction is directed, wherein the subject is in a first direction from a center of the first image;

receiving, at a handheld input device, a directional input in a second direction;

based at least in part on the second direction being substantially the same as the first direction, displaying, with the display device, a second image representing a second field of view, wherein the subject is centered in the second image; and

wherein the subject being centered in the second image comprises;

the subject being centered horizontally in the second image, and

the subject is not centered vertically in the second image.

2. The method for rotating a field of view represented by a displayed image of claim 1, wherein the display device comprises:

a head mounted display worn by the user.

3. The method for rotating a field of view represented by a displayed image of claim 1, wherein the display device comprises:

a television, monitor, or projection display device.

4. The method for rotating a field of view represented by a displayed image of claim 1, wherein the second direction being substantially the same as the first direction comprises:

the second direction being in the same horizontal direction as the first direction.

5. The method for rotating a field of view represented by a displayed image of claim 1, wherein:

the second field of view represents a horizontal pan from the first field of view.

6. A system for rotating a field of view represented by a displayed image, wherein the system comprises:

a display device;

an eye tracking device;

a handheld input device; and

a processing device in communication with the display device, the eye tracking device, and the handheld input device, wherein the processing device is configured to;

cause the display device to display a first image representative of a first field of view;

receive, from the eye tracking device, data regarding a gaze direction of the user toward the first image;

identify a subject in the first image at which the gaze direction is directed, wherein the subject is in a first direction from a center of the first image;

receiving data regarding a directional input in a second direction received at the handheld input device;

based at least in part on the second direction being substantially the same as the first direction, causing the display device to display a second image representing a second field of view, wherein the subject is centered in the second image; and

wherein the subject being centered in the second image comprises;

the subject being centered horizontally in the second image, and

the subject is not centered vertically in the second image.

7. The system for rotating a field of view represented by a displayed image of claim 6, wherein the display device comprises:

a head mounted display worn by the user.

8. The system for rotating a field of view represented by a displayed image of claim 6, wherein the display device comprises:

a television, monitor, or projection display device.

9. The system for rotating a field of view represented by a displayed image of claim 6, wherein the second direction being substantially the same as the first direction comprises:

the second direction being in the same horizontal direction as the first direction.

10. The system for rotating a field of view represented by a displayed image of claim 6, wherein:

the second field of view represents a horizontal pan from the first field of view.

11. A non-transitory machine readable medium having instructions stored thereon for rotating a field of view represented by a displayed image, wherein the instructions are executable by one or more processors to at least:

display, with a display device, a first image representing a first field of view;

determine, with an eye tracking device, a gaze direction of a user toward the first image;

identify a subject in the first image at which the gaze direction is directed, wherein the subject is in a first direction from a center of the first image;

receive, from a handheld input device, a directional input in a second direction;

based at least in part on the second direction being substantially the same as the first direction, display, with the display device, a second image representing a second field of view, wherein the subject is centered in the second image; and

wherein the subject being centered in the second image comprises;

the subject being centered horizontally in the second image, and

the subject is not centered vertically in the second image.

12. The non-transitory machine readable medium of claim 11, wherein the display device comprises:

a head mounted display worn by the user.

13. The non-transitory machine readable medium of claim 11, wherein the display device comprises:

a television, monitor, or projection display device.

14. The non-transitory machine readable medium of claim 11, wherein

the second direction being substantially the same as the first direction comprises:

the second direction being in the same horizontal direction as the first direction.