Eye gaze to alter device behavior

• US 10,120,438 B2
• Filed: 05/25/2011
• Issued: 11/06/2018
• Est. Priority Date: 05/25/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Osman, Steven, Stafford, Jeffrey Roger
• Primary Examiner(s): Abimbola, Kehinde O
• Application Number:
  US13/115,927
• Publication Number:
  US 20120300061A1
• Time in Patent Office: 2,722 Days
• Field of Search: 348135
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Methods, devices, and computer programs for controlling behavior of an electronic device are presented. A method includes an operation for operating the electronic device in a first mode of operation, and an operation for tracking the gaze of a user interfacing with the electronic device. The electronic device is maintained in a first mode of operation as long as the gaze is directed towards a predetermined target. In another operation, when the gaze is not detected to be directed towards the predetermined target, the electronic device is operated in a second mode of operation, which is different from the first mode of operation.

Claims

1. A method for controlling behavior of an electronic device, the method comprising:

operating the electronic device to present on a display a picture-in-picture display, the display having a first area and a second area, wherein a First video is presented in the first area of the display and a second video is presented in the second area of the display for the picture-in-picture display, the first video is associated with a first audio component and the second video is associated with a second audio component, the first audio component is presented;

tracking a gaze of a user interfacing with the electronic device;

maintaining the presenting of the first video in the first area the second video in the second area, and the first audio component while the gaze is directed to the first area of the display, but away from the second area of the display;

detecting that the gaze is directed towards the second area of the display for a predetermined amount of time;

switching, in response to the detecting that the gaze is directed towards the second area, a placement of the first video and the second video such that the second video is displayed in the first area and the first video is displayed in the second area; and

presenting the second audio component associated with the second video and muting the

first audio component associated with the first video in response to the detecting that the gaze is directed toward the second area.

2. The method as recited in claim 1, further including:

changing a mode of operation when the gaze of the user is directed away from the

display, wherein the changing the mode of operation includes one or more of pausing the first video, recording the first video, raising or lowering a volume of the first video, and turning off the electronic device.

3. The method as recited in claim 1, wherein tracking the gaze of the user further includes:

checking a direction of the gaze of the user periodically.

4. The method as recited in claim 3, wherein detecting the gaze further includes:

determining that the gaze is directed towards the second area when the gaze is directed towards the second area for a plurality of consecutive periods of checking the direction of the gaze.

5. The method as recited in claim 1, further including:

powering off the electronic device after detecting the gaze is away from the display for a predetermined amount of time.

6. The method as recited in claim 5, further including:

suspending the electronic device alter detecting the gaze is away from the display for a predetermined amount of time.

7. The method as recited in claim 1, wherein tracking the gaze of the user includes operating a plurality of gaze detecting units.

9. The method as recited in claim 1, wherein the switching the placement of the first video and the second video further includes:

prompting the user for authorization to switch the placement of the first video and the second video when the gaze is detected to be directed towards the second area of the display; and

switching the placement of the first video and the second video based on a response from the user to the prompting.

8. An electronic device, comprising:

a display with picture-in-picture display for showing a first video in a first area of the display and a second video is presented in a second area of the display for the picture-in-picture display, wherein the first video is associated with a first audio component and the second video is associated with a second audio component;

a plurality of gaze detection modules, wherein information from the plurality of gaze detection modules is combined to determine a direction of a gaze of a user interfacing with the display;

a processor that detects if the direction of the gaze is directed towards the first area or the second area, wherein the processor switches a placement of the first video and the second video when the processor detects that the gaze is directed towards the second area for a predetermined amount of time, such that the first video is shown in the second area of the display and the second video is shown in the first area of the display; and

wherein the first audio component is presented and the second audio component is muted when the first video is shown in the first area of the display and wherein the second audio component is presented and the first audio component is muted when the second video is shown in the first area of the display.

10. The electronic device of claim 8, wherein the processor is configured to change a mode of operation when the gaze of the user is directed away from the display, wherein the changing the mode of operation includes one or more of pausing the first video, recording the First video, raising or lowering a volume of the first video, and turning off the electronic device.

11. The electronic device of claim 8, wherein the processor is configured to check the direction of the gaze of the user periodically.

12. The electronic device of claim 11, wherein the processor is further configured for determining that the gaze is directed towards the second area when the gaze is directed towards the second area for a plurality of consecutive periods of checking the direction of the gaze.

13. The electronic device of claim 8, wherein the processor is further configured to power off the electronic device after detecting the gaze is away from the display for a predetermined amount of time.

14. The electronic device of claim 8, wherein the processor is further configured to suspend the electronic device after detecting the gaze is away from the display for a predetermined amount of time.

15. The electronic device of claim 8, further comprising:

a plurality of gaze detection units for tracking the direction of the gaze.