Eye Gaze to Alter Device Behavior

• US 20120300061A1
• Filed: 05/25/2011
• Published: 11/29/2012
• Est. Priority Date: 05/25/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven, Stafford, Jeffrey Roger
• Granted Patent: US 10,120,438 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Methods, devices, and computer programs for controlling behavior of an electronic device are presented. A method includes an operation for operating the electronic device in a first mode of operation, and an operation for tracking the gaze of a user interfacing with the electronic device. The electronic device is maintained in a first mode of operation as long as the gaze is directed towards a predetermined target. In another operation, when the gaze is not detected to be directed towards the predetermined target, the electronic device is operated in a second mode of operation, which is different from the first mode of operation.

Claims

1. A method for controlling behavior of an electronic device, the method comprising:

operating the electronic device in a first mode of operation;

tracking a gaze of a user interfacing with the electronic device;

maintaining the electronic device in the first mode of operation as long as the gaze is directed towards a predetermined target;

detecting that the gaze is not directed towards the predetermined target; and

operating the electronic device in a second mode of operation different from the first mode of operation after detecting that the gaze is not directed towards the predetermined target.

2. The method as recited in claim 1,

wherein the electronic device is a car navigation system;

wherein the predetermined target is a display for the car navigation system;

wherein the first mode of operation includes presenting directions on a map in the display; and

wherein the second mode of operation includes giving verbal driving directions when the user is not looking at the display.

3. The method as recited in claim 1,

wherein the electronic device is an electronic toy;

wherein the predetermined target is the electronic toy;

wherein the first mode of operation includes interactive play; and

wherein the second mode of operation includes issuing verbal cues to attract attention to the electronic toy.

4. The method as recited in claim 1,

wherein the electronic device is a television screen with picture-in-picture display;

wherein the television screen switches a video shown in the television screen with a video shown in the picture-in-picture display when the gaze of the user is on the picture-in-picture display for a predetermined amount of time.

5. The method as recited in claim 1,

wherein the electronic device is a television screen;

wherein the predetermined target is the television screen;

wherein the first mode of operation includes showing a video on the television screen; and

wherein the second mode of operation includes one or more of pausing the video, recording the video, raising or lowering volume of the television screen, and turning off the television screen.

6. The method as recited in claim 1,

wherein the electronic device includes a display;

wherein the predetermined target is the display;

wherein the first mode of operation includes a first level of brightness for the display; and

wherein the second mode of operation includes a second level of brightness for the display different from the first level of brightness.

7. The method as recited in claim 1, wherein tracking the gaze of the user further includes:

checking a direction of the gaze of the user periodically.

8. The method as recited in claim 7, wherein detecting the gaze further includes:

determining that the gaze is away from the predetermined target when the gaze is away from the predetermined target for a plurality of consecutive periods of checking the direction of the gaze.

9. The method as recited in claim 8, further including:

switching the electronic device to operate in the first mode of operation when the gaze is directed again to the predetermined target while operating the device in the second mode of operation.

10. The method as recited in claim 1, further including:

operating the electronic device in a third mode of operation after a predetermined amount of time operating in the second mode of operation while the gaze stays directed away from the predetermined target.

11. The method as recited in claim 10, wherein the third mode of operation includes powering off or suspending the electronic device.

12. An electronic device, comprising:

a gaze detection module that determines if a gaze of a user interfacing with the electronic device is directed towards a predetermined target; and

a processor with different modes of operation which include a first mode of operation and a second mode of operation,

wherein the processor operates in the first mode of operation while the gaze of the user is directed towards the predetermined target, wherein the processor operates in the second mode of operation after the gaze detection module detects that the gaze of the user is not directed towards the predetermined target.

13. The electronic device of claim 12, wherein the electronic device, when operating in the second mode of operation, is controlled by the user without requiring visual cues from the electronic device to the user.

14. The electronic device of claim 12, further including:

a television screen in communication with the processor, wherein the predetermined target is the television screen,

wherein the first mode of operation includes the television screen powered on,

wherein the second mode of operation includes the television screen powered off.

15. The electronic device of claim 12, further including:

input devices in communication with the processor; and

a display, wherein the predetermined target is the display,

wherein the second mode of operation includes reducing power delivered to the input devices while the user is looking away from the display.

16. The electronic device of claim 12, further including:

a mobile phone in communication with the processor, wherein the predetermined target is the mobile phone.

17. The electronic device of claim 12, further including:

a car that includes the processor, wherein the predetermined target is an area in front of the car,

wherein the first mode of operation does not generate alarms,

wherein the second mode of operation generate alarms if the car enters a possible collision situation with an obstacle in the area in front of the car.

18. The electronic device of claim 12, further including:

a plurality of video screens, each video screen showing one of a plurality of videos, one of the video screens being a main video screen,

wherein the gaze detection module determines the gaze of all users watching the plurality of video screens and determines which of the video screens has the gaze of a majority of the users,

wherein the processor switches the video associated with the video screen that has the gaze of the majority of the users to the main screen when the majority is not associated with the main screen.

19. The electronic device of claim 12, further including:

a display divided in a plurality of display areas, each display area showing one of a plurality of videos,

wherein the gaze detection module determines the gaze of all users watching the plurality of display areas and determines which of the display area has the gaze of a minority of the users,

wherein the processor changes a video feed of the display area that has the gaze of the minority of the users, and wherein an audio associated with the video feed is reproduced.

20. The electronic device of claim 12, further including:

a plurality of video screens, each video screen showing one of a plurality of videos,

wherein the gaze detection module determines which of the video screens has the gaze the user,

wherein the processor provides more computing resources for processing the video shown on the video screen that has the gaze of the user than computing resources provided to the video screens that do not have the gaze of the user.

21. The electronic device of claim 12, further including:

a display which is divided into a plurality of display areas, wherein the gaze detection module determines which display area corresponds to the gaze of the user,

wherein the processor delivers a higher quality of service to the display area corresponding to the gaze of the user.

22. The electronic device of claim 21, wherein the higher quality of service includes reproducing the audio of the display area corresponding to the gaze of the user.

23. The electronic device of claim 12, further including:

an alarm clock, wherein the predetermined target is the alarm clock,

wherein the first mode of operation includes an alarm ringing,

wherein the second mode of operation includes turning off the alarm ringing after detecting the gaze on the alarm clock.

24. The electronic device of claim 12, further including:

a display, wherein the predetermined target is the display,

wherein the first mode of operation includes disabling a screensaver program while the gaze of the user is on the display,

wherein the second mode of operation includes enabling a timer to start the screensaver program while the gaze of the user is not the display.

25. A computer program embedded in a non-transitory computer-readable storage medium, when executed by one or more processors, for controlling behavior of an electronic device, the computer program comprising:

program instructions for operating the electronic device in a first mode of operation;

program instructions for tracking a gaze of a user interfacing with the electronic device;

program instructions for maintaining the electronic device in a first mode of operation as long as the gaze is directed towards a predetermined target;

program instructions for detecting that the gaze is not directed towards the predetermined target; and

program instructions for operating the electronic device in a second mode of operation different from the first mode of operation after detecting that the gaze is not directed towards the predetermined target.

26. The computer program as recited in claim 25, wherein tracking the gaze of the user includes operating a plurality of gaze detecting units.

27. The computer program as recited in claim 26, further including:

program instructions for operating the electronic device in a third mode of operation when the gaze is directed towards a second target.

28. An electronic device, comprising:

a plurality of gaze detection modules, wherein information from the plurality of gaze detection modules is combined to determine a direction of a gaze of a user interfacing with the electronic device; and

a processor, wherein the electronic device has a plurality of modes of operation, each mode of operation being associated with a corresponding target area for the direction of the gaze, wherein the processor detects if the direction of the gaze is directed to one of the target areas, and wherein the electronic device operates in one of the modes of operation from the plurality of modes of operation when the processor detects that the direction of the gaze is towards the corresponding target area associated with the mode of operation.