EYE-TRACKING-BASED METHODS AND SYSTEMS OF MANAGING MULTI-SCREEN VIEW ON A SINGLE DISPLAY SCREEN

• US 20170160799A1
• Filed: 10/09/2015
• Published: 06/08/2017
• Est. Priority Date: 05/04/2015
• Status: Active Grant

• Current Assignee: Huizhou TCL Mobile Communication Company Limited (TCL Technology Group Corp.)
• Original Assignee: Huizhou TCL Mobile Communication Company Limited (TCL Technology Group Corp.)
• Inventors: SHI, Zhen
• Granted Patent: US 10,802,581 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

An eye-tracking-based method and system of managing multi-screen view on a single display screen are disclosed. The method includes: when the display screen of a mobile terminal is under the multi-screen mode, determining the user'"'"'s current eye gaze coordinates on the display screen using eye-tracking technology; determining whether a duration the user'"'"'s gaze point stays within a specified range about the current gaze coordinates exceeds a preset gaze threshold, if exceeds, analyzing the user'"'"'s eye movement information using the eye-tracking technology; when the eye movement information denotes an open-eye state, then determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds a preset full-screen threshold, if exceeds, display in full screen the split screen where the current gaze coordinates are located. Thus, the eye-tracking technology can be used to obtain the current gaze coordinates and the corresponding gaze duration and the eye movement information, and determinations can then be made whether to make the split screen of interest full-screen or to exit the full-screen view, providing convenience to the user.

Claims

1. An eye-tracking-based method of managing multi-screen view on a display screen, comprising:

current eye gaze coordinates of a user on the display screen by capturing eye images in real time and employing eye-tracking technology when the display screen is under a multi-screen mode, wherein a plurality of split screens are displayed on the display screen in the multi-screen mode;

determining whether a duration a gaze point of the user stays within a specified range about the current gaze coordinates exceeds a preset gaze threshold, and if exceeds, analyzing eye movement information of the user using the eye-tracking technology;

determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds a preset full-screen threshold when the eye movement information denotes an open-eye state, and if exceeds, displaying on the display screen in a full-screen view, at least one split screen of the plurality of split screens where the current gaze coordinates are located;

determining the current eye gaze coordinates on the display screen using the eye-tracking technology after having entered the full-screen view;

exiting the full-screen view and restoring to the multi-screen mode if the current gaze coordinates are located within a specified area of the display screen and the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold; and

closing the at least one split screen where the current gaze coordinates are located when the eye movement information denotes a closed-eye state.

2. The eye-tracking-based method according to claim 1, wherein determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold comprises:

determining whether the current eye gaze coordinates are located within a multi-screen region or a split-screen templates region of the display screen;

determining further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold when the current gaze coordinates are located within the multi-screen region, and if exceeds, analyzing the eye movement information using the eye-tracking technology; and

determining further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold when the current gaze coordinates are located within the split-screen templates region, and if exceeds, entering a preset split-screen templates selecting and managing interface.

3. The eye-tracking-based method according to claim 1, wherein the preset gaze threshold lies in a range of 5-10 seconds.

4. The eye-tracking-based method according to claim 1, wherein the display screen of the mobile terminal comprises 4 split screens under the multi-screen mode.

5. An eye-tracking-based method of managing multi-screen view on a display screen, comprising:

determining current eye gaze coordinates of a user on the display screen by capturing eye images in real time and employing eye-tracking technology when the display screen is under a multi-screen mode, wherein a plurality of split screens are displayed on the display screen in the multi-screen mode;

determining whether a duration a gaze point of the user stays within a specified range about the current gaze coordinates exceeds a preset gaze threshold, and if exceeds, analyzing eye movement information of the user using the eye-tracking technology;

determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds a preset full-screen threshold when the eye movement information denotes an open-eye state, and if exceeds, displaying on the display screen in a full screen view, one split screen of the plurality of split screens where the current gaze coordinates are located;

6. The eye-tracking-based method according to claim 5, further comprising, after determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds a preset full-screen threshold when the eye movement information denotes an open-eye state:

closing the split screen where the current gaze coordinates are located when the eye movement information denotes a closed-eye state.

7. The eye-tracking-based method according to claim 6, wherein determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold comprises:

determining whether the current eye gaze coordinates are located within a multi-screen region or a split-screen templates region of the display screen;

determining further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold when the current gaze coordinates are located within the multi-screen region, and if exceeds, analyzing the eye movement information using the eye-tracking technology; and

determining further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold when the current gaze coordinates are located within the split-screen templates region of the display screen, and if exceeds, entering a preset split-screen templates selecting and managing interface.

8. The eye-tracking-based method according to claim 5, wherein determining whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset full-screen threshold when the eye movement information denotes an open-eye state further comprises:

determining the current eye gaze coordinates on the display screen using the eye-tracking technology after having entered the full-screen view, and if determining the current eye gaze coordinates are located within a specified area of the display screen and the duration the gaze point stays within a specified range about the current gaze coordinates exceeds the preset gaze threshold, exiting the full-screen view and restoring to the multi-screen mode.

9. The eye-tracking-based method according to claim 5, wherein the preset gaze threshold lies in a range of 5-10 seconds.

10. The eye-tracking-based method according to claim 5, wherein the display screen of the mobile terminal comprises 4 split screens under the multi-screen mode.

11. An eye-tracking-based system of managing multi-screen view on a display screen, comprising a processor, and a non-transitory program storage medium which comprises:

a positioning module configured to, when the display screen is under a multi-screen mode, determine current eye gaze coordinates of a user on the display screen by capturing eye images in real time and employing eye-tracking technology;

a determination and acquisition module configured to determine whether a duration the a gaze point of the user stays within a specified range about the current gaze coordinates exceeds a preset gaze threshold, and if exceeds, analyze the eye movement information of the user using the eye-tracking technology; and

a full-screen control module configured to, when the eye movement information denotes an open-eye state, determine whether the duration the gaze point stays within a specified range about the current gaze coordinates exceeds a preset full-screen threshold, and if exceeds, display in a full-screen view, a split screen where the current gaze coordinates are located.

12. The eye-tracking-based system according to claim 11, further comprising:

a split-screen-closing module configured to, when the eye movement information denotes a closed-eye state, close the split screen where the current gaze coordinates are located.

13. The eye-tracking-based system according to claim 11, wherein the determination and acquisition module comprises:

a region-determination unit configured to determine whether the current eye gaze coordinates are located within a multi-screen region or a split-screen templates region of the display screen;

a first control unit configured to, if the current gaze coordinates are located within the multi-screen region, determine further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold, and if exceeds, analyze the eye movement information using the eye-tracking technology; and

a second control unit configured to, if the current gaze coordinates are located within the split-screen templates region, determine further whether the duration the gaze point stays within the specified range about the current gaze coordinates exceeds the preset gaze threshold, and if exceeds, enter a preset split-screen templates selecting and managing interface.

14. The eye-tracking-based system according to claim 11, wherein the full-screen control module is further configured to, after having entered the full-screen view, determine the current eye gaze coordinates on the display screen using the eye-tracking technology, and if determining the current eye gaze coordinates are located within a specified area of the display screen and the duration the gaze point stays within a specified range about the current gaze coordinates exceeds the preset gaze threshold, exit the full-screen view and restore to the multi-screen mode.

15. The eye-tracking-based system according to claim 11, wherein the preset gaze threshold lies in a range of 5-10 seconds.

16. The eye-tracking-based system according to claim 11, wherein the display screen of the mobile terminal comprises 4 split screens under the multi-screen mode.