Multiple display modes on a mobile device

• US 10,394,316 B2
• Filed: 04/07/2016
• Issued: 08/27/2019
• Est. Priority Date: 04/07/2016
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Todeschini, Erik
• Primary Examiner(s): Levy, Amy M
• Application Number:
  US15/092,662
• Publication Number:
  US 20170293352A1
• Time in Patent Office: 1,237 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A display control system for controlling visual features on a mobile device, according to one embodiment, includes a display screen disposed on a front side of the mobile device and configured to display at least one user interface element for a user. A front-facing camera disposed on the front side of the mobile device is configured to sense characteristics of the eyes of the user. A processing device processes these characteristics to determine a corresponding point on the display screen where the eyes of the user are directed. A user interface control unit switches the display screen from a regular display mode to a detailed display mode when it is determined that the corresponding point on the display screen where the eyes are directed coincides with a specific user interface element. The detailed display mode includes supplemental information related to the specific user interface element.

Claims

1. A mobile device comprising:

a display screen disposed of the mobile device, the display screen configured to display a user interface element for a user of the mobile device;

a camera disposed of the mobile device, the camera configured to sense a characteristic of an eye of the user;

a processing device configured to process the characteristic of the eye of the user to determine a corresponding point on the display screen where the eye of the user is directed; and

a user interface control unit configured to;

switch the display screen from a regular display mode to a detailed display mode when the processing device determines that the corresponding point on the display screen where the eye of the user is directed coincides with the user interface element, wherein the user interface control unit is configured to switch the display screen from the regular display mode to the detailed display mode based on determination that the eye of the user is directed to a first point on the display screen coinciding with the user interface element for at least a predetermined period of time, wherein the detailed display mode includes supplemental information related to the user interface element,

switch the display screen from the detailed display mode back to the regular display mode based on the determination that the eye of the user is directed to a second point no longer coinciding with the user interface element or supplemental information for at least the predetermined period of time, and

in an instance, when the display screen is switched from the detailed display mode to the regular display mode, further switch the display screen back from the regular display mode to the detailed display mode if the processing device determines that the eye of the user is directed back to the first point in a time period within the predetermined period of time, wherein the time period is shorter than the predetermined period of time.

2. The mobile device of claim 1, wherein each of the user interface element is at least one of an image, a graphic, text, a text box, and a check box.

3. The mobile device of claim 1, wherein the display screen is configured to display a plurality of user interface elements simultaneously.

4. The mobile device of claim 1, wherein the regular display mode includes the user interface element in a simple format.

5. The mobile device of claim 4, wherein the detailed display mode includes the user interface element selected by the user in the simple format and the supplemental information related to the user interface element superimposed over other non-selected user interface elements displayed on the display screen.

6. The mobile device of claim 1, wherein the processing device is configured to determine whether or not the corresponding point on the display screen where the eye of the user is directed falls within a gaze target area, from amongst plurality of gaze target areas of the display screen, the gaze target area being associated with the user interface element.

7. The mobile device of claim 6, wherein the gaze target area is a different size compared to the user interface element.

8. The mobile device of claim 1, wherein the user interface control unit comprises a user interface element position monitoring module, a graphics buffer, and a display mode switching module.

9. The mobile device of claim 8, wherein the user interface element position monitoring module is configured to determine a position of the user interface element on the display screen.

10. The mobile device of claim 8, wherein the graphics buffer is configured to store the supplemental information related to the user interface element.

11. The mobile device of claim 1, wherein the user interface control unit is configured to display a graphical indicator associated with the user interface element to indicate to the user that the detailed display mode is available.

12. The mobile device of claim 1, further comprising a user input device configured to enable the user to set the predetermined period of time and the time period.

13. The mobile device of claim 1, wherein at least one of the predetermined period of time and the time period is user-configurable.

14. The mobile device of claim 1, wherein the user interface control unit is configured to display a graphical indicator associated with the user interface element to indicate to the user that the detailed display mode is available.

15. A mobile device comprising:

a body having a front side and a back side;

a display device disposed on the front side of the body, the display device configured to display user interface elements simultaneously, at least one of the user interface elements comprising at least one of an image element and a text element;

a camera disposed on the front side of the body, the camera configured to capture an image of an eye of a user when the user is viewing the display device; and

a user interface control unit configured to;

receive the image of the eye of the user from the camera;

determine a direction that the eye of the user is pointed;

determine a corresponding point on the display device where the eye of the user is directed; and

switch between a regular display mode and a detailed display mode based on whether the corresponding point on the display device where the eye of the user is directed coincides with one of a plurality of gaze target areas, wherein the user interface control unit is configured to control the display device to switch a display screen of the display device from a regular display mode to the detailed display mode based on determination that the eye of the user is directed to a first point on the display screen coinciding with a gaze target area of the plurality of gaze target areas for at least a predetermined period of time, wherein each gaze target area is associated with a respective user interface element,

switch the display screen from the detailed display mode back to the regular display mode based on the determination that the eye of the user is directed to a second point no longer coinciding with the gaze target area or another predetermined area for at least the predetermined period of time; and

in an instance, when the display screen is switched from the detailed display mode to the regular display mode, further switch the display screen back from the regular display mode to the detailed display mode based on a determination that the eye of the user is directed back to the first point in a time-period within the predetermined period of time, wherein the time-period is shorter than the predetermined period of time.

16. The mobile device of claim 15, wherein the detailed display mode includes a visually-selected user interface element and supplemental information related to the visually-selected user interface element superimposed over at least one other user interface element.

17. The mobile device of claim 15, wherein the user interface control unit comprises:

an eye-gaze module configured to determine a corresponding point on the display device where the eye of the user is directed;

a user interface element position monitoring module configured to determine a position of the gaze target areas on the display device;

a graphics buffer configured to store supplemental information related to the user interface elements; and

a display mode switching module configured to switch between the regular display mode and the detailed display mode based on input from the eye-gaze module and user interface element position monitoring module.

18. The mobile device of claim 15, wherein at least one of the predetermined period of time and the time-period is user-configurable.

19. The mobile device of claim 15, wherein the display device comprises plurality of gaze target areas and wherein the user interface control unit is configured to determine whether or not the corresponding point on the display device where the eye of the user is directed falls within a gaze target area, from amongst plurality of gaze target areas of the display device, the gaze target area being associated with one or more user interface elements.

20. A non-transitory computer-readable medium disposed in a mobile device storing an executable program being executable by a processing device disposed in the mobile device, the executable program comprising logic configured for:

processing an image of an eye of a user of the mobile device to determine a corresponding point on a display screen of the mobile device where the eye of the user is directed;

controlling visual features of the display screen based on whether or not the eye of the user is directed to a user interface element being displayed on the display screen, wherein controlling the visual features of the display screen comprises switching the display screen from a regular display mode to a detailed display mode when the eye of the user is directed to a first point on the display screen coinciding with the user interface element for at least a predetermined period of time, wherein the user interface element is an image element and/or a text element associated with supplemental information to be displayed in a detailed display mode;

switching the display screen from the detailed display mode back to the regular display mode when the eye of the user is directed to a second point no longer coinciding with the user interface element or supplemental information for at least the predetermined period of time; and

in an instance, when the display screen is switched from the detailed display mode to the regular display mode, further switching the display screen back from the regular display mode to the detailed display mode when the eye of the user is directed back to the first point in a time-period within the predetermined period of time, wherein the time-period is shorter than the predetermined period of time.