Delay of display event based on user gaze

• US 10,162,478 B2
• Filed: 10/08/2015
• Issued: 12/25/2018
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chaudhri, Imran, Lemay, Stephen O.
• Primary Examiner(s): Lam, Vinh T
• Application Number:
  US14/878,983
• Publication Number:
  US 20160034141A1
• Time in Patent Office: 1,174 Days
• Field of Search: 345156, 345157-158, 345173-174
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Methods and systems of delaying the execution of a display event based on a detected user gaze are provided. Display events may be generated and executed to change a user interface of a display. For example, an autocorrect algorithm can automatically replace a typed word with a corrected word in a text field, generating a display event that causes the corrected word to be displayed instead of the typed word. Such a display event may be executed as soon as possible after its generation. However, a gaze detection device can obtain information that indicates a user is not looking at the typed word on the display. In such a situation, it may be more intuitive to delay the execution of the display event until the gaze information indicates that the user is looking at the typed word.

Claims

1. A method comprising:

at an electronic device with a display and a gaze detection device;

displaying, in a target region of the display, one or more words entered by a user of the electronic device;

while displaying the one or more words, receiving a sequence of character inputs;

in response to receiving the sequence of character inputs, displaying, in the target region, a sequence of user-input characters that corresponds to the sequence of character inputs, wherein the sequence of user-input characters has an associated sequence of autocorrect characters that is different from the sequence of user-input characters; and

after displaying the sequence of user-input characters;

determining whether a gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delaying replacing the sequence of user-input characters with the sequence of autocorrect characters.

2. The method of claim 1, further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters, determining that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters.

3. The method of claim 2, further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters;

receiving a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, displaying, on the display, a second sequence of user-input characters that corresponds to the second sequence of character inputs,

wherein determining that the gaze region overlaps with the target region, and replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters, occur while displaying, on the display, the second sequence of user-input characters.

4. The method of claim 1, further comprising:

concurrently displaying the sequence of user-input characters with an autocorrect indicator that includes the sequence of autocorrect characters,

wherein;

replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters includes ceasing display of the autocorrect indicator, and

delaying replacing the sequence of user-input characters with the sequence of autocorrect characters includes maintaining display of the autocorrect indicator.

5. The method of claim 1, further comprising:

while displaying the sequence of user-input characters;

receiving a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, displaying, in the target region, a second sequence of user-input characters that corresponds to the second sequence of character inputs, wherein the second sequence of user-input characters has an associated second sequence of autocorrect characters that is different from the second sequence of user-input characters; and

while displaying the sequence of user-input characters and the second sequence of user-input characters;

determining whether the gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replacing, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delaying replacing;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

6. The method of claim 5, further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters and the second sequence of user-input characters with the second sequence of autocorrect characters, determining that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replacing, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

7. The method of claim 1, wherein the target region includes an entirety of the display.

8. A non-transitory computer readable storage medium having stored therein instructions, which when executed by an electronic device with a display and a gaze detection device, cause the electronic device to perform a method comprising:

displaying, in a target region of the display, one or more words entered by a user of the electronic device;

while displaying the one or more words, receiving a sequence of character inputs;

in response to receiving the sequence of character inputs, displaying, in the target region, a sequence of user-input characters that corresponds to the sequence of character inputs, wherein the sequence of user-input characters has an associated sequence of autocorrect characters that is different from the sequence of user-input characters; and

after displaying the sequence of user-input characters;

determining whether a gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delaying replacing the sequence of user-input characters with the sequence of autocorrect characters.

9. The non-transitory computer readable storage medium of claim 8, the method further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters, determining that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters.

10. The non-transitory computer readable storage medium of claim 9, the method further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters;

receiving a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, displaying, on the display, a second sequence of user-input characters that corresponds to the second sequence of character inputs,

wherein determining that the gaze region overlaps with the target region, and replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters, occur while displaying, on the display, the second sequence of user-input characters.

11. The non-transitory computer readable storage medium of claim 8, the method further comprising:

concurrently displaying the sequence of user-input characters with an autocorrect indicator that includes the sequence of autocorrect characters,

wherein;

replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters includes ceasing display of the autocorrect indicator, and

delaying replacing the sequence of user-input characters with the sequence of autocorrect characters includes maintaining display of the autocorrect indicator.

12. The non-transitory computer readable storage medium of claim 8, the method further comprising:

while displaying the sequence of user-input characters;

receiving a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, displaying, in the target region, a second sequence of user-input characters that corresponds to the second sequence of character inputs, wherein the second sequence of user-input characters has an associated second sequence of autocorrect characters that is different from the second sequence of user-input characters; and

while displaying the sequence of user-input characters and the second sequence of user-input characters;

determining whether the gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replacing, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delaying replacing;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

13. The non-transitory computer readable storage medium of claim 12, the method further comprising:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters and the second sequence of user-input characters with the second sequence of autocorrect characters, determining that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replacing, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

14. The non-transitory computer readable storage medium of claim 8, wherein the target region includes an entirety of the display.

15. An electronic device, comprising:

a display;

a gaze detection device;

a memory configured to store instructions; and

a processor coupled to the memory and configured to execute the instructions and;

display, in a target region of the display, one or more words entered by a user of the electronic device;

while displaying the one or more words, receive a sequence of character inputs;

in response to receiving the sequence of character inputs, display, in the target region, a sequence of user-input characters that corresponds to the sequence of character inputs, wherein the sequence of user-input characters has an associated sequence of autocorrect characters that is different from the sequence of user-input characters; and

after displaying the sequence of user-input characters;

determine whether a gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replace, on the display, the sequence of user-input characters with the sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delay replacing the sequence of user-input characters with the sequence of autocorrect characters.

16. The electronic device of claim 15, the processor further configured to:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters, determine that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replace, on the display, the sequence of user-input characters with the sequence of autocorrect characters.

17. The electronic device of claim 16, the processor further configured to:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters;

receive a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, display, on the display, a second sequence of user-input characters that corresponds to the second sequence of character inputs,

wherein determining that the gaze region overlaps with the target region, and replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters, occur while displaying, on the display, the second sequence of user-input characters.

18. The electronic device of claim 15, the processor further configured to:

concurrently display the sequence of user-input characters with an autocorrect indicator that includes the sequence of autocorrect characters,

wherein;

replacing, on the display, the sequence of user-input characters with the sequence of autocorrect characters includes ceasing display of the autocorrect indicator, and

delaying replacing the sequence of user-input characters with the sequence of autocorrect characters includes maintaining display of the autocorrect indicator.

19. The electronic device of claim 15, the processor further configured to:

while displaying the sequence of user-input characters;

receive a second sequence of character inputs; and

in response to receiving the second sequence of character inputs, display, in the target region, a second sequence of user-input characters that corresponds to the second sequence of character inputs, wherein the second sequence of user-input characters has an associated second sequence of autocorrect characters that is different from the second sequence of user-input characters; and

while displaying the sequence of user-input characters and the second sequence of user-input characters;

determine whether the gaze region of the user of the electronic device, obtained from the gaze detection device, and the target region of the display overlap;

in accordance with a determination that the gaze region and the target region do overlap, replace, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters; and

in accordance with a determination that the gaze region and the target region do not overlap, delay replacing;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

20. The electronic device of claim 19, the processor further configured to:

after delaying replacing the sequence of user-input characters with the sequence of autocorrect characters and the second sequence of user-input characters with the second sequence of autocorrect characters, determine that the gaze region of the user of the electronic device overlaps with the target region of the display; and

in response to determining that the gaze region overlaps with the target region, replace, on the display;

the sequence of user-input characters with the sequence of autocorrect characters; and

the second sequence of user-input characters with the second sequence of autocorrect characters.

21. The electronic device of claim 15, wherein the target region includes an entirety of the display.