Automatic text scrolling on a display device

• US 9,395,811 B2
• Filed: 05/23/2014
• Issued: 07/19/2016
• Est. Priority Date: 07/22/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Vaught, Benjamin I, Kipman, Alex Aben-Athar, Crocco, Robert
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US14/285,938
• Publication Number:
  US 20140253437A1
• Time in Patent Office: 788 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2380/02 Flexible displays

Abstract

A see-through head-mounted display (HMD) device, e.g., in the form of glasses, provides view an augmented reality image including text, such as in an electronic book or magazine, word processing document, email, karaoke, teleprompter or other public speaking assistance application. The presentation of text and/or graphics can be adjusted based on sensor inputs indicating a gaze direction, focal distance and/or biological metric of the user. A current state of the text can be bookmarked when the user looks away from the image and subsequently resumed from the bookmarked state. A forward facing camera can adjust the text if a real word object passes in front of it, or adjust the appearance of the text based on a color of pattern of a real world background object. In a public speaking or karaoke application, information can be displayed regarding a level of interest of the audience and names of audience members.

Claims

1. A head-mounted display device, comprising:

a microdisplay, the microdisplay projects augmented reality images comprising advancing text to an eye of a user;

a sensor, the sensor tracks a biological metric of the user while the user performs a vocal presentation of the advancing text; and

a control circuit, the control circuit is configured to control the augmented reality images responsive to the biological metric.

2. The head-mounted display device of claim 1, wherein:

the control circuit, to control the augmented reality images responsive to the biological metric, is configured to advance the text at an advance rate which is based on the biological metric.

3. The head-mounted display device of claim 2, wherein:

the advance rate is increased when the biological metric indicates the user is in a depressed state based on the biological metric being below a threshold and decreased when the biological metric indicates the user is in an excited state based on the biological metric being above the threshold.

4. The head-mounted display device of claim 1, wherein:

the control circuit, to control the augmented reality images responsive to the biological metric, is configured to provide a message which informs the user to speak slower when the biological metric indicates the user is in an excited state based on the biological metric being above a threshold, and informs the user to speak faster when the biological metric indicates the user is in a depressed state based on the biological metric being below the threshold.

5. The head-mounted display device of claim 1, wherein:

the biological metric comprises a heart rate of the user.

6. The head-mounted display device of claim 1, wherein:

the biological metric comprises a speech rate of the user.

7. The head-mounted display device of claim 1, wherein:

the biological metric comprises a pitch of the user.

8. The head-mounted display device of claim 1, wherein:

the biological metric comprises an amount of pupil dilation of the eye.

9. The head-mounted display device of claim 1, wherein:

the biological metric comprises a body temperature of the user.

10. The head-mounted display device of claim 1, wherein:

the biological metric comprises a blood glucose level of the user.

11. A processor-implemented method for providing augmented reality images in a head-mounted display device, the processor-implemented method comprising:

controlling a microdisplay associated with the head-mounted display device to emit light to an eye of a user, the light represents augmented reality images comprising advancing text;

during the controlling, and while a user performs a vocal presentation of the advancing text, receiving a wireless signal at the head-mounted display device from a wireless terminal of another person; and

controlling the augmented reality images, responsive to the wireless signal, to display a message to the eye of the user during the vocal presentation.

12. The processor-implemented method of claim 11, wherein:

the message indicates a level of interest of the another person in the vocal presentation.

13. The processor-implemented method of claim 12, wherein:

the level of interest is entered manually by the another person using the wireless terminal.

14. The processor-implemented method of claim 12, wherein:

the level of interest is determined based on a biological metric of the another person.

15. The processor-implemented method of claim 11, wherein the wireless signal comprises an identifier of the another person, the processor-implemented method further comprising:

determining a name associated with the identifier, the message identifies the name to allow the user to acknowledge a presence of the another person during the vocal presentation.

16. The processor-implemented method of claim 11, wherein the wireless signal comprises an identifier of the another person, the processor-implemented method further comprising:

determining an affiliation associated with the identifier, the message identifies the affiliation to allow the user to acknowledge the affiliation during the vocal presentation.

17. A processor-implemented method for providing augmented reality images in a head-mounted display device, the processor-implemented method comprising:

projecting augmented reality images comprising text to an eye of a user;

tracking a gaze direction of the user;

in response to the tracking, advancing the text based on the gaze direction of the user; and

using a processor;

determining a reading speed of the user based on changes in the gaze direction of the user over time, the changes comprise saccades in which the gaze direction of the user moves along a line of the text and a return sweep saccade in which gaze direction moves to a beginning of a next line of the text; and

advancing the text based on the reading speed of the user.

18. The processor-implemented method of claim 17, further comprising:

determining a gaze pattern of the user based on changes in the gaze direction of the user over time; and

determining a template from among a plurality of templates which is a closest match to the gaze pattern, at least one template of the plurality of templates is associated with a reading disability.

19. The processor-implemented method of claim 17, wherein:

the determining of the reading speed of the user is based on a transition between gaze locations and a number of words encompassed by the transition.

20. The processor-implemented method of claim 17, further comprising:

determining a comprehension level of the user in reading the text based on the gaze direction.