REACTIVE USER INTERFACE FOR HEAD-MOUNTED DISPLAY

• US 20130336629A1
• Filed: 03/13/2013
• Published: 12/19/2013
• Est. Priority Date: 06/19/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: MULHOLLAND, James Joseph, MACIOCCI, Giuliano
• Granted Patent: US 9,219,901 B2
• Time in Patent Office: Days
• US Class Current: 386/230
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: H04N 9/87 Regeneration of colour television signals H04N9/80 takes precedence

Abstract

Embodiments of the present invention are directed toward allowing a see-through HMD to intelligently react to sensor and other input to adjust how visual elements showing video are displayed. The HMD can react to different sensor input in different ways, enabling the HMD to provide graduated responses based on the degree and type of sensor input received. These responses may further be based on a determination of the context of the user to help ensure an intelligent, appropriate response.

Claims

1. A method of providing a reactive video playback interface for a head-mounted display, the method comprising:

causing a display of the head-mounted display to show at least one visual element including a video, wherein the display is at least semi-transparent from a view of a user of the head-mounted display when no content is displayed;

detecting a triggering event using a first set of sensor data from one or more sensors; and

changing a visibility of the video based on the triggering event while the video continues to be shown.

2. The method of claim 1, further comprising determining a context of the user, based on a second set of sensor data, wherein changing the visibility of the video is further based on the determined context.

3. The method of claim 1, wherein the triggering event comprises at least one of:

a certain movement of the user,

a transition of the user from one state of movement to another,

a proximity of an object to the user,

a proximity of the user to a specific person,

entry of the user into a vehicle,

exit of the user from a vehicle,

a pattern of events,

a loud noise, or

arrival of the user at a certain location.

4. The method of claim 3, wherein the certain movement of the user comprises at least one of:

walking, running, or lying down,

a turning of the user'"'"'s head,

the user standing up,

the user sitting down, or

the user falling down.

5. The method of claim 1, wherein the one or more sensors comprise at least one of:

a motion sensor,

an inertial sensor,

a light sensor,

a microphone, or

an antenna.

6. The method of claim 5, wherein the one or more sensors are attached to the head-mounted display.

7. The method of claim 1, wherein changing the visibility of the video includes increasing a transparency of the video.

8. The method of claim 1, wherein changing the visibility of the video includes reducing a size of the video.

9. The method of claim 1, further including displaying at least one new visual element based on the triggering event.

10. The method of claim 1, further including pausing playback of the video.

11. The method of claim 1, wherein:

detecting the triggering event comprises detecting a movement; and

changing the visibility of the video includes changing the visibility of the video in proportion to either or both a speed or an amount of the detected movement.

12. The method of claim 1, further including:

causing the head-mounted display to playback audio related to the video; and

reducing a volume of the audio based on the triggering event.

13. A head-mounted display system comprising:

display means for displaying one or more visual elements to a user such that the user is able to at least selectively view the user'"'"'s surroundings;

sensing means configured to gather sensor data;

means for causing the display means to show a visual element including a video;

means for detecting a triggering event using a set of sensor data from the sensing means; and

means for changing a visibility of the video shown with the display means, based on the triggering event, while the video continues to be shown.

14. The head-mounted display system of claim 13, further comprising

means for determining a context of the user, based on a second set of sensor data;

wherein the visibility change of the video is based on the determined context.

15. The head-mounted display system of claim 13, wherein the means for detecting the triggering event comprise means for determining an event comprising at least one of:

a certain movement of the user,

a transition of the user from one state of movement to another,

a proximity of an object to the user,

a proximity of the user to a specific person,

entry of the user into a vehicle,

exit of the user from a vehicle,

a pattern of events,

a loud noise, or

arrival of the user at a certain location.

16. The head-mounted display system of claim 15, wherein the means for determining comprise means for determining a certain movement by determining an event comprising at least one of:

walking, running, or lying down,

a turning of the user'"'"'s head,

the user standing up,

the user sitting down, or

the user falling down.

17. The head-mounted display system of claim 13, wherein the sensing means comprises at least one of:

a motion sensor,

an inertial sensor,

a light sensor,

a microphone, or

an antenna.

18. The head-mounted display system of claim 13, wherein the means for changing comprise means for changing the visibility of the video by increasing a transparency of the video.

19. The head-mounted display system of claim 13, wherein the means for changing comprise means for changing the visibility of the video by reducing a size of the video.

20. The head-mounted display system of claim 13, further comprising means for causing the display means to display at least one new visual element, based on the triggering event.

21. The head-mounted display system of claim 13, further comprising means for pausing playback of the video.

22. The head-mounted display system of claim 13, further comprising:

means for detecting a movement comprising the triggering event; and

means for changing the visibility of the video in proportion to either or both a speed or an amount of the detected movement.

23. The head-mounted display system of claim 13, further including:

audio means configured to playback audio related to the video;

wherein means for changing reduce a volume of the audio based on the triggering event.

24. A head-mounted display device comprising:

a body configured to be worn on a user'"'"'s head;

a display, coupled with the body and configured to allow the user to view;

the user'"'"'s surroundings, and

a visual element shown with the display;

at least one sensor configured to provide sensor data; and

a processing unit communicatively coupled to the display and the at least one sensor and configured to;

cause the display to show the visual element, the visual element including a video;

detect a triggering event using a first set of sensor data from the at least one sensor; and

cause the display to change a visibility of the video shown with the display, based on the triggering event, while the video continues to be shown.

25. The head-mounted display device of claim 24, wherein the processing unit is further configured to:

determine a context of the user, based on a second set of sensor data; and

cause the display to change the visibility of the video further based on the determined context.

26. The head-mounted display device of claim 24, wherein the processing unit is further configured to determine the triggering event by determining an event comprising at least one of:

a certain movement of the user,

a transition of the user from one state of movement to another,

a proximity of an object to the user,

a proximity of the user to a specific person,

entry of the user into a vehicle,

exit of the user from a vehicle,

a pattern of events,

a loud noise, or

arrival of the user at a certain location.

27. The head-mounted display device of claim 26, wherein the processing unit is further configured to determine the certain movement by determining an event comprising at least one of:

walking, running, or lying down,

a turning of the user'"'"'s head,

the user standing up,

the user sitting down, or

the user falling down.

28. The head-mounted display device of claim 24, wherein the at least one sensor comprises at least one of:

a motion sensor,

an inertial sensor,

a light sensor,

a microphone, or

an antenna.

29. The head-mounted display device of claim 24, wherein the processing unit is configured to cause the display to change the visibility of the video by increasing a transparency of the video.

30. The head-mounted display device of claim 24, wherein the processing unit is configured to cause the display to change the visibility of the video by reducing a size of the video.

31. The head-mounted display device of claim 24, wherein the processing unit is further configured to cause the display to show at least one new visual element, based on the triggering event.

32. The head-mounted display device of claim 24, wherein the processing unit is further configured to pause playback of the video.

33. The head-mounted display device of claim 24, wherein the processing unit is further configured to:

detect the triggering event by detecting a movement; and

change the visibility of the video in proportion to either or both a speed or an amount of the detected movement.

34. The head-mounted display device of claim 24, wherein the processing unit is further configured to:

cause audio related to the video to be played back; and

reduce a volume of the audio based on the triggering event.

35. A non-transitory computer-readable medium having instructions embedded thereon for providing a reactive video playback interface for a head-mounted display, the instructions including computer-executable code for:

causing a display of the head-mounted display to show at least one visual element including a video, wherein the display is at least semi-transparent from a view of a user of the head-mounted display when no content is displayed;

detecting a triggering event using a first set of sensor data from one or more sensors; and

changing a visibility of the video based on the triggering event while the video continues to be shown.