Graphical menu and interaction therewith through a viewing window

• US 8,643,951 B1
• Filed: 03/15/2012
• Issued: 02/04/2014
• Est. Priority Date: 03/15/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wheeler, Aaron Joseph, Kauffmann, Alejandro, Chi, Liang-Yu (Tom), Raffle, Hayes Solos, Gomez, Luis Ricardo Prada
• Primary Examiner(s): Sugarman, Scott J
• Assistant Examiner(s): Choudhury, Mustak
• Application Number:
  US13/421,709
• Time in Patent Office: 691 Days
• Field of Search: 359630-632
• US Class Current: 359/630
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

Methods and systems involving a navigable area in a head-mounted display (HMD) are disclosed herein. An exemplary system may be configured to: (a) cause a head-mounted display (HMD) to provide: (i) a viewing window that is fully visible in a field of view of the HMD, (ii) a navigable area that is not fully visible in a field of view of the HMD such that the viewing window displays a first portion of the navigable area, and (iii) a media object that is associated with a viewing-window state; (b) receive first head-movement data that is indicative of head movement from a first position of the HMD to a second position of the HMD; and (c) based on (i) the first head-movement data and (ii) the viewing-window state, cause the viewing window to display a second portion of the navigable area which includes the media object.

Claims

1. A system comprising:

a non-transitory computer readable medium; and

program instructions stored on the non-transitory computer readable medium and executable by at least one processor to;

cause a head-mounted display (HMD) to provide (i) a viewing window that is fully visible in a field of view of the HMD, (ii) a navigable area that is not fully visible in a field of view of the HMD, wherein the viewing window displays a first portion of the navigable area, and (iii) at least one media object, wherein the at least one media object is associated with at least one viewing-window state;

receive first head-movement data that is indicative of head movement from a first position of the HMD to a second position of the HMD; and

based on at least (i) the first head-movement data, (ii) the at least one viewing-window state associated with the at least one media object, and (iii) a determination that at least one threshold velocity has been exceeded, cause the viewing window to display a second portion of the navigable area, wherein the second portion includes the at least one media object.

2. The system of claim 1, wherein the at least one media object is associated with at least a gravity-viewing-window state, the system further comprising program instructions stored on the non-transitory computer readable medium and executable by at least one processor to:

determine that the at least one viewing-window state is the gravity-viewing-window state associated with the media object; and

cause the viewing window to fixate upon the media object.

3. The system of claim 2, wherein the at least one threshold velocity corresponds to the gravity-viewing-window state, the system further comprising program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to:

while the viewing window is fixated upon the media object, receive second head-movement data;

determine that a velocity corresponding to the second head-movement data is greater than the at least one threshold velocity; and

based on the determination, cause the viewing window to move away from the media object.

4. The system of claim 1, wherein the at least one media object is associated with at least a rail-viewing-window state, the system further comprising program instructions stored on the non-transitory computer readable medium and executable by at least one processor to:

after causing the viewing window to display the second portion of the navigable area, receive input-selection data indicating a selection of the at least one media object;

based on the selection of the at least one media object, determine that the at least one viewing-window state is the rail-viewing-window state associated with the at least one media object;

receive second head-movement data that is indicative of head movement from a third position of the HMD to a fourth position of the HMD;

determine a portion of the head movement indicated by the second head-movement data that corresponds to a rail defined by the rail-viewing-window state; and

after the determination, cause the viewing window to move corresponding to the rail.

5. The system of claim 4, wherein a direction parameter maps a parallel direction indicated by the second head-movement data to a viewing-window direction along the rail, the system further comprising program instructions stored on the non-transitory computer readable medium and executable by at least one processor to:

after determining the at least one viewing-window state is the rail-viewing-window state, use the parallel direction indicated by second head-movement to control the viewing-window direction along the rail.

6. The system of claim 5, wherein the direction parameter maps a perpendicular direction indicated by the second head-movement data to a viewing-window direction perpendicular to the rail, and wherein the at least one threshold velocity comprises a perpendicular-direction-velocity threshold, the system further comprising program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to:

determine that a velocity corresponding to the perpendicular direction in the second head-movement data exceeds the perpendicular-direction-velocity threshold; and

after the determination, cause the viewing window to move perpendicular to the rail.

7. The system of claim 4, wherein a speed parameter maps a given amount of head movement to an amount of viewing window movement, the system further comprises program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to:

after determining the at least one viewing-window state is the rail-viewing-window state, increase the speed parameter so as to increase the amount of viewing window movement that is mapped to the given amount of head movement.

8. The system of claim 4, wherein the rail defines a rail portion of the navigable area allowing viewing window movement, and wherein causing the viewing window to move corresponding to the rail comprises constraining the movement of the viewing window to be within the rail portion of the navigable area.

9. A computer-implemented method comprising:

causing a head-mounted display (HMD) to provide (i) a viewing window that is fully visible in a field of view of the HMD, (ii) a navigable area that is not fully visible in a field of view of the HMD, wherein the viewing window displays a first portion of the navigable area, and (iii) at least one media object, wherein the at least one media object is associated with at least one viewing-window state;

receiving first head-movement data that is indicative of head movement from a first position to a second position; and

based on at least (i) the first head-movement data, (ii) the at least one viewing-window state associated with the at least one media object, and (iii) a determination that at least one threshold velocity has been exceeded, causing the viewing window to display a second portion of the navigable area, wherein the second portion includes the at least one media object.

10. The method of claim 9, wherein (i) the viewing window, (ii) the navigable area, and (iii) the at least one media object are provided through the HMD, and wherein the head movement comprises movement of human eyes.

11. The method of claim 9, wherein the at least one media object is associated with at least a gravity-viewing-window state, the method further comprises:

determining that the at least one viewing-window state is the gravity-viewing-window state associated with the media object; and

causing the viewing window to fixate upon the media object.

12. The method of claim 10, wherein the at least one threshold velocity comprises a threshold acceleration, and wherein determining the at least one viewing-window state is a gravity-viewing-window state associated with the gravity object comprises:

while the viewing window is fixated upon the media object, receiving second head-movement data;

determining that an acceleration corresponding to the second head-movement data is greater than the threshold acceleration; and

based on the determination, causing the viewing window to move away from the media object.

13. The method of claim 9, wherein the at least one media object is associated with at least a rail-viewing-window state, the method further comprises:

after causing the viewing window to display the second portion of the navigable area, receiving input-selection data indicating a selection of the at least one media object;

based on the selection of the at least one media object, determining that the at least one viewing-window state is a rail-viewing-window state associated with the at least one media object;

receiving second head-movement data that is indicative of head movement from a third position of the HMD to a fourth position of the HMD;

determining a portion of the head movement indicated by the second head-movement data that corresponds to a rail defined by the rail-viewing-window state; and

after the determination, causing the viewing window to move corresponding to the rail.

14. The method of claim 13, wherein a direction parameter maps a parallel direction indicated by the second head-movement data to a viewing-window direction along the rail, the method further comprises:

after determining the at least one viewing-window state is the rail-viewing-window state, using the parallel direction indicated by second head-movement to control the viewing-window direction along the rail.

15. The method of claim 13, wherein the direction parameter maps a perpendicular direction indicated by the second head-movement data to a viewing-window direction perpendicular to the least rail, and wherein the at least one threshold velocity comprises a perpendicular-direction-velocity threshold, the method further comprises:

determining that a velocity corresponding to the perpendicular direction in the second head-movement data exceeds the perpendicular-direction-velocity threshold; and

after the determination, causing the viewing window to move perpendicular to the rail.

16. The method of claim 13, wherein a speed parameter maps a given amount of head movement to an amount of viewing window movement, the method further comprises:

after determining the at least one viewing-window state is the rail-viewing-window state, increasing the speed parameter so as to increase the amount of viewing window movement that is mapped to the given amount of head movement.

17. The method of claim 13, wherein the rail defines a rail portion of the navigable area allowing viewing window movement, and wherein causing the viewing window to move corresponding to the rail comprises constraining the movement of the viewing window to be within the rail portion of the navigable area.

18. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising:

instructions for causing a head-mounted display (HMD) to provide (i) a viewing window that is fully visible in a field of view of the HMD, (ii) a navigable area that is not fully visible in a field of view of the HMD, wherein the viewing window displays a first portion of the navigable area, and (iii) at least one media object, wherein the at least one media object is associated with at least one viewing-window state;

instructions for receiving first body-movement data that is indicative of body movement from a first position to a second position; and

instructions for, based on at least (i) the first body-movement data, (ii) the at least one viewing-window state associated with the at least one media object, and (iii) a determination that at least one threshold velocity has been exceeded, causing the viewing window to display a second portion of the navigable area, wherein the second portion includes the at least one media object.

19. The non-transitory computer-readable medium of claim 18, wherein the at least one media object is associated with at least a gravity-viewing-window state, the non-transitory computer-readable medium further includes instructions for:

determining that the at least one viewing-window state is the gravity-viewing-window state associated with the media object; and

causing the viewing window to fixate upon the media object.

20. The non-transitory computer-readable medium of claim 19, wherein determining the at least one viewing-window state is a gravity-viewing-window state associated with the gravity object, wherein the at least one threshold velocity corresponds to the gravity-viewing-window state, the non-transitory computer-readable medium further includes instructions for:

while the viewing window is fixated upon the media object, receiving second head-movement data;

determining that a velocity corresponding to the second head-movement data is greater than the at least one threshold velocity; and

based on the determination, causing the viewing window to move away from the media object.

21. The non-transitory computer-readable medium of claim 18, wherein the at least one media object is associated with at least a rail-viewing-window state, the non-transitory computer-readable medium further includes instructions for:

after causing the viewing window to display the second portion of the navigable area, receiving input-selection data indicating a selection of the at least one media object;

based on the selection of the at least one media object, determining that the at least one viewing-window state is a rail-viewing-window state associated with the at least one media object;

receiving second head-movement data that is indicative of head movement from a third position of the HMD to a fourth position of the HMD;

determining a portion of the head movement indicated by the second head-movement data that corresponds to a rail defined by the rail-viewing-window state; and

after the determination, causing the viewing window to move corresponding to the rail.

22. The non-transitory computer-readable medium of claim 21, wherein a direction parameter maps a parallel direction indicated by the second head-movement data to a viewing-window direction along the rail, the non-transitory computer-readable medium further includes instructions for:

after determining the at least one viewing-window state is the rail-viewing-window state, using the parallel direction indicated by second head-movement to control the viewing window direction along the rail.

23. The non-transitory computer-readable medium of claim 22, wherein the direction parameter maps a perpendicular direction indicated by the second head-movement data to a viewing-window direction perpendicular to the rail, and wherein the at least one threshold velocity comprises a perpendicular-direction-velocity threshold, the non-transitory computer-readable medium further includes instructions for:

determining that a velocity corresponding to the perpendicular direction in the second head-movement data exceeds the perpendicular-direction-velocity threshold; and

after the determination, causing the viewing window to move perpendicular to the rail.

24. The non-transitory computer-readable medium of claim 21, wherein a speed parameter maps a given amount of head movement to an amount of viewing window movement, the non-transitory computer-readable medium further includes instructions for:

after determining the at least one viewing-window state is the rail-viewing-window state, increasing the speed parameter so as to increase the amount of viewing window movement that is mapped to the given amount of head movement.

25. The non-transitory computer-readable medium of claim 21, wherein the rail defines a rail portion of the navigable area allowing viewing window movement, and wherein causing the viewing window to move corresponding to the rail comprises constraining the movement of the viewing window to be within the rail portion of the navigable area.