Methods to pan, zoom, crop, and proportionally move on a head mountable display

• US 9,811,154 B2
• Filed: 12/14/2015
• Issued: 11/07/2017
• Est. Priority Date: 03/27/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raffle, Hayes Solos, Patel, Nirmal, Braun, Max Benjamin
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US14/968,316
• Publication Number:
  US 20160103483A1
• Time in Patent Office: 694 Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

Methods, apparatus, and computer-readable media are described herein related to displaying and cropping viewable objects. A viewable object can be displayed on a display of a head-mountable device (HMD) configured with a hand-movement input device. The HMD can receive both head-movement data corresponding to head movements and hand-movement data from the hand-movement input device. The viewable object can be panned on the display based on the head-movement data. The viewable object can be zoomed on the display based on the hand-movement data. The HMD can receive an indication that navigation of the viewable object is complete. The HMD can determine whether a cropping mode is activated. After determining that the cropping mode is activated, the HMD can generate a cropped image of the viewable object on the display when navigation is complete.

Claims

1. A method, comprising:

displaying at least one viewable object on a display provided by a head-mountable device (HMD) configured with a hand-movement input device, wherein the HMD is configured to receive head-movement data corresponding to head movements and to receive hand-movement data corresponding to inputs from the hand-movement input device;

receiving head-movement data at the HMD;

panning the at least one viewable object on the display based on the head-movement data;

receiving hand-movement data at the HMD;

zooming the at least one viewable object on the display based on the hand-movement data;

receiving, at the HMD, an indication that navigation of the at least one viewable object is complete, wherein the navigation of the at least one viewable object includes panning the at least one viewable object on the display based on the head-movement data, and zooming the at least one viewable object on the display based on the hand-movement data;

determining whether a cropping mode is activated using the HMD; and

responsive to determining that the cropping mode is activated, the HMD generating a cropped image of the at least one viewable object on the display resulting from the navigation of the at least one viewable object being complete.

2. The method of claim 1, wherein the at least one viewable object comprises an image, and wherein the display when navigation of the at least one viewable object is complete comprises a portion of the image, and wherein the portion of the image is related to a cropping window for the image.

3. The method of claim 1, wherein the at least one viewable object comprises a map, and wherein zooming the at least one viewable object on the display comprises zooming the map on the display to find a feature indicated on the map.

4. The method of claim 1, wherein the display provided by the HMD is shown in a display plane, and wherein the hand-movement input device is disposed on the HMD and comprises a touchpad configured to be substantially perpendicular to the display plane.

5. The method of claim 1, further comprising performing an operation on the cropped image, wherein the operation on the cropped image includes at least one of sharing the cropped image, storing the cropped image, or uploading the cropped image.

6. The method of claim 1, wherein the at least one viewable object comprises a timeline display of one or more images.

7. The method of claim 1, wherein zooming the at least one viewable object on the display comprises:

determining a movement regarding a Z-axis within a Z-axis oriented display (ZAOD) based on the hand-movement data;

generating a moved ZAOD based on the movement regarding the Z-axis; and

displaying the moved ZAOD using the HMD.

8. The method of claim 1, wherein panning the at least one viewable object on the display comprises determining a movement based on the head-movement data within a Z-axis oriented display (ZAOD) regarding an X-axis, determining a movement based on the head-movement data within the ZAOD regarding a Y-axis, or determining a movement based on the head-movement data within the ZAOD regarding both the X-axis and the Y-axis.

9. The method of claim 8, wherein the movement based on the head-movement data within the ZAOD regarding the X-axis corresponds to a movement of NP_x pixels within the ZAOD of X°

along the X-axis, and wherein NP_x=k_x X°

Z, with k_x being a constant of proportionality for the X-axis, and Z being a Z-axis coordinate value.

10. The method of claim 8, wherein the movement based on the head-movement data within the ZAOD regarding the Y-axis corresponds to a movement of NP_y pixels within the ZAOD of Y°

along the Y-axis, and wherein NP_y=k_y Y°

Z, with k_y being a constant of proportionality for the Y-axis, and Z being a Z-axis coordinate value.

11. The method of claim 1, wherein the at least one viewable object comprises a structured document, wherein the structured document can include a definition of a plurality of columns, and wherein panning the at least one viewable object on the display comprises panning along one or more columns of the plurality of columns.

12. An apparatus, including a non-transitory computer-readable storage medium having stored thereon program instructions that, upon execution by a computing device, cause the apparatus to perform functions comprising:

displaying at least one viewable object on a display, wherein the apparatus is configured to receive head-movement data corresponding to head movements and to receive hand-movement data corresponding to inputs from a hand-movement input device;

panning the at least one viewable object on the display based on received head-movement data;

zooming the at least one viewable object on the display based on received hand-movement data;

receiving an indication that navigation of the at least one viewable object is complete, wherein the navigation of the at least one viewable object includes panning the at least one viewable object on the display based on the head-movement data, and zooming the at least one viewable object on the display based on the hand-movement data;

determining whether a cropping mode is activated; and

responsive to determining that the cropping mode is activated, generating a cropped image of the at least one viewable object on the display resulting from the navigation of the at least one viewable object being complete.

13. The apparatus of claim 12, wherein the at least one viewable object comprises an image, wherein the display when navigation of the at least one viewable object is complete comprises a portion of the image, and wherein the portion of the image is related to a cropping window for the image.

14. The apparatus of claim 12, wherein the functions further comprise performing an operation on the cropped image, wherein the operation on the cropped image includes at least one sharing the cropped image, storing the cropped image, or uploading the cropped image.

15. The apparatus of claim 12, wherein zooming the at least one viewable object on the display comprises:

determining a movement regarding a Z-axis within a Z-axis oriented display (ZAOD) based on the hand-movement data;

generating a moved ZAOD based on the movement within the ZAOD; and

displaying the moved ZAOD.

16. The apparatus of claim 15, wherein panning the at least one viewable object on the display comprises determining a movement based on the head-movement data within a Z-axis oriented display (ZAOD) regarding an X-axis, determining a movement based on the head-movement data within the ZAOD regarding a Y-axis, or determining a movement based on the head-movement data within the ZAOD regarding both the X-axis and Y-axis.

17. The apparatus of claim 16, wherein the movement based on the head-movement data within the ZAOD regarding the X-axis corresponds to a movement of NP_x pixels within the ZAOD of X°

along the X-axis, and wherein NP_x=k_x X°

Z, with k_x being a constant of proportionality for the X-axis, and Z being a Z-axis coordinate value.

18. The apparatus of claim 16, wherein the movement based on the head-movement data within the ZAOD regarding the Y-axis corresponds to a movement of NP_y pixels within the ZAOD of Y°

along the Y-axis, and wherein NP_x=k_y Y°

Z, with k_y being a constant of proportionality for the Y-axis, and Z being a Z-axis coordinate value.

19. A computing device, comprising:

a hand-movement input device;

a display;

a processor; and

a non-transitory computer-readable storage medium having stored thereon program instructions that, upon execution by the processor, cause the computing device to perform operations comprising;

displaying at least one viewable object on the display, wherein the computing device is configured to receive head-movement data corresponding to head movements and to receive hand-movement data corresponding to inputs from the hand-movement input device;

receiving head-movement data;

panning the at least one viewable object on the display based on the head-movement data;

receiving hand-movement data;

zooming the at least one viewable object on the display based on the hand-movement data;

receiving an indication that navigation of the at least one viewable object is complete, wherein the navigation of the at least one viewable object includes panning the at least one viewable object on the display based on the head-movement data, and zooming the at least one viewable object on the display based on the hand-movement data;

determining whether a cropping mode is activated; and

responsive to determining that the cropping mode is activated, generating a cropped image of the at least one viewable object on the display resulting from the navigation of the at least one viewable object being complete.

20. The computing device of claim 19, further comprising:

one or more sensors configured to generate the head-movement data; and

a frame configured to be worn as a head-mountable device, wherein at least the hand-movement input device, the display, and the one or more sensors are coupled to the frame,

wherein the display defines a display plane,

and wherein the hand-movement input device is a touchpad that is configured to be substantially perpendicular to the display plane.