Visual Display of Interactive, Gesture-Controlled, Three-Dimensional (3D) Models for Head-Mountable Displays (HMDs)

• US 20150169070A1
• Filed: 12/17/2013
• Published: 06/18/2015
• Est. Priority Date: 12/17/2013
• Status: Abandoned Application

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jain, Sumit, Kuffner, James Joseph, Harp, Andrew Logan
• Application Number:
  US14/108,365
• Publication Number:
  US 20150169070A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

Methods and systems are provided for controlling a three-dimensional (3D) model for a head-mountable display (HMD). The HMD can receive a 3D model for the object, where the 3D model includes three-dimensional shape and texture information about the object, the three-dimensional shape and texture information about the object specified with respect to at least a first axis, a second axis, and a third axis, where each of the first axis, the second axis, and the third axis differs. The HMD can display a view of the 3D model. The HMD can receive an input gesture. The HMD can determine whether the input gesture includes a 3D model gesture. After determining that the input gesture does includes a 3D model gesture, the HMD can update the view of the 3D model based on the input gesture and can display the updated view of the 3D model.

Claims

1. A method, comprising:

receiving a 3D model for an object at a head-mountable display (HMD), wherein the 3D model comprises three-dimensional shape and texture information about the object, the three-dimensional shape and texture information about the object specified with respect to at least a first axis, a second axis, and a third axis, wherein each of the first axis, the second axis, and the third axis differs;

displaying a view of the 3D model using the HMD;

receiving an input gesture at the HMD;

determining whether the input gesture comprises a 3D-model gesture of a plurality of 3D-model gestures, wherein the plurality of 3D-model gestures comprise a first 3D-model gesture to modify display of the 3D model with respect to the first axis, a second 3D-model gesture to modify display of the 3D model with respect to the second axis, and a third 3D-model gesture to modify display of the 3D model with respect to the third axis; and

after determining that the input gesture is related to the 3D model, the HMD;

updating the view of the 3D model based on the input gesture, and

displaying the updated view of the 3D model.

2. The method of claim 1, wherein the input gesture comprises a gesture to rotate the 3D model.

3. The method of claim 2, wherein updating the view of the 3D model comprises:

rotating the 3D model as indicated by the gesture to rotate the 3D model; and

generating a display of the rotated 3D model.

4. The method of claim 2, wherein the gesture to rotate the 3D model comprises a first gesture to rotate the 3D model on a first axis and a second gesture to rotate the 3D model on a second axis, and wherein the first axis differs from the second axis.

5. The method of claim 1, wherein the information about the object comprises an image related to the object.

6. The method of claim 5, wherein the image related to the object is at least one image selected from the group consisting of an image that depicts at least part of the object, an image of a Quick Response (QR) code related to the object, and an image of a bar code related to the object.

7. The method of claim 1, further comprising:

receiving a second input gesture at the HMD;

determining whether the second input gesture corresponds to a close gesture; and

in response to determining that the second input gesture corresponds to the close gesture, terminating the view of the 3D model.

8. A head-mountable display (HMD), comprising:

a display;

a processor; and

a non-transitory computer readable medium, configured to store at least instructions, wherein the instructions are executable by the processor to cause the HMD to perform functions comprising;

receiving a 3D model for an object, wherein the 3D model comprises three-dimensional shape and texture information about the object, the three-dimensional shape and texture information about the object specified with respect to at least a first axis, a second axis, and a third axis, wherein each of the first axis, the second axis, and the third axis differs;

displaying a view of the 3D model using the display;

receiving an input gesture;

determining whether the input gesture comprises a 3D-model gesture of a plurality of 3D-model gestures, wherein the plurality of 3D-model gestures comprise a first 3D-model gesture to modify display of the 3D model with respect to the first axis, a second 3D-model gesture to modify display of the 3D model with respect to the second axis, and a third 3D-model gesture to modify display of the 3D model with respect to the third axis; and

after determining that the input gesture is related to the 3D model;

updating the view of the 3D model based on the input gesture, and

displaying the updated view of the 3D model using the display.

9. The HMD of claim 8, wherein the input gesture comprises a gesture to rotate the 3D model.

10. The HMD of claim 9, wherein updating the view of the 3D model comprises:

rotating the 3D model as indicated by the gesture to rotate the 3D model; and

generating a display of the rotated 3D model.

11. The HMD of claim 9, wherein the gesture to rotate the 3D model comprises a first gesture to rotate the 3D model on a first axis and a second gesture to rotate the 3D model on a second axis, and wherein the first axis differs from the second axis.

12. The HMD of claim 8, further comprising a camera, and wherein receiving the input gesture comprises:

capturing a plurality of images using the camera; and

determining the input gesture from the plurality of images.

13. The HMD of claim 8, wherein the information about the object comprises an image related to the object.

14. The HMD of claim 8, further comprising:

receiving a second input gesture;

determining whether the second input gesture corresponds to a close gesture; and

in response to determining that the second input gesture corresponds to the close gesture, terminating the view of the 3D model.

15. A non-transitory computer readable medium having stored therein instructions executable by a processor of a head-mountable display (HMD) to cause the HMD to perform functions, the functions comprising:

receiving a 3D model for an object, wherein the 3D model comprises three-dimensional shape and texture information about the object, the three-dimensional shape and texture information about the object specified with respect to at least a first axis, a second axis, and a third axis, and wherein each of the first axis, the second axis, and the third axis differs;

displaying a view of the 3D model;

receiving an input gesture;

determining whether the input gesture comprises a 3D-model gesture of a plurality of 3D-model gestures, wherein the plurality of 3D-model gestures comprise a first 3D-model gesture to modify display of the 3D model with respect to the first axis, a second 3D-model gesture to modify display of the 3D model with respect to the second axis, and a third 3D-model gesture to modify display of the 3D model with respect to the third axis; and

after determining that the input gesture is related to the 3D model;

updating the view of the 3D model based on the input gesture, and

displaying the updated view of the 3D model.

16. The non-transitory computer readable medium of claim 15, wherein the input gesture comprises a gesture to rotate the 3D model.

17. The non-transitory computer readable medium of claim 16, wherein updating the view of the 3D model comprises:

rotating the 3D model as indicated by the gesture to rotate the 3D model; and

generating a display of the rotated 3D model.

18. The non-transitory computer readable medium of claim 16, wherein the gesture to rotate the 3D model comprises a first gesture to rotate the 3D model on a first axis and a second gesture to rotate the 3D model on a second axis, and wherein the first axis differs from the second axis.

19. The non-transitory computer readable medium of claim 15, wherein the information about the object comprises an image related to the object.

20. The non-transitory computer readable medium of claim 15, further comprising:

receiving a second input gesture;

determining whether the second input gesture corresponds to a close gesture; and

in response to determining that the second input gesture corresponds to the close gesture, terminating the view of the 3D model.