Methods and devices for rendering interactions between virtual and physical objects on a substantially transparent display

• US 10,379,346 B2
• Filed: 09/25/2017
• Issued: 08/13/2019
• Est. Priority Date: 10/05/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wong, Adrian, Miao, Xiaoyu
• Primary Examiner(s): Hong, Stephen S
• Assistant Examiner(s): Robinson, Marshon L
• Application Number:
  US15/713,995
• Publication Number:
  US 20180024363A1
• Time in Patent Office: 687 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Disclosed are methods and devices for rendering interactions between virtual and physical objects on a substantially transparent display are disclosed. In one embodiment, the method includes displaying a user-interface on a substantially transparent display of a wearable computing device. The method further includes displaying a virtual object in the view region at a focal length along a first line of sight and detecting a physical object at a physical distance along a second line of sight. The method still further includes determining that a relationship between the focal length and the physical distance is such that the virtual object and the physical object appear substantially co-located in a user-view through the view region and, responsive to the determination, initiating a collision action between the virtual object and the physical object.

Claims

1. A computer-implemented method comprising:

displaying a user-interface on a display of a mobile computing device, wherein the user-interface comprises a view region via which virtual objects and real-world objects are viewable;

displaying a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determining a physical distance, along the first line of sight, between the display and a physical object;

determining that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located, in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiating a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the mobile computing device causing the virtual object to move away from the physical object.

2. The method of claim 1, wherein the display comprises a see-through display.

3. The method of claim 2, wherein the see-through display is a virtual see-through display.

4. The method of claim 1, wherein determining the physical distance to the physical object comprises:

capturing a first image of the physical object; and

based on the first image, determining the physical distance to the physical object.

5. The method of claim 4, further comprising:

capturing a second image of the physical object, wherein the first image is captured at a first time and the second image is captured at a second time; and

based on the first image, the second image, the first time, and the second time, determining a velocity of the physical object.

6. The method of claim 1, wherein determining the physical distance to the physical object comprises:

emitting a series of pulsed signals;

detecting reflections of at least some of the pulsed signals; and

based on the reflections, detecting the physical object.

7. The method of claim 1, wherein determining the physical distance to the physical object comprises:

emitting optical beams forming a three-dimensional lattice;

detecting disturbances in the lattice; and

based on the disturbances, detecting the physical object.

8. The method of claim 1, further comprising:

using at least one sensor to detect a velocity of the physical object.

9. The method of claim 1, wherein the graphical modification of the virtual object is based on a velocity of the physical object.

10. The method of claim 1, wherein the graphical modification of the virtual object further comprises the virtual object changing shape.

11. The method of claim 1, wherein the graphical modification of the virtual object further comprises moving the virtual object out of the view region.

12. The method of claim 1, wherein the user-interface is initially configured such that a content region is substantially arranged along an at least partial ring above the view region.

21. The method of claim 1, wherein the focal length is a perceived distance from the viewing location to the virtual object.

13. A computing device comprising:

at least one processor; and

data storage comprising logic executable by the at least one processor to;

display a user-interface on a display, wherein the user-interface comprises a view region;

display a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determine a physical distance, along a first line of sight, between the display and a physical object;

determine that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiate a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the computing device causing the virtual object to move away from the physical object.

14. The computing device of claim 13, wherein the display comprises a see-through graphic display.

15. The computing device of claim 13, wherein the see-through graphic display is a virtual see-through display.

16. The computing device of claim 13, further comprising:

an emitter configured to emit a series of pulsed signals; and

a detector configured to detect reflections of at least some of the pulsed signals, wherein the logic is further executable by the at least one processor to detect the physical object based on the reflections.

17. The computing device of claim 13, further comprising:

at least one laser configured to emit a three-dimensional lattice; and

at least one detector configured to detect disturbances in the lattice, wherein the logic is further executable by the at least one processor to detect the physical object based on the disturbances.

18. The computing device of claim 13, wherein the graphical modification of the virtual object further comprises the virtual object changing shape.

19. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

displaying a user-interface on a display of a computing device, wherein the user-interface comprises a view region;

displaying a virtual object in the view region, wherein the virtual object is displayed at a focal length from the display along a first line of sight;

determining a physical distance, along the first line of sight, between the display and a physical object;

determining that the focal length is substantially equal to the physical distance to the physical object, such that the virtual object and the physical object appear substantially co-located in the view region; and

responsive to the determination that the focal length is substantially equal to the physical distance, initiating a collision action between the virtual object and the physical object, wherein the collision action comprises a graphical modification of the virtual object, and wherein the graphical modification of the virtual object comprises the computing device causing the virtual object to move away from the physical object.

20. The non-transitory computer-readable medium of claim 19, wherein the user-interface further comprises a content region that is located outside of the view region, and wherein the user-interface is initially configured such that the at least one content region is arranged along an at least partial ring above the view region.