SURFACE AWARE LENS

• US 20190188920A1
• Filed: 02/22/2019
• Published: 06/20/2019
• Est. Priority Date: 01/09/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: McPhee, Andrew James, Charlton, Ebony James, Hare, Samuel Edward, Evans, Michael John, Dargis, Jokubas, Sanchez-Saez, Ricardo
• Granted Patent: US 10,740,978 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real-world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real-world through utilization of a redundant tracking system comprising multiple tracking sub-systems. Virtual object rendering can be with respect to a reference surface in a real-world three-dimensional space depicted in a camera view of a mobile computing device.

Claims

1. A system, comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising a virtual object rendering system, the virtual object rendering system causing the system to perform operations comprising;

rendering a virtual object to appear at a position in a three-dimensional space captured within a camera view, the position in the three-dimensional space being based on a reference surface detected in the three-dimensional space;

tracking, by a first tracking subsystem from among a set of tracking subsystems, the virtual object at the position in the three-dimensional space using tracking indicia;

detecting, by the first tracking subsystem, an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems. The system of claim 1, wherein the operations further comprise;

detecting a reference surface in the three-dimensional space captured within the camera view based on user input.

3. The system of claim 1, wherein the position in the three-dimensional space is a predefined distance from the reference surface.

4. The system of claim 3, wherein the rendering of the virtual object includes rendering and maintaining the virtual object at the predefined distance from the reference surface.

5. The system of claim 1, wherein the system further includes a redundant tracking system comprising the first and second tracking subsystems.

6. The system of claim 1, wherein the set of tracking subsystem comprises one or more of:

a gyroscopic tracking system, a natural feature tracking system, and simultaneous localization and mapping tracking.

7. The system of claim 1, wherein the rendering of the virtual object includes rendering a separate virtual representation related to the virtual object.

8. The system of claim 7, wherein the separate virtual representation comprises a simulated shadow of the virtual object.

9. The system of claim 1, wherein the operations further comprise:

receiving user selection of a template from among a set of templates, the template corresponding to the virtual object.

10. The system of claim 1, wherein the virtual object comprises customized personal avatar content.

11. The system of claim 1, wherein the virtual object comprises content accessed based on external factors related to the three-dimensional space. The system of claim 1, wherein the operations further comprise:

maintaining customized user content;

receiving a selection of the customized user content; and

wherein the rendering of the virtual object includes rendering the selection of the customized user content.

13. The system of claim 12, wherein the operations further comprise:

sharing the selection of the customized user content with a user other than the user that created the customized user content.

14. A method comprising:

rendering, by one or more processors of a computing machine, a virtual object to appear at a position in a three-dimensional space captured within a camera view, the position in the three-dimensional space being based on a reference surface detected in the three-dimensional space;

tracking, by a first tracking subsystem from among a set of tracking subsystems, the virtual object at the position in the three-dimensional space using tracking indicia;

detecting, by the first tracking subsystem, an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems.

15. The method of claim 14, further comprising detecting a real-world fixed surface within the three-dimensional space, the real-world fixed surface corresponding to the reference surface.

16. The method of claim 15, wherein the rendering of the, virtual object comprises:

rendering and maintaining the virtual object at predefined distance from the real-world fixed surface.

17. The method of claim 14, wherein the rendering of the virtual object includes rendering a separate virtual representation related to the virtual object.

18. The method of claim 14, further comprising:

receiving a selection of customized user content or customized personal avatar content; and

wherein the rendering of the virtual object comprises rendering the customized user content or customized personal avatar content to appear at the position in the three-dimensional space.

19. The method of claim 14, further comprising:

receiving user selection of the virtual object from among a set of templates.

20. A non-transitory machine-readable storage medium including a virtual object rendering system that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

rendering a virtual object to appear at a position in a three-dimensional space captured within a camera view, the position in the three-dimensional space being based on a reference surface detected in the three-dimensional space;

tracking, by a first tracking subsystem from among a set of tracking subsystems, the virtual object at the position in the three-dimensional space using tracking indicia;

detecting, by the first tracking subsystem, an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems.