SURFACE AWARE LENS

• US 20180197343A1
• Filed: 01/05/2018
• Published: 07/12/2018
• Est. Priority Date: 01/09/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Hare, Samuel Edward, McPhee, Andrew James
• Granted Patent: US 10,242,503 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real world, and the three-dimensional virtual object can exist similarly for multiple users. Virtual object rendering can be with respect to a reference surface in a real world environment, which reference surface can be selected by a user as part of the virtual object rendering process.

Claims

1. A system, comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising a virtual object rendering system, the virtual object rendering system causing the system to perform operations comprising;

receiving an input to activate a surface aware lens that facilitates rendering of a virtual object in a three-dimensional space captured within a camera view;

detecting a reference surface in the three-dimensional space captured within the camera view;

orienting the virtual object based on the reference surface; and

rendering the virtual object to appear in the three-dimensional space while captured within the camera view, the virtual object being rendered with respect to the reference surface.

2. The system of claim 1, wherein the detecting of the reference surface is based on user input.

3. The system of claim 1, wherein the orienting of the virtual object based on the reference surface comprises:

assigning the virtual object to a position in three-dimensional space based on the detected reference surface; and

identifying, based on the assigned position, tracking indicia operable to track the virtual object in the three-dimensional space.

4. The system of claim 3, wherein the position in the three-dimensional space is a predefined distance from the reference surface.

5. The system of claim 4, wherein the rendering of the virtual object with respect to the reference surface includes rendering and maintaining the virtual object at the predefined distance from the reference surface.

6. The system of claim 3, wherein the operations further comprise:

tracking, by a first tracking subsystem from among a set of tracking subsystems, the virtual object at the position in the three-dimensional space using the tracking indicia;

detecting, by the first tracking subsystem, an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems.

7. The system of claim 6, wherein the system further includes a redundant tracking system comprising the set of tracking subsystems, the set of tracking subsystems comprising a gyroscopic tracking system, a natural feature tracking system, and simultaneous localization and mapping tracking.

8. The system of claim 1, wherein the rendering of the virtual object includes rendering a separate virtual representation related to the virtual object at the reference surface.

9. The system of claim 8, wherein the separate virtual representation is a simulated shadow of the virtual object.

10. The system of claim 1, further comprising:

receiving user selection of a template from among a set of templates, the template corresponding to the virtual object.

11. The system of claim 10, wherein the virtual object includes customized personal avatar content.

12. The system of claim 1, wherein the virtual object includes content based on external factors related to the three-dimensional space.

13. The system of claim 12, wherein the content based on external factors is dynamically changed over time.

14. The system of claim 1, wherein the operations further comprise:

maintaining customized user content;

receiving a selection of the customized user content; and

wherein the rendering of the virtual object includes rendering the selection of the customized user content with respect to the reference surface.

15. The system of claim 14, wherein the operations further comprise:

sharing the selection of the customized user content with a user other than the user that created the customized user content.

17. The method of claim 14, wherein the detecting of the reference surface includes detecting a real-world fixed surface within the three-dimensional space.

18. The method of claim 15, wherein:

the orienting of the virtual object based on the reference surface comprises;

assigning the virtual object to a position in three-dimensional space based on the detected reference surface; and

identifying, based on the assigned position, tracking indicia operable to track the virtual object in the three-dimensional space;

the rendering of the virtual object with respect to the reference surface comprises;

rendering and maintaining the virtual object at the assigned position.

16. A method for rendering a virtual object in a three-dimensional space captured within a camera view by a virtual object rendering system, the method comprising:

receiving an input to activate a surface aware lens that facilitates the rendering of the virtual object in the three-dimensional space captured within a camera view;

detecting a reference surface in the three-dimensional space captured within the camera view;

orienting the virtual object based on the reference surface; and

rendering the virtual object to appear in the three-dimensional space while captured within the camera view, the virtual object being rendered with respect to the reference surface.

19. The method of claim 16, further comprising:

maintaining customized user content;

receiving a selection of the customized user content for a new reference surface; and

rendering the selection of the customized user content at the new reference surface.

20. A non-transitory machine-readable storage medium including a virtual object rendering system that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving an input to activate a surface aware lens that facilitates the rendering of a virtual object in three-dimensional space captured within a camera view;

detecting a reference surface in the three-dimensional space captured within the camera view;

orienting the virtual object based on the reference surface; and

rendering the virtual object to appear in the three-dimensional space while captured within the camera view, the virtual object being generated with respect to the reference surface.