AUGMENTED REALITY CONTENT ADAPTED TO CHANGES IN REAL WORLD SPACE GEOMETRY

• US 20150187108A1
• Filed: 12/31/2013
• Published: 07/02/2015
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: DAQRI, LLC
• Inventors: Mullins, Brian
• Granted Patent: US 9,898,844 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/61 Scene description

Abstract

A system and method for augmented reality content adapted to changes in real world space geometry are described. A device captures an image of a local environment and maps a real world space geometry of the local environment using the image of the local environment. The device generates a visualization of a virtual object in the display relative to the mapped real world space geometry of the local environment. A content of the virtual object is adjusted to changes in the real world space geometry of the local environment.

Claims

1. A device comprising:

an optical sensor configured to capture an image of a local environment;

a display; and

a processor comprising an augmented reality application, the augmented reality application having;

a recognition module configured to map a real world space geometry of the local environment using the image of the local environment; and

a visualization module configured to generate a visualization of a virtual object in the display relative to the mapped real world space geometry of the local environment, to adjust a content of the virtual object to changes in the real world space geometry of the local environment.

2. The device of claim 1, wherein the local environment comprises a scenery captured with the optical sensor, the scenery comprising a plurality of real world physical objects.

3. The device of claim 2, wherein the recognition module comprises:

a space geometry module configured to identify feature points of the plurality of real world physical objects, to identify the plurality of real world physical objects using the feature points of the plurality of real world physical objects, to map the real world space geometry using the feature points of the plurality of real world physical objects; and

a local image module configured to identify the virtual object and a corresponding behavior of the virtual object relative to the identified plurality of real world physical objects, based on the identified plurality of real world physical objects.

4. The device of claim 3, wherein the visualization module comprises:

a space geometry change detector configured to identify a physical change in the real world space geometry based on the mapped real world space geometry; and

a virtual object content modifier configured to modify the content of the virtual object in response to the physical change in the real world space geometry of the local environment.

5. The device of claim 4, wherein the physical change comprises at least one of a change in a location of a real world physical object within the scenery, a removal of a real world physical object from the scenery, and an addition of a new real word physical object to the scenery.

6. The device of claim 5, wherein the virtual object content modifier is configured to modify a display and a behavior of the virtual object in response to the physical change.

7. The device of claim 6, wherein a size, shape, and location of the virtual object in the display is modified in relation to the change in the location of the real world physical object within the scenery.

8. The device of claim 6, wherein the behavior of the virtual object is modified in relation to the change in the location of the real world physical object within the scenery.

9. The device of claim 4, wherein the virtual object content modifier is configured to dynamically generate a new behavior for the virtual object in response to the physical change in the real world space geometry of the local environment using a library of virtual object alternative behaviors.

10. The device of claim 9, wherein the virtual object content modifier is configured to:

generate a physical change identifier in response to the physical change in the real world space geometry of the local environment;

communicate the physical change identifier to a server;

receive from the server a new behavior model for the virtual object corresponding to the physical change identifier; and

modify a behavior of the virtual object based on the new behavior model.

11. A method comprising:

capturing an image of a local environment;

mapping a real world space geometry of the local environment using the image of the local environment;

generating a visualization of a virtual object in a display relative to the mapped real world space geometry of the local environment; and

adjusting a content of the virtual object to changes in the real world space geometry of the local environment.

12. The method of claim 11, wherein the local environment comprises a scenery captured with an optical sensor, the scenery comprising a plurality of real world physical objects.

13. The method of claim 12, further comprising:

identifying feature points of the plurality of real world physical objects;

identifying the plurality of real world physical objects using the feature points of the plurality of real world physical objects;

mapping the real world space geometry using the feature points of the plurality of real world physical objects; and

identifying the virtual object and a corresponding behavior of the virtual object relative to the identified plurality of real world physical objects, based on the identified plurality of real world physical objects.

14. The method of claim 13, further comprising:

identifying a physical change in the real world space geometry based on the mapped real world space geometry; and

modifying the content of the virtual object in response to the physical change in the real world space geometry of the local environment.

15. The method of claim 14, wherein the physical change comprises at least one of a change in a location of a real world physical object within the scenery, a removal of a real world physical object from the scenery, and an addition of a new real word physical object to the scenery.

16. The method of claim 15, wherein modifying the content of the virtual object further comprises:

modifying a display and a behavior of the virtual object in response to the physical change.

17. The method of claim 16, further comprising:

modifying a size, shape, and location of the virtual object in the display in relation to the change in the location of the real world physical object within the scenery; and

modifying the behavior of the virtual object in relation to the change in the location of the real world physical object within the scenery.

18. The method of claim 14, further comprising:

dynamically generating a new behavior for the virtual object in response to the physical change in the real world space geometry of the local environment using a library of virtual object alternative behaviors.

19. The method of claim 18, further comprising:

generating a physical change identifier in response to the physical change in the real world space geometry of the local environment;

communicating the physical change identifier to a server;

receiving from the server a new behavior model for the virtual object corresponding to the physical change identifier; and

modifying a behavior of the virtual object based on the new behavior model.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

capturing an image of a local environment;

mapping a real world space geometry of the local environment using the image of the local environment;

generating a visualization of a virtual object in a display relative to the mapped real world space geometry of the local environment; and

adjusting a content of the virtual object to changes in the real world space geometry of the local environment.