Method and System for Compositing an Augmented Reality Scene

• US 20120069051A1
• Filed: 09/13/2009
• Published: 03/22/2012
• Est. Priority Date: 09/11/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hagbi, Netanel, Bergig, Oriel Y., Elsana, Jihad A.
• Granted Patent: US 9,824,495 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Disclosed are systems and methods for compositing an augmented reality scene, the methods including the steps of extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device; geometrically reconstructing at least one virtual model from at least one object; and compositing AR content from at least one virtual model in order to augment the AR content on the real-world image, thereby creating AR scene. Preferably, the method further includes; extracting at least one annotation from the real-world image into the memory of the data-processing machine for modifying at least one virtual model according to at least one annotation. Preferably, the method further includes: interacting with AR scene by modifying AR content based on modification of at least one object and/or at least one annotation in the real-world image.

Claims

1. A system for compositing an augmented reality (AR) scene, the system comprising:

(a) an extraction component, operationally connected to a sensing device for capturing a real-world image, configured for;

(i) extracting at least one object from said real-world image into a memory of a data-processing machine;

(ii) geometrically reconstructing at least one virtual model from said at least one object; and

(iii) compositing AR content from said at least one virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

2. The system of claim 1, wherein said extraction component is further configured for:

(iv) extracting at least one annotation from said real-world image into said memory of said data-processing machine for modifying said at least one virtual model according to at least one annotation.

3. The system of claim 1, wherein said extracting is performed independently of a predefined content repository.

4. The system of claim 1, wherein said real-world image includes at least one image type selected from the group consisting of:

a printed image, a hand-sketched image, a hand-drawn image, a sprayed image, an image displayed on a screen, an image projected on a surface, a weaved image, a carved-surface image, and an engraved-surface image.

5. The system of claim 1, wherein said real-world image includes at least two correlated annotations for modifying an effect of other annotations.

6. The system of claim 1, wherein said real-world image includes a legend having at least one item selected from the group consisting of an object referring to at least one image object, an annotation referring to at least one image annotation, an annotation referring to, said at least one object, and at least two correlated annotations referring to said at least one object.

7. The system of claim 1, wherein said at least one abject includes the entirety of said real-world image.

8. The system of claim 1, wherein said extraction component includes at least one functional component selected from the group consisting of:

an object-extractor component, a geometry-analysis component, a modeling-cues-analysis component, an animation-analysis component, a space-warp-analysis component, a physical-property-analysis component, a user-interaction-analysis component, an inter-object-connection-analysis component, a mesh-generation component, a texture-generation component, and an object-factory component.

9. The system of claim 1, wherein said extraction component is further configured for;

(iv) interacting with the AR scene by modifying said AR content based on modification of said at least one object and/or at least one annotation in said real-world image.

11. The method of claim 1, the method further comprising the step of:

(d) extracting at least one annotation from said real-world image into said memory of said data-processing machine for modifying said at least one virtual model according to at least one annotation.

10. A method for compositing an augmented reality (AR) scene, the method comprising the steps of:

(a) extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device;

(b) geometrically reconstructing at least one virtual model from said at Least one object; and

(c) compositing AR content from said at least one virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

12. The method of claim 10, wherein said step of extracting is performed independently of a predefined content repository.

13. The method of claim 10, wherein said real-world image includes a legend having at least one item selected from the group consisting of an object referring to at least one image object, an annotation referring to at least one image annotation, an annotation referring to said at least one object, and at least two correlated annotations referring to said at least one object.

14. The method of claim 10, the method further comprising the step of:

(d) interacting with the AR scene by modifying said AR content based on modification of said at least one object and/or at least one annotation in said real-world image.

15. A computer-readable storage medium having computer-readable code embodied on the computer-readable storage medium, the computer-readable code comprising:

(a) program code for extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device;

(b) program code for geometrically reconstructing at least one virtual model from said at least one object; and

(c) program code for compositing AR content from said at least one virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.