Method and System for Providing and Reconstructing a Photorealistic Three-Dimensional Environment

• US 20090109240A1
• Filed: 10/24/2007
• Published: 04/30/2009
• Est. Priority Date: 10/24/2007
• Status: Abandoned Application

• Current Assignee: Deutsche Telekom AG
• Original Assignee: Deutsche Telekom AG
• Inventors: Elovici, Yuval, Englert, Roman, Henke, Katja, Kurze, Martin
• Application Number:
  US11/877,736
• Publication Number:
  US 20090109240A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

The present invention relates to a method and system for providing and reconstructing a photorealistic environment, by integrating a virtual item into it, comprising: (a) a dedicated marker, placed in a predefined location within an environment, in which a virtual item has to be integrated, for enabling determining the desired location of said virtual item within said environment; (b) a conventional camera for taking a picture or shooting a video clip of said environment, in which said marker was placed, and then providing a corresponding images of said environment; and (c) one or more servers for receiving said corresponding image of said environment from said camera, processing it, and outputting a photorealistic image that contains said virtual item integrated within it, comprising: (c.1.) a composer for composing a photorealistic image from said corresponding image of said environment; (c.2.) an image processing unit for processing said corresponding image and for determining the location of said marker within said environment; (c.3.) a configuration database for storing configurations and other data; and (c.4.) an image rendering unit for reconstructing the photorealistic image by integrating said virtual item into said predefined location of the photographed environment, wherein said marker is located.

Claims

1. A system for providing and reconstructing a photorealistic environment, by integrating a virtual item into it, comprising:

a) a dedicated marker, placed in a predefined location within an environment, in which a virtual item has to be integrated, for enabling determining the desired location of said virtual item within said environment;

b) a conventional camera for taking a picture or shooting a video clip of said environment, in which said marker was placed, and then providing a corresponding images of said environment; and

c) one or more servers for receiving said corresponding image of said environment from said camera, processing it, and outputting a photorealistic image that contains said virtual item integrated within it, comprising;

c.1. a composer for composing a photorealistic image from said corresponding image of said environment;

c.2. an image processing unit for processing said corresponding image and for determining the location of said marker within said environment;

c.3. a configuration database for storing configurations and other data; and

c.4. an image rendering unit for reconstructing the photorealistic image by integrating said virtual item into said predefined location of the photographed environment, wherein said marker is located.

2. System according to claim 1, wherein the marker enables the image processing unit to determine a spatial location of the virtual item to be integrated into the environment.

3. System according to claim 1, wherein providing the marker enables determining lighting and corresponding shadow conditions of the photographed environment.

4. System according to claim 1, wherein the image rendering unit further simulates lighting and corresponding shadow conditions of the photographed environment.

5. System according to claim 1, wherein the marker is composed of a black-and-white board.

6. System according to claim 1, wherein the marker is composed of a board, having a predefined texture for enabling to determine a spatial orientation of said marker within the photographed environment.

7. System according to claim 1, wherein the marker comprises a mirror reflecting sphere for determining the lighting and corresponding shadow conditions of the environment, in which it is located.

8. System according to claim 7, wherein the mirror reflecting sphere of the marker is connected to said marker by means of a rod.

9. System according to claim 7, wherein the image processing unit by means of the marker mirror reflecting sphere further determines which color and/or intensity of the light, within the photographed environment, comes from which direction.

10. System according to claim 1, wherein the image processing unit is further used for estimating camera parameters.

11. System according to claim 10, wherein the camera parameters are selected from one or more of the following:

a. the focal distance of the lens of said camera;

b. the viewing direction and orientation of said camera; and

c. the position of said camera in a space.

12. System according to claim 1, further comprising providing a model and material database for storing predefined models and materials to be integrated into the taken image of the environment, or storing links to said models and materials, if they are stored on another server.

13. System according to claim 1, wherein the marker is displayed on a mobile device screen or provided in a printed form.

14. System according to claim 1, wherein a user can select and configure the virtual item to be integrated into the photographed environment.

15. A method for providing and reconstructing a photorealistic environment by integrating a virtual item into it, comprising:

a. placing a dedicated marker in a predefined location within an environment, in which a virtual item has to be integrated, for enabling determining the desired location of said virtual item within said environment;

b. taking a picture or shooting a video clip of said environment, in which said marker was placed, by means of a conventional camera; and

c. receiving said image of said environment with a maker from said camera by means of one or more servers, processing said image, and outputting a photorealistic image that contains said virtual item integrated within it, said one or more servers comprising;

c.1. a composer for enabling a user to compose a photorealistic image from said corresponding image of said environment;

c.2. an image processing unit for processing said corresponding image and for determining the location of said marker within said environment;

c.3. a configuration database for storing users'"'"' configurations and other data; and

c.4. an image rendering unit for reconstructing the photorealistic image by integrating said virtual item into said predefined location of said environment.

16. Method according to claim 15, further comprising determining by means of the image processing unit a spatial location of the virtual item to be integrated into the environment, in a place wherein the marker is located.

17. Method according to claim 15, further comprising determining the current lighting and corresponding shadow conditions of the photographed environment by means of the image processing unit, due to placing the marker within said environment.

18. Method according to claim 15, further comprising reconstructing, by means of the image rendering unit, the photorealistic image of the photographed environment by simulating its current lighting and corresponding shadow conditions.

19. Method according to claim 15, further comprising providing the marker with a predefined texture for enabling to determine its spatial orientation within the photographed environment.

20. Method according to claim 15, further comprising providing the marker with a mirror reflecting sphere for determining the lighting and corresponding shadow conditions of the environment, in which it is located.

21. Method according to claim 20, further comprising determining by means of the image processing unit, due to providing the marker mirror reflecting sphere, which color and/or intensity of the light within the photographed environment, comes from which direction.

22. Method according to claim 15, further comprising estimating camera parameters by means of the image processing unit.

23. Method according to claim 22, further comprising selecting the camera parameters from one or more of the following:

a. the focal distance of the lens of said camera;

b. the viewing direction and orientation of said camera; and

c. the position of said camera in a space.

24. Method according to claim 15, further comprising providing a model and material database for storing models and materials to be integrated into the taken image of the environment, or storing links to said models and materials, if they are physically stored on another server.

25. Method according to claim 15, further comprising displaying the marker on a mobile device screen or providing said marker in a printed form.