Method and apparatus for creating a composite image

• US 8,031,210 B2
• Filed: 09/30/2007
• Issued: 10/04/2011
• Est. Priority Date: 09/30/2007
• Status: Expired due to Fees

• Current Assignee: RDV Systems Limited
• Original Assignee: RDV Systems Limited
• Inventors: Elsberg, Nathan, Hazanov, Alex
• Primary Examiner(s): AMINI, JAVID A
• Application Number:
  US11/865,003
• Publication Number:
  US 20090085910A1
• Time in Patent Office: 1,465 Days
• Field of Search: 345/632
• US Class Current: 345/632
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2210/62 Semi-transparency
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A method of rendering a computer generated 3D scene integrated with a base image, the method comprising loading a base image, such as a photograph, and a computer generated 3D scene or model. The base image is displayed on a monitor, and in one embodiment the calibration of the camera which generated the base image is determined. The 3D model is rendered as an overlay of the base image responsive thereto. In another embodiment, the camera calibration of the 3D scene is made consonant with the base image by selecting corresponding points. The base image is then additionally displayed at a predetermined transparency as an overlay of the 3D model. A user then selects pixels of the base image overlay for placement in the foreground. The selected pixels are then displayed without transparency and the balance of the base image overlay removed rendering the integrated image.

Claims

1. A computing platform arranged to render an integrated two dimensional image constituted of a computer generated three dimensional scene and a two dimensional base image the computing platform comprising a computing device, a memory, a monitor, and a user input device, the computer being arranged to:

provide the two dimensional base image;

display said two dimensional base image on said monitor;

provide the computer generated three dimensional scene;

determine the camera calibration of said displayed two dimensional base image;

render said provided computer generated three dimensional scene on said monitor responsive to said determined camera calibration of said displayed two dimensional base image as an overlay over said displayed two dimensional base image;

additionally display the two dimensional base image in the foreground of said rendered three dimensional scene at a predetermined transparency;

identify elements, via said user input device, from said additionally displayed two dimensional base image to appear in the foreground of the integrated image, wherein said identified elements are a subset of the total displayed elements of said additionally displayed two dimensional base image;

display said identified elements of said additionally displayed two dimensional base image without transparency on said monitor; and

remove all unidentified elements of said additionally displayed two dimensional base image thereby rendering said integrated two dimensional image on said monitor as a combination of said displayed two dimensional base image overlaid by said rendered three dimensional scene with said identified elements of said additionally displayed two dimensional base image displayed without transparency in the foreground.

2. A computing platform according to claim 1, wherein said determination of the camera calibration of said displayed two dimensional base image is responsive to at least one user selected point on said displayed two dimensional base image and a user input coordinate of said at least one selected point.

3. A computing platform according to claim 1, wherein said determination of the camera calibration of said displayed two dimensional base image is responsive to at least one user selected point on said displayed two dimensional base image and a corresponding respective at least one point on said rendered computer generated three dimensional scene.

4. A computing platform according to claim 1, wherein said computing device is further arranged, after said rendering of said provided computer generated three dimensional scene on said monitor responsive to said determined camera calibration as an overlay over said displayed two dimensional base image, to:

adjust, responsive to a user input, said determined camera calibration for rendering said provided computer generated three dimensional scene.

5. A computing platform according to claim 1, wherein said removing all unidentified elements of said additionally displayed two dimensional base image comprises:

display said unidentified elements at 100% transparency.

6. A computing platform according to claim 1, wherein said provided computer generated three dimensional scene comprises at least one element exhibiting matte texture.

7. A computing platform according to claim 1, wherein said provided two dimensional base image is a photograph.

8. A computer-readable medium containing instructions for controlling a computing device to perform a method of rendering an integrated two dimensional image constituted of a computer generated three dimensional scene and a two dimensional base image, the method comprising:

providing the two dimensional base image;

displaying said two dimensional base image;

providing the computer generated three dimensional scene;

determining the camera calibration of said displayed two dimensional base image;

rendering said provided computer generated three dimensional scene responsive to said determined camera calibration of said displayed two dimensional base image as an overlay over said displayed two dimensional base image;

displaying said two dimensional base image additionally in the foreground of said rendered three dimensional scene at a predetermined transparency;

identifying elements from said additionally displayed two dimensional base image to appear in the foreground of the integrated two dimensional image, wherein said identified elements are a subset of the total displayed elements of said additionally displayed two dimensional base image;

displaying said identified elements of said additionally displayed two dimensional base image without transparency; and

removing all unidentified elements of said additionally displayed two dimensional base image thereby rendering said integrated two dimensional image as a combination of said displayed two dimensional base image overlaid by said rendered three dimensional scene with said identified elements of said additionally displayed two dimensional base image displayed without transparency in the foreground.

9. A computer-readable medium according to claim 8, wherein said determining the camera calibration of said displayed two dimensional base image is responsive to at least one user selected point on said displayed two dimensional base image and a user input coordinate of said at least one selected point.

10. A computer-readable medium according to claim 8, wherein said determining the camera calibration of said displayed two dimensional base image is responsive to at least one user selected point on said displayed two dimensional base image and a corresponding respective at least one point on said rendered computer generated three dimensional scene.

11. A computer-readable medium according to claim 8, wherein the method further comprises after said rendering said provided computer generated three dimensional scene as an overlay:

adjusting, responsive to a user input, said camera calibration for rendering said provided computer generated three dimensional scene.

12. A computer-readable medium according to claim 8, wherein said removing all unidentified elements of said additionally displayed two dimensional base image comprises displaying said unidentified elements at 100% transparency.

13. A computer-readable medium according to claim 8, wherein said provided computer generated three dimensional scene comprises at least one element exhibiting matte texture.