Generating registration data for a virtual set

• US 6,741,241 B1
• Filed: 02/19/1999
• Issued: 05/25/2004
• Est. Priority Date: 02/20/1998
• Status: Expired due to Term

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Malenfant, Bernard, Jaubert, Fabrice B.
• Primary Examiner(s): Tung, Kee M.
• Assistant Examiner(s): MANCUSO, HUEDUNG XUAN CAO
• Application Number:
  US09/252,928
• Time in Patent Office: 1,922 Days
• Field of Search: 345/419, 345/427, 345/429, 345/425, 345/418, 345/581, 345/679, 345/680
• US Class Current: 345/419
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

In a virtual set a computer generated virtual image is combined with a real set. Registration data defines a transformation allowing virtual image data to be combined with image data derived from a real set. A reference plane is identified (902) in the real set which is associated (903) to a plane defined in the virtual world space. Differences in positions between the real and virtual planes are determined (904) to produce registration data.

Claims

1. A method of generating registration data for a virtual set comprising:

identifying a plane in a real set;

associating said real plane to a plane defined in a virtual scene; and

determining differences in position between said real plane and said virtual plane to produce registration data, wherein said registration data defines a transformation for the virtual scene to facilitate the combination of said virtual scene with image data derived from the real set.

2. A method according to claim 1, wherein planes in said real set are defined by identifying the position of a plurality of co-planar points.

3. A method according to claim 1, wherein a plurality of planes are identified in the real set and each of said planes is associated to a respective plane in the virtual scene.

4. A method according to claim 1, wherein said virtual scene data is defined by a scene tree, relating a plurality of objects with respect to a root origin.

5. A method according to claim 1, wherein said position determining step includes removing scaling effects from said virtual plane with reference to a scene tree of objects.

6. A method according to claim 1, wherein said difference determining step includes determining a relationship between a real plane and a corresponding virtual plane in terms of translations and rotations.

7. A method according to claim 6, wherein said rotations and translations are defined by matrices and a registration matrix is generated by a concatenation of said rotation and translation matrices.

8. The method of claim 1 wherein:

the identifying of the real plane in the real set, is effected with respect to a first three dimensional co-ordinate origin;

identifying the virtual plane in the virtual set is effected with respect to a second three dimensional co-ordinate origin;

the determining of said differences is effected by comparing a vertex on each said plane with its respective origin, to obtain a transformation; and

said transformations are combined to generate said registration data.

9. A method according to claim 8, wherein said registration data is further modified by an operator, such that multiple registration data sets may be defined for a unique real set virtual set combination.

10. A method according to claim 8, wherein a plurality of planes is identified for combination in each of said real and virtual sets.

11. A method of generating registration data for a virtual set, comprising:

identifying a three-dimensional registration object in a real set;

identifying a three-dimensional model of said registration object in a virtual set; and

combining real and virtual positions of said registration object in order to generate said registration data.

12. A method of combining virtual scene data with images from a real set, comprising:

generating an image of a real set;

rendering an image of a virtual set from said virtual scene data, in combination with registration data; and

combining said real image with said rendered image wherein said registration data is determined by a process of matching a positional attribute of a plane common to said real set and said virtual set.

13. A method of producing image data in a virtual set, comprising:

loading virtual scene data;

generating registration data; and

using said registration data to combine real and virtual images, wherein said registration generation comprises;

identifying a real plane in said real images;

associating said real plane to a plane defined in a virtual scene, and

determining differences in position between said real plane and said virtual plane to produce registration data.

14. An apparatus for generating registration data for a virtual set, comprising:

means for identifying the real plane in the real set;

means for associating said real plane to a virtual plane defined in a virtual set;

means for determining differences in position between said real plane and said virtual plane to produce registration data, wherein said registration data defines a transformation for virtual scene data to facilitate the combination of said virtual data with image data derived from the real set; and

means for employing said registration data to align the virtual plane with the real plane.

15. Apparatus according to claim 14, including means for identifying the position of a plurality of co-planar points in order to define a plane in said real set.

16. Apparatus according to claim 14, including means for identifying a plurality of planes in the real set and means for associating each of said planes to a respective plane in the virtual scene.

17. Apparatus according to claim 14, including means for defining the virtual scene data by a scene tree, wherein said scene tree relates a plurality of objects with respect to a root origin.

18. Apparatus according to claim 14, including means for removing scaling effects from the virtual plane with reference to a scene tree of objects.

19. Apparatus according to claim 14, including means for determining a relationship between a real plane and a corresponding virtual plane in terms of translations and rotations.

20. Apparatus according to claim 19, including means for defining said rotations and translations in the form of matrices, including means for generating a registration matrix by a concatenation of said rotation and translation matrices.

21. A computer-readable medium having computer-readable instructions executable by a computer to generate registration data for a virtual set, said registration data being required to facilitate alignment of virtual scene data with real set data, such that the computer performs the steps of

reading data representing a selected reference plane in a real set;

identifying a corresponding virtual plane from said virtual scene data; and

computing a registration matrix to map said real reference plane to said corresponding virtual plane.

22. A computer-readable medium having computer-executable instructions for performing the steps according to claim 21, wherein a translation between said vertices is calculated.

23. A computer-readable medium having computer-executable instructions for performing the steps according to claim 22, wherein rotations of said virtual plane are calculated.

24. A computer-readable medium having computer-executable instructions for performing the steps according to claim 23, wherein said registration data is defined in terms of a four-by-four array of characters.