Method and apparatus for determining the position of a TV camera for use in a virtual studio

• US 6,304,298 B1
• Filed: 07/02/1997
• Issued: 10/16/2001
• Est. Priority Date: 09/08/1995
• Status: Expired due to Fees

• Current Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Original Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Inventors: Livshits, Zinovy, Nissim, Moshe, Aufhauser, David, Tamir, Michael, Steinberg, Alexander, Wilf, Itzhak, Sharir, Avi
• Primary Examiner(s): Eisenzopf, Reinhard J.
• Assistant Examiner(s): Lo, Linus H.
• Application Number:
  US08/765,898
• Time in Patent Office: 1,567 Days
• Field of Search: H04/N.974, 348/587, 348/586, 348/590, 348/591, 348/592, 348/578, 348/140, 348/25, 348/700, 348/118, 382/284, 702/152
• US Class Current: 348/587
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

A method of determining the position of a TV camera relative to a patterned panel being viewed by the TV camera including the steps of:

identifying a plurality of edge points of the pattern from the video signal produced by said camera and using these edge points to calculate the perspective of the pattern relative to the camera.

Claims

1. A method of determining the position of a TV camera relative to a patterned panel, comprising a pattern of vertical and horizontal straight lines that delineate a color difference, being viewed by the TV camera, comprising:

identifying a plurality of edge points, which are each disposed on one of said horizontal or vertical straight lines, from a video signal of the patterned panel produced by said camera;

using these edge points to calculate a perspective of the patterned panel relative to the camera, comprising

(i) clustering the plurality of edge points to associate edge points and to specific lines using a slope and intercept process;

(ii) analyzing all identified edge points and grouping at least some of the identified edge points into a first group that corresponds to the horizontal lines and a second group that corresponds to the vertical lines;

(iii) allocating the first and second groups to specific horizontal and vertical lines;

(iv) computing vanishing points of the specific horizontal and vertical lines, said vanishing points being computed within a defined location error and projecting the edge points that were allocated to specific horizontal lines from the computed vanishing point of the horizontal lines to obtain an edge projection profile map comprising peaks and troughs;

and determining a relative position of the camera relative to the patterned panel by reference to the perspective of the patterned panel.

2. The method as claimed in claim 1 wherein identifying a plurality of edge points comprises identifying a plurality of first edge points and a plurality of second edge points from the plurality of edge points;

and wherein using these edge points further comprises producing an edge image from the plurality of first edge points and the plurality of second edge points.

3. The method as claimed in claim 1, further comprising:

assigning each edge point that was allocated to a specific horizontal line to a most probable peak of an edge projection profile map and for a given most probable peak producing a list of lines from a plurality of candidate lines that may be indicated by the given most probable peak.

4. The method as claimed in claim 3 further comprising either specifying a line from the patterned panel for each list of lines or discarding each list of lines that does not correspond to a line of the patterned panel.

5. The method as claimed in claim 1 further comprising:

projecting the edge points that were allocated to specific vertical lines from the computed vanishing point of the vertical lines to obtain an edge projection profile map comprising peaks and troughs;

assigning each edge point that was allocated to a specific vertical line to a most probable peak and for a given most probable peak producing a list of lines from a plurality of candidate lines that may be indicated by the given most probable peak; and

for each list of lines either specifying a line from the patterned panel that is indicative of the respective list of lines or discarding each list of lines that does not correspond to a line of the patterned panel.

6. The method as claimed in claim 4, wherein computing vanishing points comprises computing accurate vanishing points.

7. The method as claimed in claim 6 further comprising determining a shift and a scale of the patterned panel from the accurate vanishing points.

8. The method as claimed in claim 7, wherein determining a shift and a scale comprises producing an accurate line pattern by means of inverse perspective transformation and comparing the patterned panel with the accurate line pattern.

9. The method as claimed in claim 8, in which comparing the patterned panel comprises identifying a first horizontal line in the accurate line pattern, identifying a second horizontal line in the accurate line pattern, calculating a distance between said first and said second horizontal lines, comparing the calculated distance between the first and the second horizontal lines with the known patterned panel to produce a horizontal position and scale determination, identifying a first vertical line in the accurate line pattern, identifying a second vertical line in the accurate line pattern, calculating a distance between the first and the second vertical lines, comparing the calculated distance between the first and the second vertical lines with the known patterned panel to produce a vertical position and scale determination and from said horizontal and vertical position and scale determinations determining the position of the TV camera relative to the patterned panel.

10. The method as claimed in claim 1 wherein the patterned panel comprises a chroma-key panel having two separately identifiable chroma-key colours.

11. The method as claimed in claim 1 in which the patterned panel comprises two or more distance coded families of lines.

12. The method as claimed in claim 1 in which the patterned panel comprises two or more families of lines such that the lines of each family intersect at a common point.

13. The method as claimed in claim 10 in which the determination of the relative position of the TV camera relative to the patterned panel is used to calculate a perspective of a background video picture relative to a foreground object.