Method and system for manipulation of objects in a television picture

• US 6,373,508 B1
• Filed: 03/23/1999
• Issued: 04/16/2002
• Est. Priority Date: 04/19/1996
• Status: Expired due to Fees

• Current Assignee: Spotzoom AS
• Original Assignee: Spotzoom AS
• Inventors: Moengen, Harald K.
• Primary Examiner(s): Bayerl, Raymond J.
• Application Number:
  US09/171,424
• Time in Patent Office: 1,120 Days
• Field of Search: 345/848-849, 345/726, 345/719, 345/716, 345/156, 345/419, 345/427, 345/632-633, 345/672, 345/682-683, 345/629, 348/25, 348/29, 348/157, 348/159, 348/169, 348/589, 382/103, 382/154, 473/415, 473/570, 473/569, 273/371, 273/317, 700/91, 702/150, 702/152, 342/125-126, 342/450-451, 342/453, 342/463
• US Class Current: 715/848
• CPC Class Codes: G01S 13/46 Indirect determination of position data
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 2013/466 by Trilateration, i.e. two antennas or two sensors determine separately the distance to a target, whereby with the knowledge of the baseline length, i.e. the distance between the antennas or sensors, the position data of the target is determined
• CPC Class Codes: G01S 3/786 the desired condition being maintained automatically
• CPC Class Codes: G10K 2210/1051 Camcorder
• CPC Class Codes: G10K 2210/113 Elevators
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

In a method for manipulation of a movable object displayed in a television picture, the distance between the object and fixed basic positions is detected at a time t together with the distance between the object and a television camera in a known position. The object'"'"'s position is converted to a position X,Y in the television camera'"'"'s picture plane, generating therein a synthetic object which overlays the movable object and represents it in the television picture. In a method for generating a synthetic track which represents the path of a movable object displayed in television pictures during a period θ, the path of the object is calculated on the basis of its detected positions, and these positions are used for generating a synthetic track which is displayed in a television picture in order to represent the path of the object in the period T.

Claims

1. A method for manipulation of at least one movable, natural object in a natural television picture, wherein the television picture is generated by one or more television cameras, characterized by detecting the distance between the object and at least 2 fixed basic positions in a preselected x,y,z co-ordinate system at a time t, each basic position corresponding to a known position of a detector, determining an x,y,z co-ordinate for the object in the preselected x,y,z co-ordinate system at the time t, determining the distance between the camera'"'"'s lens centre and the object at the time t as an object vector in the preselected co-ordinate system, determining the television camera'"'"'s optical axis in the preselected co-ordinate system at the time t as a camera vector in the preselected co-ordinate system, determining a line from the television camera'"'"'s lens centre to the point of intersection between the edge of the generated television picture and the plane formed between the object vector and the camera vector at the time t as a zoom vector, the object vector being located between the camera vector and the zoom vector when the object is visible in the television picture at the time t, and when the object is not visible in the television picture at the time t, the zoom vector is located between the camera vector and the object vector, and determining an X,Y position of the object referred to the television camera'"'"'s picture plane and the camera vector on the basis of the object vector and the camera vector at the time t, and if the object vector is located between the camera vector and the zoom vector, to insert a synthetic object in the X,Y position in the television picture at the time t, the synthetic object constituting a representation of the natural object recorded by the camera at the time t, or, if the zoom vector is located between the camera vector and the object vector at the time t, to insert a symbol in the television picture, the symbol indicating the location of the X,Y position of the natural object outside the edge of the picture referred to the television camera'"'"'s picture plane and the camera vector.

2. A method according to claim 1, characterized in that the distance between the object and the basic positions, together with the object'"'"'s x,y,z co-ordinate are determined by trilateration.

3. A method according to claim 1, characterized in that there are employed 2 fixed basic positions in the preselected x,y,z co-ordinate system if the natural object is located for every value of t in a plane defined by the fact that one of the co-ordinates x,y,z of the natural object is equal to zero, or that at least 4 fixed basic positions are employed in the preselected x,y,z co-ordinate system if the natural object is located for at least one or some values of t in a space defined by the fact that none of the co-ordinates x,y,z is equal to 0, thus obtaining an unambiguous determination of the natural object'"'"'s x,y,z co-ordinate at time t.

4. A method according to claim 1, characterized in that the synthetic object is either created with given attributes for size, shape and colour, or that the synthetic object'"'"'s attributes are freely selected, or that the synthetic object'"'"'s attributes are chosen within preselected limits, or that the synthetic object'"'"'s attributes are determined on the basis of respective reference values for the attributes.

5. A method according to claim 4, wherein the synthetic object'"'"'s attributes are determined on the basis of respective reference values for the attributes, characterized in that the synthetic object'"'"'s attributes are manipulated automatically on the basis of setting values for the television camera such as camera angle and zoom setting and on the basis of the colour of the background of the natural object as shown in the television picture.

6. A method according to claim 1, characterized in that the synthetic object is assigned alphanumerical or symbolic signs which indicate the value of one or more parameters of the represented natural object, and the parameters may be constants which identify the natural object or dynamic values such as the object'"'"'s momentary x,y,z position, distance to a freely selected fixed point, course and speed.

7. A method according to claim 1, characterized in that the settings of the television camera such as camera angle and zoom setting are controlled in approximate real time on the basis of the calculated X,Y position of the natural object at the time t and referred to the camera'"'"'s picture plane and the camera vector, with the result that the natural object is located within the edge of the picture of the generated television picture at any time t, the control being performed via a control system assigned to the camera.

8. A method according to claim 7, wherein more than one camera is employed, characterized in that a choice is made via the control system as to which camera should be employed to generate the television picture which is shown at time t.

9. A method for generating at least one synthetic track in a television picture, wherein the synthetic track represents the path of a movable, natural object in a natural television picture during a given period θ

, and wherein a method is employed for manipulating the movable, natural object in the television picture as indicated in claim 1, characterized by calculating the path of the natural object on the basis of detected positions x,y,z for the a natural object in the preselected x,y,z co-ordinate system at the time t, where tε

θ

, converting the detected positions at the time t to an X,Y position in the television camera'"'"'s picture plane at the time t, and generating the synthetic track in the television picture as the connecting line between all X,Y positions of the natural object in the picture planes of the natural television pictures which are recorded sequentially during the period θ

.

10. A method according to claim 9, characterized in that the synthetic track is generated and updated continuously for each individual picture which is recorded during the period θ

.

11. A method according to claim 10, characterized in that the synthetic track is generated and displayed as the path of the natural object during the period θ

in a freely selected television picture which forms a background for the natural object, the X,Y positions being scaled for each time tε

θ and

assigned to the picture plane of the freely selected television picture.

12. A method according to claim 11, characterized in that the freely selected television picture is a natural television picture, the freely selected television picture forming part of the sequence of the natural television pictures which are recorded during the period θ

, or that the freely selected television picture is a synthetic television picture.

13. A method according to claim 9, characterized in that the X,Y positions for one or more of the times t are indicated on the synthetic track, the indication of the X,Y position in the synthetic track being formed as a synthetic object with given attributes for size, shape and colour, and in such a manner that the synthetic object constitutes a representation of the natural object recorded by the camera at the time tε

θ

.

14. A method according to claim 13, characterized in that the indication of the X,Y positions or the synthetic object is assigned alphanumerical or symbolic signs which indicate the value or one or more parameters for the represented natural object, and the parameters may be constants which identify the natural object or dynamic values such as the object'"'"'s momentary x,y,z position, distance from a freely selected fixed point, course and speed at the time tε

θ

.

15. A system for implementing the method for manipulation of at least one movable, natural object (N) in a natural television picture, wherein the television picture is generated by one or more television cameras (K), together with implementation of the method for generating a synthetic track in a television picture, wherein the synthetic track represents the path of a movable, natural object in the television picture during a given period θ

, and wherein a method is employed for manipulating the movable, natural object in the television picture, characterized in that the system comprises a transponder (T) provided in the natural object (N) and arranged to react to an optical, acoustic or electromagnetic signal received by the transponder with transmission of a response signal, at least one position module (M) with at least 2 position detectors (D) for transmitting optical, acoustic or electromagnetic signals and receiving response signals from the transponder (T) and provided in respective basic positions in a preselected x,y,z co-ordinate system, together with a signal processor (1) arranged to determine the distance between a position detector (D) and the object (N) at a time t, a computing module (2) connected with the signal process (1) and arranged to calculate the x,y,z co-ordinates for the object (N) in the preselected co-ordinate system at the time t and on the basis of the calculated x,y,z co-ordinates for a number of times t to calculate a path for the object (N), together with an object vector (V_N) given by the distance between the camera'"'"'s (K) lens centre and the object (N) at the time t, a camera vector (V_K) given by the camera'"'"'s (K) optical axis at the time t and a zoom vector (V_z) between the camera'"'"'s lens centre and the point of intersection (Z) between the edge of the picture and the plane formed between the object vector (V_N) and the camera vector (V_K) at the time t, the object vector (V_N) being either located between the camera vector (V_K) and the zoom vector (V_Z) or the zoom vector (V_Z) between the camera vector (V_K) and the object vector (V_N), a camera control system (3) connected with the computing module (2) and arranged to detect or generate values for the camera settings, and a manipulator module (4) connected with the camera control system (3) and the computing module (2) arranged to a) create a synthetic object (S) in an X,Y position for the natural object (N) in the recorded television picture at the time t, the synthetic object )S) constituting a representation of the natural object (N) recorded by the camera at the time t or b) create a symbol n the television picture, the symbol indicating the X,Y position for the natural object outside the edge of the picture at the time t, c) generate and select attributes for the synthetic object (S), or d) generate at the time t a synthetic track in a recorded or generated television picture, the synthetic track representing the path for the natural object (N) during a period θ

before or up to the time t.

16. A system according to claim 15, characterized in that the system comprises a number of position modules (M) each of which is assigned to one or more respective television cameras (K), each position module (M) being arranged to cover a predefined area in the x,y,z co-ordinate system.

17. A system according to claim 15, characterized in that the position detector (D) comprises a microwave transceiver and an antenna (II) for transmission and receipt of microwave signals, and that there is further provided a transponder T in each of the position detectors (D) and/or in each of the cameras (K) respectively.

18. A system according to claim 17, wherein each said position module comprises at least 4 position detectors (D1, . . . D4) for unambiguous determination of the x,y,z co-ordinates for the object (N) at the time t.

19. A system according to claim 15, characterized in that the transponder (T) is either a passive transponder, or an active transponder, the active transponder being a surface wave component (SAW component), and that the response signal from the active transponder (T) in each case constitutes a code which unambiguously identifies the transponder by delaying the response signal from the active transponder (T) by a predetermined value τ

_d in relation to the time of receipt of a signal which causes the response signal to be transmitted form the transponder.

20. A system according to claim 15, characterized in that the camera control system (3) comprises a control loop for automatic control of camera settings in approximate real time, the camera settings being influenced via a regulator module (R) provided on the camera (K) which module further comprises sensors for detection of the camera settings, and/or that the manipulator module (4) comprises a device for automatic generation of attributes for the synthetic object.