Automatic event videoing, tracking and content generation

• US 9,094,615 B2
• Filed: 04/18/2005
• Issued: 07/28/2015
• Est. Priority Date: 04/16/2004
• Status: Expired due to Fees

• Current Assignee: Maxx Holdings, Inc.
• Original Assignee: Intheplay, Inc.
• Inventors: Aman, James A., Bennett, Paul Michael
• Primary Examiner(s): Nguyen, Angela
• Application Number:
  US11/578,710
• Publication Number:
  US 20070279494A1
• Time in Patent Office: 3,753 Days
• Field of Search: 348/157
• US Class Current: 1/1
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/745 Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/262 : Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/2627 : for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80
• CPC Class Codes: H04N 5/268 : Signal distribution or switching
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/278 : Subtitling

Abstract

An automatics system 100 that uses one to three grids 20cm of overhead cameras 20c to first video an event area 2. Overall bandwidth is greatly reduced by intelligent hubs 26 that extract foreground blocks 10m based upon initial and continuously updated background images 2r. The hubs also analyze current images 10c to constantly locate, classify and track in 3D the limited number of expected foreground objects 10. As objects 10 of interest are tracked, the system automatically directs ptz perspective view cameras 40c to follow the activities. These asynchronous cameras 40c limit their images to defined repeatable pt angles and zoom depths. Pre-captured venue backgrounds 2r at each repeatable ptz setting facilitate perspective foreground extraction. The moving background, such as spectators 13, is removed with various techniques including stereoscopic side cameras 40c-b and 40c-c flanking each perspective camera 40c. The tracking data 101 derived from the overhead view 102 establishes event performance measurement and analysis data 701. The analysis results in statistics and descriptive performance tokens 702 translatable via speech synthesis into audible descriptions of the event activities corresponding to overhead 102 and perspective video 202.

Claims

1. A system for creating blended content associated with an event, the system comprising:

a live-content recording apparatus including two or more cameras for recording video data throughout the entire duration of the event, where the video recordings are either of the event activities or event spectators, where each of the two or more cameras outputs original distinct streams of temporally sequential video frames as a multiple data source of event live-content, and where the two or more original streams form multi-camera video data;

an event activity tracking system for measuring in real-time the event activity of one or more event objects throughout the entire duration of the event, where event objects are either objects or one or more people associated with the event, where people are either participants in the event or spectators, and for outputting the tracking data as a multiple data source of event live-content; and

a content assembly system including a blending algorithm for creating new blended content in real time from the multiple data sources of event live-content, where the new blended content at least includes one or more new streams of video frames, where each new stream is unique in its composition of video frames with respect to any and all of the original video streams, where each new stream contains at least two video frames taken from different original streams, and where the content assembly system receives the multi-camera video data and tracking data and selects each of the at least two video frames taken from different original streams at least in part using the tracking data,

thereby creating a broadcast with undetectable delay to the human viewer of the on-going event.

2. The system of claim 1 where the content assembly system conditionally blends other non-live-content with the blended content, the system further comprising:

non-live-content, where the non-live-content includes any one of, or any combination of;

(a) event replay content comprising one or more portions of prior-captured live-content of or relating to the event, as prior captured by any of the live-content recording apparatus, or (b) other content that was not captured by any of the live-content recording apparatus including advertising content, where advertising content includes any of video, audio, images, graphics, text or any other electronic representations of information,

where the blending algorithm is adapted to receive and blend any of the non-live-content with the blended content, where the blending algorithm performs any one of or any combination of;

(a) selecting which non-live-content to insert into, or overlay onto, the blended content, or (b) selecting the time at which non-live-content is inserted into, or overlaid onto, the blended content.

3. The system of claim 1 wherein the live-content recording apparatus includes at least one calibrated camera for recording calibrated video data for inclusion in the multi-camera video data, where the calibrated camera is any of the following types including:

(i) cameras that are fixed and non-movable with respect to the event venue, where the fixed X, Y, Z location as well as the fixed pan, tilt, and zoom settings of the camera with respect to the event venue are known to the system;

(ii) cameras that are fixed and movable with respect to the event venue, where the fixed X, Y, Z location of the camera with respect to the event venue is known to the system, where any one of, or any combination of the movable pan, tilt, or zoom settings of the camera with respect to the event venue are electronically adjustable in response to control signals, and where the control signals emanate from the system, such that the current pan, tilt, and zoom settings are known to the system;

(iii) cameras that are fixed and movable with respect to the event venue, where the fixed X, Y, Z location of the camera with respect to the event venue is known to the system, where any one of, or any combination of the movable pan, tilt, or zoom settings of the camera with respect to the event venue are manually adjustable and determinable by the system, and where the current pan, tilt, and zoom settings are known to the system;

or (iv) cameras that are free-floating and manually operable at the event venue, where the current movable X, Y, Z location, pan and tilt orientation, as well as zoom setting of the camera with respect to the event venue are determinable by and known by the system.

4. They system of claim 3 where the at least one calibrated camera is attached to any combination of movement apparatus controllable by the system for effecting the adjustment of any of the calibrated camera'"'"'s settings, where the camera'"'"'s settings include any one of, or any combination of the camera'"'"'s:

(i) X, Y, Z location with respect to the venue;

(ii) pan or tilt angle with respect to the venue, or (iii) zoom settings with respect to the venue, the system further comprising;

the live-content recording apparatus including two or more cameras, where at least one camera is a calibrated camera, further comprising any one of, or any combination of a controllable movement apparatus attached to any of the calibrated cameras for;

(a) adjusting the X or Y location of the attached calibrated camera with respect to the venue, where the movement apparatus includes continuous rail systems;

(b) adjusting the Z location of the attached calibrated camera with respect to the venue, where the movement apparatus includes extendable arm systems;

(c) adjusting the pan or tilt angle setting of the attached calibrated camera with respect to the venue, where the movement apparatus includes swivel mounts, or (d) adjusting the zoom setting of the attached calibrated camera with respect to the venue; and

a filming system for controlling the live-content recording apparatus and any of the attached movement apparatus, where the filming system receives the video data from the at least one calibrated camera, the tracking data, as well as the known current camera settings for the at least one calibrated camera, where the filming system uses the known and current X, Y, Z location as well as pan, tilt, or zoom settings of the calibrated camera with respect to the event venue in order to determine the spatial portion of the event venue that is currently being viewed within each distinct video frame captured by the calibrated camera, where the filming system uses any of the tracking data relating to any one or more event objects in order to determine adjustments to any one of or any combination of (a) the X or Y location of the calibrated camera with respect to the venue;

(b) the Z location of the calibrated camera with respect to the venue;

(c) the pan or tilt angle settings of the calibrated camera with respect to the venue;

or (d) the zoom setting of the calibrated camera with respect to the venue, where the filming system sends electronic signals to the one or more movement apparatus controlling the calibrated camera to be adjusted, and where the controlling apparatus makes the adjustments to the attached calibrated camera responsive to the system'"'"'s signal.

5. They system of claim 3 where the blending algorithm for creating blended video streams uses a mapping database that correlates the identity, location, or orientation of distinct event objects to distinct calibrated video frames captured by calibrated cameras, the system further comprising:

a filming system for controlling the live-content recording apparatus, where the filming system receives the video data from the at least one calibrated camera, the tracking data as well as the known current camera settings for the at least one calibrated camera, where the filming system uses the known and current X, Y, Z location as well as pan, tilt or zoom settings of the calibrated camera with respect to the event venue in order to determine the spatial portion of the event venue that is currently being viewed within each distinct calibrated video frame captured by the calibrated camera, where the filming system uses any of the tracking data relating to any one or more event objects in order to correlate any of the event object information including identity, location, or orientation to each of the distinct calibrated video frames, and where the filming system outputs the information correlating event objects to calibrated video frames in an objects-to-video mapped database as a multiple data source of event live-content,

wherein the blending algorithm is adapted to receive the objects-to-video mapped database, where the blending algorithm selects which of the two or more video frame streams from the multi-camera video data to include in any of the one or more blended video streams using at least in part the objects-to-video mapped database.

6. The system of claim 5 where the event objects correlated with the calibrated video data are non-participants, the system further comprising:

one or more electronic devices that are worn by one or more event non-participants to be tracked, where non-participants includes spectators, where each electronic device is adapted to output electronic signals, wherein

the tracking system measures in real-time the event activity, and is adapted to receive and analyze the electronic signals output from the one or more electronic devices in order to determine tracking data including the identities and locations of the event non-participants wearing the devices;

the filming system controls the live-content recording apparatus, where the filming system receives the video data from the at least one calibrated camera, the tracking data as well as the known current camera settings for the calibrated camera, where the filming system uses any of the tracking data relating to any one or more event non-participants in order to correlate any of the related tracking data including identity or location to each of the distinct video frames in which the tracked non-participants are determined to be in view, and where the filming system outputs this information correlating the event non-participants to calibrated video frames to the objects-to-video mapped database; and

the blending algorithm of the content assembly system creates blended content from the multiple data sources of event live-content, and is adapted to receive the objects-to-video mapped database, where the blending algorithm selects which of the two or more video frame streams from the multi-camera video data to include in any of the one or more blended video streams using at least in part the objects-to-video mapped database including the correlations between the non-participants and calibrated video frames.

7. The system of claim 3, where the calibrated video data are segmented and associated with the event objects using a mapping database that correlates the identity, location, or orientation of distinct event objects to distinct video frames, the system further comprising:

a filming system for controlling the live-content recording apparatus, where the filming system receives the video data from the at least one calibrated camera, the tracking data, as well as the known current camera settings for the at least one calibrated camera, where the filming system uses the known and current X, Y, Z location as well as pan, tilt, or zoom settings of the calibrated camera with respect to the event venue in order to determine the spatial portion of the event venue that is currently being viewed within each distinct calibrated video frame captured by the calibrated camera, where the filming system uses any of the tracking data relating to any one or more event objects in order to correlate any of the event object information including identity, location, or orientation to each of the distinct calibrated video frames, and where the filming system outputs the information correlating event objects to calibrated video frames in an objects-to-video mapped database as a multiple data source of event live-content; and

a segmenting algorithm for receiving the calibrated video data and the objects-to-video mapped database and for creating segmented video data associated with the event objects, where the segmenting algorithm operates upon the at least one calibrated video frame stream by analyzing each distinct calibrated video frame in order to determine which pixels of the calibrated video frame substantially represent any one of, or any combination of;

(i) the foreground, comprising event objects;

(ii) the background, comprising the event venue;

or (iii) the moving background, comprising spectators, where the segmenting algorithm uses the objects-to-video mapped database to associate tracking information with the foreground pixels, where the associated tracking information includes any one of or any combination of the event object'"'"'s type, identity, location, orientation, size or color, and where the segmenting algorithm outputs the segmented video data as a multiple data source of event live-content.

8. The system of claim 7 for further distinguishing the segmented foreground pixels from the segmented moving background pixels, the system further comprising:

a calibrated venue model describing the three-dimensional layout of the event venue including the locations of a performance area and spectator areas, where the event activities of the participants are expected to substantially occur within the performance area, where the event activities of the spectators are expected to substantially occur within the spectator areas, and where the coordinate system used in the calibrated venue model is relatable to the coordinate system used in the dataset of current calibrated camera settings,

wherein the segmenting algorithm receives the calibrated video data and the objects-to-video mapped database and creates segmented video data associated with the event objects, and is adapted to receive the calibrated venue model, where the segmenting algorithm additionally uses the calibrated venue model to determine within each video frame which spatial pixel areas pertain to the performance area verses the spectator areas, and where the segmenting algorithm uses the determined spatial pixel areas in order to at least distinguish (i) foreground pixels including event objects from (ii) moving background including spectators.

9. The system of claim 3 where event participants are identified using a participant identification system, the system further comprising:

one or more identification marks placed on at least one participant;

one or more electronically adjustable calibrated cameras for capturing images of any participant; and

an identification system for videoing participants in order to identify a participant based upon their identification marks, where the identification system receives the tracking data from the tracking system relating to the current location of one or more event participants to be identified, where the identification system is adapted to determine one or more adjustments to the pan, tilt, or zoom settings of any one or more of the electronically adjustable calibrated cameras using at least the combination of;

(a) the known fixed X, Y, Z location of the calibrated camera with respect to the venue;

(b) the current combination of the pan, tilt, and zoom settings of the calibrated camera with respect to the venue; and

(c) the tracking data related to a participant, where the identification system transmits one or more control signals comprising the one or more adjustments to the calibrated camera during the event so that the electronically adjustable calibrated camera captures one or more images of a participant to be identified, where the identification system analyzes each image to determine if any identification mark is present, where any marks determined to be present are further analyzed to determine the participant'"'"'s identity, and where the identification system communicates the identity of the participant to the tracking system for storage in the tracking database concurrent with the participant'"'"'s tracking information.

10. The system of claim 9 where the event is a sporting event and where the event participants are any one of, or any combination of players or game officials, and the identification marks are symbols including either numbers or letters placed anywhere on the participant'"'"'s clothing or equipment.

11. The system of claim 3, where the calibrated video data is segmented, the system further comprising:

a segmenting algorithm for receiving the calibrated video data and creating segmented video data, where the segmenting algorithm operates upon at least one calibrated video frame stream captured from at least one calibrated camera, where for each calibrated video frame stream to be segmented, the segmenting algorithm analyzes the calibrated video frames within the stream in order to determine which pixels of the calibrated video frames substantially represent any one of, or any combination of segment types including;

(i) the foreground, comprising event objects;

(ii) the background, comprising the event venue;

or (iii) the moving background, comprising spectators, where the segmenting algorithm outputs the segmented video data as a multiple data source of event live-content.

12. The system of claim 11, wherein the segmenting algorithm creates the segmented video data in any format including:

(i) the original calibrated video data with additional segmenting information at least indicating which pixels comprise the foreground, background, or moving background segments;

or (ii) segmented foreground, background, or moving background video data with additional segmenting information indicating the spatial relationship of the segmented video data to the original calibrated video data.

13. The system of claim 11, wherein the segmented foreground video data are represented as either:

(i) the original foreground pixel data;

or (ii) a translation of the original foreground pixel data into graphic line art using an edge detection process.

14. The system of claim 11 wherein the segmenting algorithm for receiving calibrated video data and creating segmented video data is executed on any one of, or any combination of (i) the system camera capturing the original video frame stream;

or (ii) a hub connected to one or more of the system cameras.

15. The system of claim 11 wherein the segmented video data are usable as a substitute for the video data.

16. The system of claim 11 where the content assembly system conditionally blends other non-live-content overlaid onto or in between any of the segmented video data segments while creating blended content, the system further comprising:

non-live-content, where the non-live-content includes any one of, or any combination of;

(a) event replay content comprising one or more portions of prior-captured live-content of or relating to the event, as prior captured by any of the live-content recording apparatus, or (b) other content that was not captured by any of the live-content recording apparatus including advertising content, where advertising content includes any of video, audio, images, graphics, text or any other electronic representations of information,

where the blending algorithm is adapted to receive and blend any of the non-live-content with the blended content, where the blending algorithm performs anyone of or any combination of;

(a) selecting which non-live-content to insert into, or overlay onto, the blended content, or (b) selecting the time at which non-live-content is inserted into, or overlaid onto, the blended content, and where the blending algorithm receives and blends the segmented video such that any of the non-live-content is overlaid onto or in between any of the segments.

17. The system of claim 11, where the blended content is encoded and where the content assembly system includes an encoder, where during or after the content assembly and compression system selectively assembles any one of, or any combination of event live-content including video data, tracking data, or segmented video data into blended content, the content assembly system encodes the blended content.

18. The system of claim 17, wherein the encoder uses a selective encoding algorithm for processing the segmented video such that the encoding algorithm encodes the segment types of either (i) the foreground pixels, comprising event objects;

or (ii) the moving background pixels, comprising spectators, differently from (iii) the background pixels, comprising the event venue.

19. The system of claim 18, wherein the encoding algorithm further uses any of, or any combination of the tracking data to alter the encoding of at least the segment type of the foreground pixels.

20. The system of claim 18, adapted to decode the encoded blended content including segment types of segmented video that have been encoded differently, the system further comprising:

a decoder for decoding the encoded blended content, where the decoder receives and decodes the encoded blended content using a selective decoding algorithm for processing any of the blended segmented video included within the encoded blended content such that the decoding algorithm decodes the segment types of either (i) the foreground pixels, comprising event objects;

or (ii) the moving background pixels, comprising spectators, differently from (iii) the background pixels, comprising the event venue.

21. The system of claim 20 where the decoder conditionally blends other non-live-content with the decoded blended content, the system further comprising:

non-live-content, where the non-live-content includes any one of, or any combination of;

(a) event replay content comprising one or more portions of prior-captured live-content of or relating to the event, as prior captured by any of the live-content recording apparatus, or (b) other content that was not captured by any of the live-content recording apparatus including advertising content, where advertising content includes any of video, audio, images, graphics, text or any other electronic representations of information,

where the decoder includes a second blending algorithm adapted to receive and blend any of the non-live-content with the decoded blended content, where the second blending algorithm performs any one of, or any combination of;

(a) selecting which non-live-content to insert into, or overlay onto, the decoded blended content, or (b) selecting the time at which non-live-content is inserted into, or overlaid onto, the decoded blended content.

22. The system of claim 21 wherein the encoded blended content includes segmented video data and where the second blending algorithm included within the decoder conditionally overlays onto, or in between any of the decoded segmented video data segments, any of the non-live-content.

23. The system of claim 11, adapted to analyze at least the video segment type of foreground pixels comprising event objects in order to generate tracking data,

where the event activity tracking system includes a video analyzer, where the video analyzer is adapted to receive and analyze the segmented video data at least including the video segment type of foreground pixels comprising event objects in order to generate at least one datum of tracking data regarding the event objects.

24. The system of claim 23 where the at least one datum of tracking data regarding the event objects includes any one of, or any combination of an event object'"'"'s type, identity, location, orientation, size, or color.

25. The system of claim 23 where the video data comprise information about markings attached to event objects, where the markings are either visible or non-visible, where the segmented video data retain the marking information, and where the marking information is used by the tracking system'"'"'s video analyzer to determine at least any of the object'"'"'s type, identity, location, orientation, or size.

26. The system of claim 23 wherein the segmenting algorithm and the tracking system'"'"'s video analyzer are located on any one of, or any combination of:

(i) the system camera capturing the original video frame stream;

or (ii) a hub connected to one or more of the system cameras.

27. The system of claim 11 for extracting sub-frames from the segmenting calibrated video data, the system further comprising:

a dataset of current calibrated camera settings for all calibrated cameras, where the settings at least include the calibrated camera'"'"'s X, Y, Z location and orientation including pan and tilt angles with respect to the event venue, and where the settings are available for each captured frame within each calibrated camera'"'"'s video frame stream; and

a background image database for retrieving or determining a background image for each calibrated camera based at least upon the calibrated camera'"'"'s location and orientation,

where the segmenting algorithm is adapted to receive the dataset of current calibrated camera settings for any calibrated camera, where the segmenting algorithm uses the current settings associated with each calibrated video frame to retrieve an associated background frame, where at least in part the associated background is used to determine which video pixels in each calibrated video frame belong substantially to the (i) foreground or (ii) moving background versus the (iii) background, and where the segmented video data are then output as any one of, or any combination of;

(a) zero or more extracted sub-frames, where the extracted sub-frames comprise fewer pixels than the original video frame, where the fewer pixels substantially represent (i) the foreground comprising event objects, or (ii) the moving background comprising spectators, and where associated with each extracted sub-frame are the extraction coordinates relating the sub-frame to the original calibrated video frame from which it was derived;

or (b) the original calibrated video frame with associated segmenting information, were the segmenting information at least includes spatial indications of zero or more (i) foreground or (ii) moving background pixels contained within the original calibrated video frame.

28. The system of claim 27 where at least one of the calibrated cameras is fixed and non-movable with respect to the event venue, where the fixed X, Y, Z location as well as the fixed pan, tilt, and zoom settings of the calibrated camera with respect to the event venue are known to the system, where the associated background image database holds at least a background image of the calibrated camera'"'"'s fixed and non-movable view for use by the segmenting algorithm.

29. The system of claim 28 comprising two or more tracking cameras for capturing calibrated video data that are fixed and non-movable with respect to the event venue, where the tracking cameras are arranged in a grid such that the combination of the fields-of-views of all of the tracking cameras in the grid forms a contiguous view of an event performance area, where the event activities of the participants are expected to substantially occur within the performance area, and where the venue coordinates of the performance area are known to the system in relation to the coordinate system used in the dataset of calibrated camera settings for each of the tracking cameras.

30. The system of claim 29, wherein the tracking cameras comprising the grid are fixed such that the optical axis of each tracking camera'"'"'s view is substantially above and perpendicular to the performance area, such that each tracking camera'"'"'s view is substantially looking down at the performance area.

31. The system of claim 29 where the calibrated video data captured by the grid of tracking cameras is segmented by the segmenting algorithm, where the segmenting algorithm combines all of the zero or more extracted sub-frames from each of the calibrated video frame streams output by each of the tracking cameras into a single database of extracted tracking sub-frames, where the segmenting algorithm uses the calibrated tracking camera information in order to determine and associate with each sub-frame the coordinates of the portion of the performance area represented by each sub-frame, where the single database includes only the extracted sub-frames and otherwise no other non-extracted portions of the original video frame from any of the tracking camera'"'"'s video frame streams, where the non-extracted portions of the original tracking video frames represent background pixels, and where the segmenting algorithm outputs the single database of tracking sub-frames as a multiple data source of event live-content.

32. The system of claim 31, wherein the segmenting algorithm minimizes the information content within the extracted tracking video sub-frames by scrubbing each tracking sub-frame of background information, where scrubbing includes setting the pixel information relating to any background pixels to minimum values, where pixel information at least includes color or intensity, and where minimal values include the null value.

33. The system of claim 32, for analyzing the foreground pixels found in the single database of tracking sub-frames in order to generate tracking data, where the event activity tracking system includes a video analyzer adapted to receive and analyze the single database of tracking sub-frames including the video segment type of foreground pixels comprising event objects in order to generate at least one datum of tracking data regarding the event objects.

34. The system of claim 33, where the at least one datum of tracking data regarding the event objects includes any one of, or any combination of an event object'"'"'s type, identity, location, orientation, size, or color.

35. The system of claim 31, where the single database of extracted tracking sub-frames is used in combination with a background image of the performance area to recreate a single combined video of the entire performance area, the system further comprising:

a performance area background database of one or more background images representative of the entire performance area from the viewing perspective of the grid of tracking cameras, where each of the one or more background images are either a natural or animated image of the performance area, where a natural image is equivalent or substantially similar to an image captured from the combined grid of tracking cameras, where an animated image is generated by a computer animation algorithm, and where the content assembly and compression system assembles at least the foreground information from the single database of tracking sub-frames into the blended content such that the event activities of the event objects as viewed by the tracking cameras are represented in the blended content, where the event objects at least include the event participants; and

a viewing system for receiving blended content and for presenting a representation of the blended content to a viewer, where the blended content includes the foreground information from the single database of tracking sub-frames, where the viewing system has access to the performance area background database for retrieving any of the one or more background images of the performance area, and where the viewing system overlays the foreground information from the single database of tracking sub-frames onto a retrieved performance area background image that-such that the event activities of the event objects as viewed by the grid of tracking cameras are presented to the viewer.

36. The system of claim 27 where at least one calibrated camera is mounted in a fixed X, Y, Z location and has an adjustable range of combinations of pan, tilt, or zoom settings throughout the duration of the event such that the camera'"'"'s current view covers an adjustable range of views, where the associated background image database holds either (i) two or more stored background images, where each stored background image covers one combination of pan, tilt, or zoom settings substantially matching a potential view of the adjustable calibrated camera, or (ii) a panoramic background image covering two or more potential combinations of pan, tilt, or zoom settings, where from the panoramic image a derived background image representing a single pan, tilt, or zoom combination can be created to substantially match a potential view of the adjustable calibrated camera, and where the stored or derived background image corresponding to the adjustable calibrated camera'"'"'s current settings is used by the segmenting algorithm.

37. The system of claim 27, wherein the segmenting algorithm minimizes the information content within an extracted sub-frame by scrubbing the sub-frames of background information, where scrubbing includes setting the pixel information relating to any background pixels to minimum values, where pixel information at least includes color or intensity, and where minimal values include the null value.

38. The system of claim 27, wherein for each distinct calibrated video frame stream to be segmented, the stream is received throughout the entire duration of the event and comprises a continuous series of temporally distinct original video frames, where the segmenting algorithm analyzes each current original video frame in order to determine if the original video frame comprises one or more non-background pixels, where non-background pixels are either (i) foreground pixels representing an event object, or (ii) moving background pixels representing spectators, where for any spatially contiguous group of non-background pixels found within the current original video frame, the segmenting algorithm creates an extracted current sub-frame such that each current original video frame comprises zero or more current extracted sub-frames of non-background pixels, where the segmenting algorithm associates with each current extracted sub-frame:

(a) the extraction coordinates relating the sub-frame to the original video frame; and

(b) camera calibration information including any one or, or any combination of camera identification code and camera settings including the X, Y, Z camera location, pan angle setting, tilt angle setting, or zoom setting, and where the extracted sub-frames are output by the segmenting algorithm as a multiple data source of event-live content.

39. The system of claim 38, wherein for each current extracted sub-frame the segmenting algorithm either (a) forms a new sub-stream, or (b) appends the current sub-frame to an existing sub-stream based upon the decision of a temporal blending algorithm, where the temporal blending algorithm bases the decision to append a sub-frame to an existing sub-stream at least in part upon the determination if the current sub-frame shares at least one non-background pixel with the temporally prior sub-frame in the existing sub-stream, where the segmenting algorithm terminates any existing sub-stream that is not appended with a current sub-frame, and where any and all sub-streams are output by the segmenting algorithm as a multiple data source of event-live content.

40. The system of claim 38, for correlating the extracted sub-frames with the tracking data, the system further comprising:

a mapping device for receiving one or more sub-streams of extracted sub-frames and for correlating each sub-frame within each stream with the tracking data, where the tracking data and the camera calibration information associated with each sub-frame are either expressed in, or translatable into, a common local coordinate system, where the mapping device uses the tracking data that are at least related to the event objects that are participants in order to determine which participants are viewed as foreground pixels within which sub-frames, and where the mapping device associates with each sub-frame any of the tracking data pertaining to any of the event objects viewed within the sub-frame, where the associated tracking data include any of the event object'"'"'s type, identity, location, orientation, size, or color.

41. The system of claim 40, wherein for each current extracted sub-frame the segmenting algorithm either (a) forms a new sub-stream, or (b) appends the current sub-frame to an existing sub-stream based upon the decision of a temporal blending algorithm, where the temporal blending algorithm bases the decision to append a sub-frame to an existing sub-stream at least in part upon the determination if the current sub-frame either:

(i) shares at least one non-background pixel with the temporally prior sub-frame in the existing sub-stream, or (ii) shares at least one event object based upon the associated tracking data with the temporally prior sub-frame in the existing sub-stream, where the segmenting algorithm terminates any existing sub-stream that is not appended with a current sub-frame, and where any and all sub-streams are output by the segmenting algorithm as a multiple data source of event-live content.

42. The system of claim 41, further comprising segmented video from two or more calibrated cameras wherein the segmenting algorithm creates at least one sub-stream from sub-frames that were extracted from the calibrated video captured by the two or more calibrated cameras, where the two or more extracted sub-frames from different cameras are either:

(i) temporally concurrent, or (ii) temporally successive.

43. The system of claim 42, where any two temporally concurrent extracted sub-frames are combined into a new sub-frame representative of the same one or more event objects, and where the new combined sub-frame is used to replace the existing sub-frames from which the new combined sub-frame was created.

44. The system of claim 41, where the blended content including segmented video data in the fonn of sub-streams of sub-frames is encoded, the system further comprising:

an encoder for encoding the blended content, where during or after the content assembly system selectively assembles any one of, or any combination of event live-content including video data, tracking data, or segmented video data into blended content, the encoder encodes the blended content, where the blended content at least includes segmented video data of one or more sub-streams of extracted sub-frames, and where the encoder uses a selective encoding algorithm for processing the sub-frames such that the encoding algorithm encodes the segment types of either (i) the foreground pixels, comprising event objects;

or (ii) the moving background pixels, comprising spectators, differently from (iii) the background pixels, comprising the event venue.

45. The system of claim 44, wherein the encoder minimizes the information content within an extracted sub-frame by scrubbing the sub-frame of background information, where scrubbing includes setting the pixel information relating to any background pixels to minimum values, where pixel information at least includes color or intensity, and where minimal values include the null value.

46. The system of claim 44, wherein the encoding algorithm further uses any of the tracking data associated with any of the sub-frames within a stream to at least alter the encoding of any one of, or any combination of the segment types of (i) foreground pixels;

(ii) background pixels;

or (iii) moving background pixels.

47. The system of claim 44, wherein prior to encoding any given sub-frame the encoder performs a centering operation on the pixels in the given sub-frame in order to best orient the given sub-frame relative to either or both the prior or subsequent sub-frames within the sub-stream such that the overall displacement of the foreground pixels between successive sub-frames is minimized, where the encoder centers information associated with the given sub-frame sufficient for reversing the centering operation upon decoding, and where the centered information is encoded along with the sub-frame.

48. The system of claim 47, wherein the encoder uses any of the tracking data associated with any of the event objects represented by the foreground pixels contained within the given sub-frame in order to determine the best center of the given sub-frame for use during the centering operation, where any of the tracking data includes the event object'"'"'s type, identity, location, orientation, size, or color.

49. The system of claim 44, wherein prior to encoding any given sub-frame the encoder performs a rotation operation on the pixels in the given sub-frame in order to best orient the given sub-frame relative to either or both the prior or subsequent sub-frames within the sub-stream such that the overall displacement of the foreground pixels between successive sub-frames is minimized, where the encoder includes rotation information associated with the given sub-frame sufficient for reversing the rotation operation upon decoding, and where the rotation information is encoded along with the sub-frame.

50. The system of claim 49, wherein the encoder uses any of the tracking data associated with any of the event objects represented by the foreground pixels contained within the given sub-frame in order to determine the best rotation angle of the given sub-frame for use during the rotation operation, where any of the tracking data includes the event object'"'"'s type, identity, location, orientation, size, or color.

51. The system of claim 44, wherein prior to encoding any given sub-frame the encoder performs a zooming operation on the pixels in the given sub-frame in order to best orient the given sub-frame relative to either or both the prior or subsequent sub-frames within the sub-stream such that the overall displacement of the foreground pixels between successive sub-frames is minimized, where the encoder includes zooming information associated with the given sub-frame sufficient for reversing the zooming operation upon decoding, and where the zooming information is encoded along with the sub-frame.

52. The system of claim 51, wherein the encoder uses any of the tracking data associated with any of the event objects represented by the foreground pixels contained within the given sub-frame in order to determine the best zoom setting of the given sub-frame for use during the zooming operation, where any of the tracking data includes the event object'"'"'s type, identity, location, orientation, size, or color.

53. The system of claim 44, wherein prior to encoding any given sub-frame the encoder performs a split-join operation on the pixels in the given sub-frame in order to best match the foreground pixels to like event objects within the given sub-frame relative to either or both the prior or subsequent sub-frames within the sub-stream such that the overall synchronization of event object representation between successive sub-frames is maximized, and where the split-join operation includes either or both of:

(a) splitting a portion of the sub-frame away from the given sub-frame into a new sub-frame, where the split portion of the given sub-frame includes at least one foreground pixel, or (b) joining a different sub-frame into the given sub-frame such that the given sub-frame subsequently contains at least one new foreground pixel.

54. The system of claim 53, wherein the encoder uses any of the tracking data associated with any of the event objects represented by the foreground pixels contained within the given sub-frame in order to effect the split-join operation of the given sub-frame, where any of the tracking data includes the event object'"'"'s type, identity, location, orientation, size, or color.

55. The system of claim 44, wherein a given sub-frame within a sub-stream comprises a total number of pixels that is different from any other given sub-frame in the same sub-stream, and where prior to encoding the encoder either expands or contracts the total number of pixels in the given sub-frame so that its total pixels better match the total pixels of any other one or more sub-frames in the same sub-stream.

56. The system of claim 44, where during the encoding of the sub-frames, the encoded representation of color and saturation is limited to a table of color codes representing a set of known possible color-saturation values corresponding to either the event object'"'"'s type or identity, the system further comprising:

a color tone table of color codes for encoding the color and saturation values for at least any given foreground pixel representing either;

(i) a distinct event object type, or (ii) a distinct event object, where each code is associated with a set of values expressed in a traditional method for representing color and saturation, and where the traditional methods include RGB or UV,

where for any sub-frames within the blended content the encoder is adapted to access the color tone table for at least any foreground pixel found within the sub-frame, where the encoder accesses the table using the tracking data associated with the pixel, where the associated tracking data include either the event object type or the event object identity, where the encoder represents the color and saturation of the pixel within the encoded blended content using the color code retrieved from the table as a replacement for any other traditional pixel color and saturation encoding method, and where upon decoding the encoded color code is reversible back into the color and saturation values expressed in a traditional method for use as the color and saturation values of the pixel.

57. The system of claim 56, where the event is a sporting event and the event objects are participants that include at least players or referees wearing uniforms, where uniforms include equipment, where the color and saturation values to be encoded using a color code are any of the limited set of color and saturation values known to exist on the form of the participant, and where the form of the participant includes the participant'"'"'s uniform, equipment, and skin.

58. The system of claim 44, wherein the encoder performs one or more reversible transformations on any of the sub-frames to be encoded, and where the encoder includes the transformation parameters controlling the sub-frame transformation in the encoded blended content associated with the transformed sub-frame such that the transformation is then reversible upon decoding.

59. The system of claim 58, for presenting the encoded blended content to a viewer, the system further comprising:

a viewing system for receiving encoded blended content and for presenting a representation of the encoded blended content to a viewer, where the viewing system includes a decoder for decoding the encoded blended content, where the blended content includes at least one sub-stream of sub-frames, where the decoder reverses any transformations applied to any sub-frames during encoding, where the process of reversing any transformation uses transformation parameters found within the encoded blended content.

60. The system of claim 59, where for any one or more decoded background pixels either included or scrubbed within in the encoded sub-streams, the decoder performs any of:

(a) replacing the decoded background pixel with a pixel from a pre-known background image of the event venue that corresponds to the portion of the event venue represented within the sub-frame, or (b) replacing the decoded background pixel with an animated pixel from an animation background image of the event venue that corresponds to the portion of the event venue represented within the sub-frame.

61. The system of claim 60, where the decoder uses any of:

(a) the encoded extraction coordinates relating the sub-frame to the original video frame, or (b) the encoded camera calibration information including any one or any combination of a camera identification code and the camera settings including the X, Y, Z camera location, pan angle setting, tilt angle setting, or zoom setting, in order to determine a background image from which the replacement background pixel is derived for replacing the decoded background pixel.

62. The system of claim 61, where the decoder retrieves the background image containing the replacement background pixel from either:

(a) a database of background images, where individual background images correspond to the encoded camera calibration information, or (b) a background panoramic image from which a background image corresponding to the encoded camera calibration information is derived.

63. The system of claim 60, where the decoder uses an animated three-dimensional model of the event venue in order to derive the replacement background pixel, the system further comprising:

a calibrated venue animation model describing the three-dimensional layout of the event venue including an animation of the event venue, where the coordinate system used in the calibrated venue animation model is relatable to the coordinate system used in the encoded camera calibration information,

where the viewing system is adapted to access the calibrated venue animation model, where the decoder uses at least the combination of the venue animation model and the encoded camera calibration information in order to determine one or more animated pixels for replacing decoded background pixels.

64. The system of claim 59, where the decoder recreates a current background video frame onto which the decoded sub-frames are overlaid in order to create a representation of the original video frame from which the zero or more sub-frames were originally derived, where the recreated current background video frame is either a natural or animated image of the event venue, where a natural image is equivalent or substantially similar to an image captured from a system camera, and where an animated image is generated by a computer animation algorithm.

65. The system of claim 64, where the decoder retrieves the natural background image from either:

(a) a database of background images corresponding to the encoded camera calibration information, or (b) a background panoramic image from which a background image corresponding to the encoded camera calibration information is derived, and where the decoder uses the retrieved natural background image as the recreated current background video frame.

66. The system of claim 64, where the decoder uses an animated three-dimensional model of the event venue in order to derive the animated background image for use as the recreated current background video frame, the system further comprising:

a calibrated venue animation model describing the three-dimensional layout of the event venue including an animation of the event venue, where the coordinate system used in the calibrated venue animation model is relatable to the coordinate system used in the encoded camera calibration information,

where the decoder is adapted to access the calibrated venue animation model, where the decoder process uses at least the combination of the venue animation model and the encoded camera calibration information in order to determine an animation image for use as the recreated current background video frame.

67. The system of claim 64, where the decoder replaces recreated or decoded pixels with advertisement pixels such that an advertisement is added to the visual presentation to the viewer during the decoding process, the system further comprising:

a calibrated venue advertisement model describing the three-dimensional layout of the event venue including advertisements, where the coordinate system used in the calibrated venue advertisement model is relatable to the coordinate system used in the encoded camera calibration information, and where the advertisements are registered to the calibrated venue advertisement model such that the placement of the advertisement pixels in relation to the decoded pixels can be determined using the encoded camera calibration information,

where the viewing system is adapted to access the calibrated venue advertisement model, where the decoder uses at least the combination of the calibrated venue advertisement model and the encoded camera calibration information in order to determine one or more advertisements to add to the visual presentation to the viewer, and where advertisements are added to the visual presentation by overlaying one or more advertisement pixels onto the current background video frame either before or after decoded foreground pixels are added to the current background video frame.

68. The system of claim 67, where the advertisements contained within the calibrated venue advertisement model are either fixed or floating, where the decoder places the advertisement pixels of a fixed advertisement in relation to fixed venue locations, and where the decoder places the advertisement pixels of a floating advertisement in relation to either the current background video frame or the current location of an event object.

69. The system of claim 67, where the decoder selectively replaces recreated or decoded pixels with advertisement pixels such that an advertisement is added to the visual presentation to the viewer during the decoding process based upon a viewer profile and preferences database, the system further comprising:

a viewer profile and preferences database comprising viewer-related information, where viewer-related information includes any of the viewer'"'"'s name, age, address, relationship to the event, or other demographic information,

where the viewing system is adapted to access the viewer profile and preferences database for retrieving information usable by the decoder for selecting advertisements from the calibrated venue advertisement model to add to the visual presentation to the viewer during the decoding process.

70. The system of claim 64, where the decoder replaces recreated or decoded pixels with graphic overlay pixels such that graphic overlay content is added to the visual presentation to the viewer during the decoding process, where graphic overlays are either fixed or floating, where the decoder places the graphic overlay pixels of a fixed overlay in relation to fixed venue locations, where the decoder places the graphic overlay pixels of a floating overlay in relation to either the current background video frame or the current location of an event object, and where graphic overlays include any one of, or any combination of:

(a) changes to any one of, or any combination of the color, saturation, or intensity values of a recreated or decoded pixel, or (b) information contained within or derived from the encoded blended content, where information includes any event live-content from any of the system'"'"'s multiple data sources.

71. The system of claim 70, where the decoder selectively replaces recreated or decoded pixels with graphic overlay pixels such that a graphic overlay is added to the visual presentation to the viewer during the decoding process based upon a viewer profile and preferences database, the system further comprising:

a viewer profile and preferences database comprising viewer related information, where viewer related information includes any of the viewer'"'"'s name, age, address, relationship to the event, or other demographic information,

where the viewing system is adapted to access the viewer profile and preferences database for retrieving information usable by the decoder for selecting graphic overlay content from any of the decoded event live-content to add to the visual presentation to the viewer during the decoding process.

72. The system of claim 70, where tracking data are included in the encoded blended content, and where the decoder uses the decoded tracking data for any one of, or any combination of (a) using the tracking data for graphic overlay content, or (b) using the tracking data to determine at least in part the location of a floating graphic overlay.

73. The system of claim 70, where the event is a sporting event, and where scoreboard data are accepted or determined for inclusion in the encoded blended live content for use as a source of graphic overlay content during the decoding process, the system further comprising:

a scoring interface system for accepting or determining scoreboard data, where scoreboard data include any of the official scoring information required by the type of sport being played or information that would be displayed on a scoreboard associated with the event, and where

the scoring interface system outputs the scoreboard data as a multiple data source of event live-content,

where the content assembly system receives and assembles any of the scoreboard data into the blended content such that scoreboard data are included in the encoded blended content received by the viewing system; and

where the encoded blended content includes scoreboard data, and where any of the scoreboard data decoded from the encoded blended content is used as graphic overlay content.

74. The system of claim 40, wherein for each extracted sub-frame determined to represent two or more event objects, the segmenting algorithm divides the contiguous set of non-background pixels to best represent the two or more event objects, where each of the two or more divisions become distinct sub-frames replacing the original extracted sub-frame and where the associated tracking data are updated to reflect the division.

75. The system of claim 38, for analyzing the pixels within the extracted sub-frames to determine at least the video segment type of foreground pixels comprising event objects in order to generate tracking data,

where the event activity tracking system includes a video analyzer, where the video analyzer is adapted to receive and analyze the extracted sub-frames at least including the video segment type of the foreground pixels comprising event objects in order to generate at least one datum of tracking data regarding the event objects, and where the tracking system associates with each sub-frame any of the tracking data derived from and pertaining to any of the event objects viewed within the sub-frame.

76. The system of claim 75 where the at least one datum of tracking data regarding the event objects includes any one of, or any combination of an event object'"'"'s type, identity, location, orientation, size, or color.

77. The system of claim 76 wherein the tracking system creates a stream of symbolic data representative of the event objects and their activities, where geometric symbols are created to best fit selected event object shapes determined by the video analyzer, where geometric symbols at least include circles, ovals, and vectors, and where the tracking system outputs the stream of symbolic data as a multiple data source of event live-content.

78. The system of claim 75 where the video data comprise information about markings attached to event objects, where the markings are either visible or non-visible, where the extracted sub-frames retain the marking information, and where the marking information is used by the tracking system'"'"'s video analyzer in order to determine at least any of the object'"'"'s type, identity, location, orientation, or size.

79. The system of claim 75 for further distinguishing the segmented foreground pixels from the segmented moving background pixels within each sub-frame, the system further comprising:

a calibrated venue model describing the three-dimensional layout of the event venue including the locations of a performance area and spectator areas, where the event activities of the participants are expected to substantially occur within the performance area, where the event activities of the spectators are expected to substantially occur within the spectator areas, and where the coordinate system used in the calibrated venue model is relatable to the coordinate system used in the dataset of current calibrated camera settings,

where the segmenting algorithm is adapted to receive the calibrated venue model, where the segmenting algorithm additionally uses the calibrated venue model in comparison to the camera calibration information associated with each sub-frame in order to determine within each sub-frame which spatial pixel areas pertain to the performance area verses the spectator areas, and where the segmenting algorithm uses the determined spatial pixel areas in order to at least distinguish (i) foreground pixels including event objects from (ii) moving background including spectators.

80. The system of claim 1 where one or more of the cameras for recording the event are synchronized with event venue lighting, the system further comprising:

an electronic device for receiving a power waveform used to drive event venue lighting, for converting the waveform into synchronized digital trigger signals, and for providing the digital trigger signals to the one or more cameras; and

an external trigger included on each of the one or more cameras to be synchronized, where the external trigger receives the digital trigger signals for controlling the camera'"'"'s video frame capture time, and where the camera captures video frames timed in response to the signals.

81. The system of claim 1 where the content assembly system further converts the frame rate of the received video data, and where the frame rate of any one or more of the video frame streams forming the multi-camera video data is different from the desired frame rate of any one or more of the created blended video, and where the content assembly system further alters any one of, or any combination of the total number of frames or the frame time information of the received and different video frame stream in order to create the blended video with the desired frame rate.

82. The system of claim 81, where the frame rate of the video frame stream to be altered is either an asynchronous or synchronous rate that does not match the desired frame rate of the blended video, and where the desired frame rate of the blended video is a broadcasting standard rate.

83. The system of claim 82, where the desired frame rate of the blended video is the NTSC broadcasting standard rate.

84. The system of claim 1 where the content assembly system additionally blends either natural or synthesized ambient audio data with the blended content, and where:

the live-content recording apparatus further includes one or more microphones for recording natural ambient audio data throughout the entire duration of the event, where the natural ambient audio recordings are either of the event activities or event spectators, where each of the one or more microphones outputs a distinct ambient audio stream as a multiple data source of event live-content, and where the one or more streams fonn natural ambient audio data; and

the blending algorithm is adapted to receive the natural ambient audio data and perform any one of, or any combination of the following operations;

(a) blending any of the natural ambient audio data with the blended content;

(b) detecting audio characteristics including either the decibel or pitch levels within any natural ambient audio stream for translation into a volume and tonal map and blending the volume and tonal map with the blended content;

or (c) detecting audio characteristics including either the decibel or pitch levels within any natural ambient audio stream for translation into a volume and tonal map and converting the volume and tonal map into a new synthesized ambient audio stream as a multiple data source of event live-content for blending with the blended content.

85. The system of claim 1 where the event activity measured by the tracking system about any one or more of the event objects includes any one of, or any combination of the event object'"'"'s type, identity, location, orientation, size, or color.

86. The system of claim 1 where the tracking system for measuring in real-time the event activity further includes:

one or more electronic devices that are attached to one or more of the event objects to be tracked, where each electronic device is adapted to output electronic signals, where

the tracking system measures in real-time the event activity, and is adapted to receive and analyze the electronic signals output from the one or more electronic devices in order to determine activity data about the associated event object to which the electronic device is attached, including either the object'"'"'s location or identity for inclusion in the tracking data.

87. The system of claim 1, where the event is a sporting event, and where scoreboard data are used to affect the blending algorithm for creating blended video, the system further comprising:

a scoring interface system for accepting or determining scoreboard data, where scoreboard data include any of official scoring information required by the type of sport being played or information that would be displayed on a scoreboard associated with the event, and where the scoring interface system outputs the scoreboard data as a multiple data source of event live-content, where

the content assembly system is adapted to receive scoreboard data, and where the blending algorithm for selecting which of the two or more video frame streams from the multi-camera video data to include in any of the one or more blended video streams at least in part uses any one of, or any combination of the tracking data or scoreboard data.

88. The system of claim 87 where the content assembly system additionally blends other non-live-content with the blended content, the system further comprising:

non-live-content, where the non-live-content includes any one of, or any combination of;

(a) event replay content comprising one or more portions of prior-captured live-content of or relating to the event, as prior captured by any of the live-content recording apparatus, or (b) other content that was not captured by any of the live-content recording apparatus including advertising content, where advertising content includes any of video, audio, images, graphics, text or any other electronic representations of information,

where the blending algorithm is adapted to receive any of the non-live-content and blend any of the non-live-content with the blended content, where the blending algorithm performs any one of, or any combination of;

(a) selecting which non-live-content to insert into, or overlay onto, the blended content, or (b) selecting the time at which non-live-content is inserted into, or overlaid onto, the blended content using at least in part the tracking data or scoreboard data.

89. The system of claim 87 where at least one electronically adjustable calibrated camera is adjusted using any combination of the tracking data or scoreboard data, the system further comprising:

at least one electronically adjustable calibrated camera for recording the event, where the electronically adjustable calibrated camera is fixed and movable with respect to the event venue, where the fixed X, Y, Z location of the calibrated camera with respect to the event venue is known to the system, and where any one of, or any combination of the movable pan, tilt, or zoom settings of the calibrated camera with respect to the event venue are electronically adjustable in response to control signals; and

a filming system for controlling the live-content recording apparatus, where the filming system adjusts any one of, or any combination of the movable pan, tilt, or zoom settings of the electronic adjustable calibrated camera using at least the combination of;

(a) the known fixed X, Y, Z location of the calibrated camera with respect to the venue;

(b) the current combination of the pan, tilt, or zoom settings of the calibrated camera with respect to the venue; and

(c) any one of, or any combination of the tracking data or scoreboard data, and where the filming system further transmits one or more control signals comprising the one or more adjustments to the electronically adjustable calibrated camera during the event.

90. The system of claim 87 where the scoring interface system uses one or more electronic devices operated by a person for accepting data from, or transmitting data to, the scoring interface system, and where the electronic devices include any of wireless clickers, scorekeeping devices, scoreboard control devices, or game officiating devices including whistles that detect usage.

91. The system of claim 87, where the event is a sporting event that uses a game clock, and where timing information related to the starting or stopping of the game clock is used to affect the blending of the multi-camera video data, where:

the scoreboard data include timing information associated with the starting or stopping of the game clock, and where the scoring interface system determines the timing information; and

the blending algorithm is adapted to receive scoreboard data including the timing information associated with the starting or stopping of the game clock, where the content assembly system creates one or more blended video streams from the multi-camera video data, and where the blending algorithm selects which of the two or more video frame streams from the multi-camera video data to include in any of the one or more blended video streams using at least in part the timing information associated with the starting or stopping of the game clock.

92. The system of claim 91 where the timing information related to the starting and stopping of the game clock is generated by game clock control devices, the system further comprising:

one or more game clock control devices for generating timing information indicative of the time for starting or stopping of the game clock,

where the scoring interface system accepts or determines scoreboard data, and is adapted to receive from the one or more game clock devices the timing information indicating the time to start or to stop the game clock, where the scoring interface system then provides signals directing the game clock to start or stop, respectively.

93. The system of claim 92 where the sporting event is an ice hockey game, where the game clock control device for generating the signal commanding a game clock to start is a band worn on the hand of a referee, where the band is capable of sensing the instant that a puck is released from the referee'"'"'s grip, and where the band generates and transmits a signal to the scoring interface system indicating that the game clock should be started.

94. The system of claim 92 where the sporting event is a game in which a referee uses a whistle to indicate the time at which the running state of the game clock is to be changed, where the game clock control device for generating the signal commanding a change in the running state of the game clock is a whistle blown by a referee, where the whistle is capable of sensing the instant that the whistle is blown by the referee, and where the whistle generates and transmits a signal to the scoring interface system indicating that the running state of the game clock should be changed.

95. The system of claim 92 where the sporting event is a game in which the real-time event activity is indicative of the time at which the running state of the game clock should be changed, where the game clock control device for generating the signal commanding a change in the running state of the game clock is the tracking system, where the tracking system is capable of sensing the instant that the event activity indicates that the running state of the game clock should be changed, and where the tracking system generates and transmits a signal to the scoring interface system indicating that the state of the game clock should be changed.

96. The system of claim 1 further adapted to create performance descriptor tokens describing the event activities based upon tracking data, the system further comprising:

a performance measurement and analysis system for receiving and analyzing any of the tracking data, where the performance measurement and analysis system creates a stream of one or more performance descriptor tokens during the event, where the tokens encode information that describes a particular event activity or action based upon any one of, or any combination of the tracking data, and where the performance measurement and analysis system outputs the stream of performance descriptor tokens as a multiple data source of event live-content.

97. The system of claim 96 where the event is a sporting event, the system is adapted to create performance descriptor tokens describing the event activities based upon tracking data and scoreboard data and further comprises:

a scoring interface system for accepting or determining scoreboard data, where scoreboard data include any official scoring information required by the type of sport being played or information that would be displayed on a scoreboard associated with the event, and where the scoring interface system outputs the scoreboard data as a multiple data source of event live-content,

where the performance measurement and analysis system is adapted to receive and analyze any of the scoreboard data, where the performance measurement and analysis system creates one or more performance descriptor tokens, where the tokens encode information that describes a particular event activity or action based upon any one of, or any combination of the tracking data or scoreboard data.

98. The system of claim 96 further adapted to create synthesized commentary audio data describing the event activities, the system further comprising:

a translation software module for creating synthesized commentary audio data, where the translation software module receives and translates into synthesized commentary audio data of the event any one or more of the performance descriptor tokens, where the translation process creates the synthesized commentary audio data from any combination of either pre-recorded speech or text-to-speech synthesis, and the translation software module outputs the synthesized commentary audio data as a multiple data source of event live-content.

99. The system of claim 98 further adapted to create customized synthesized commentary audio data, the system further comprising:

a database of viewer profile information known to the system, where

the translation software module is adapted to receive the database of viewer profile information, where the translation process accesses the viewer profile information for one or more directives to either;

(a) customize the selection of tokens for translation into synthesized commentary audio data, or (b) customize the synthesized commentary audio characteristics including the choice of synthesized voice, expression style, or level of detail used for the audio commentary.

100. The system of claim 98 where the content assembly system additionally blends either performance descriptor tokens or synthesized or live commentary audio data with the blended content, and where:

the live-content recording apparatus includes one or more microphones for recording live commentary audio data during the event, where the live commentary audio recordings are of one or more live commentators, and where each of the one or more microphones outputs a distinct live commentary audio stream as a multiple data source of event live-content, and where the one or more streams form the live commentary audio data; and

the blending algorithm is adapted to receive any one of, or any combination of the performance descriptor tokens, synthesized commentary audio data, or live commentary audio data, and perform any one of, or any combination of the following operations;

(a) blending any of the performance descriptor tokens with the blended content;

(b) blending any of the synthesized commentary audio data with the blended content;

or (c) blending any of the live commentary audio data with the blended content.

101. The system of claim 100 where the content assembly system additionally blends live commentary video data with the blended content, and where:

the live-content recording apparatus includes one or more cameras for recording live commentary video data during the event, where the live commentary video recordings are of one or more live commentators, and where each of the one or more cameras outputs a distinct live commentary video stream as a multiple data source of event live-content, and where the one or more streams form the live commentary video data; and

the blending algorithm is adapted to receive the live commentary video data and blend any of the live commentary video data with the blended content.

102. The system of claim 1, further adapted to analyze event object movement to create event performance measurements and statistics forming a stream of event metrics, the system further comprising:

a performance measurement and analysis system for receiving and analyzing any of the tracking data, where the performance measurement and analysis system creates a stream of event metrics during the event comprising one or more performance measurements or statistics, where the measurements or statistics describe characteristics of one or more event activities or actions based upon any one of, or any combination of the tracking data, and where the performance measurement and analysis system outputs the stream of event metrics as a multiple data source of event live-content.

103. The system of claim 102, where the event is a sporting event, the system adapted to analyze event object movement or scoreboard data to create event performance measurements and statistics forming a stream of event metrics and further comprising:

a scoring interface system for accepting or determining scoreboard data, where scoreboard data include any official scoring information required by the type of sport being played or information that would be displayed on a scoreboard associated with the event, and where the scoring interface system outputs the scoreboard data as a multiple data source of event live-content,

where the performance measurement and analysis system is adapted to receive and analyze any of the scoreboard data, where the performance measurement and analysis system creates a stream of event metrics during the event comprising one or more performance measurements or statistics, where the measurements or statistics describe characteristics of one or more event activities or actions based upon any one of, or any combination of the tracking data or scoreboard data.

104. The system of claim 1, where all event live-content is stored in one or more databases synchronized by time, the system further comprising:

a synchronizing algorithm for receiving and synchronizing all event live-content received, captured, or created by the system, where all event live-content is;

(a) associated with a common time as it is received, captured, or created, and (b) stored in one or more databases along with the associated common time.

105. The system of claim 104, where the start and stop times of performance activities are determined in order to create distinct event-segments that are synchronized with the event live-content, the system further comprising:

a performance measurement and analysis system for receiving and analyzing any of the tracking data, where the performance measurement and analysis system determines and creates a stream of one or more event segments during the event, where the event segments describe at least the start and stop times within the event of a particular event activity or action based upon any one of, or any combination of the tracking data, and where the performance measurement and analysis system outputs the stream of event segments as a multiple data source of event live-content,

where the synchronizing algorithm is adapted to receive the stream of one or more event segments, where all received event-segments are synchronized with the live-content based upon the common start time and common stop time of each event segment, such that the stream of event segments forms an index into any of the event-live content.

106. The system of claim 105, where the event is a sporting event, the system adapted to analyze scoreboard data in order to create distinct event segments that are synchronized with the event live-content and further comprising:

a scoring interface system for accepting or determining scoreboard data, where scoreboard data include any official scoring information required by the type of sport being played or information that would be displayed on a scoreboard associated with the event, and where the scoring interface system outputs the scoreboard data as a multiple data source of event live-content,

where the performance measurement and analysis system is adapted to receive the scoreboard data, where the performance measurement and analysis system determines and creates a stream of one or more event segments during the event, and where the event segments describe at least the start and stop times within the event of a particular event activity or action based upon any one of, or any combination of the tracking data or scoreboard data.

107. The system of claim 105, where any of the two or more system cameras capture video frames at a high frame rate, and where the additional frames are either stored in the synchronized database or discarded based upon the event-segments,

where the live-content recording apparatus includes two or more cameras for recording video data throughout the entire duration of the event, where one or more of the cameras is a high-speed camera capturing video at a high-frame rate, where the high-frame rate is at least more than the normal frame rate of the blended video to be created from the multi-camera video data, and

where one or more high-frame rate video streams are present within the multi-camera video data, where the content assembly and compression system receives the stream of event-segment data, and where for each high-frame rate video stream selected for blending into any of the one or more of the blended video streams the blending algorithm further determines the number of video frames to be included in the blended video streams based at least in part on the event-segment data, such that the frame rate of any of the blended video streams is selectively transformed from a normal frame rate to a higher frame rate based upon the event-segments.

108. The system of claim 105, where any of the two or more system cameras capture video frames at a high frame rate, and where the additional frames are selectively blended into the blended video based upon the event segments,

where the live-content recording apparatus includes two or more cameras for recording video data throughout the entire duration of the event, where one or more of the cameras is a high-speed camera capturing video at a high-frame rate, where the high-frame rate is at least more than the normal frame rate of the blended video to be created from the multi-camera video data, and

where one or more high-frame rate video streams are present within the multi-camera video data, where the content assembly and compression system receives the stream of event-segment data, and where for each high-frame rate video stream selected for blending into any of the one or more of the blended video streams the blending algorithm further determines the number of video frames to be included in the blended video streams based at least in part on the event-segment data, such that the frame rate of any of the blended video streams is selectively transformed from a normal frame rate to a higher frame rate based upon the event-segments.

109. A system for processing content associated with an event, the system comprising:

at least one processor for processing detected data obtained in accordance with the event, where the detected data at least include (i) multi-camera video data as a multiple data source of event live-content, where the multi-camera video data include two or more original distinct streams of temporally sequential video frames captured from two or more cameras throughout the entire duration of the event, where the video data are either of the event or the event spectators, and (ii) real-time tracking data of the event activities as a multiple data source of event live-content, where the activities relate to one or more event objects, where event objects are either objects or one or more people associated with the event, where people are either participants in the event or spectators, and

the at least one processor additionally processes a blending algorithm for assembling new blended content in real time from any of the multiple data sources of event live-content, where the new blended content at least includes one or more new streams of video frames, where each new stream is unique in its composition of video frames with respect to any and all of the original video streams, where each new stream contains at least two video frames taken from different original streams, and where the blending algorithm receives multi-camera video data and tracking data and selects each of the at least two video frames taken from different original streams at least in part using the tracking data,

thereby creating a broadcast with undetectable delay to the human viewer of the on-going event.

110. A method for assembling blended content associated with an event, comprising the steps of:

recording video data with two or more cameras throughout the entire duration of the event, where the video recordings are either of the event activities or event spectators, where each of form multi-camera video data;

tracking the event in order to measure in real-time the event activity of one or more event objects throughout the entire duration of the event, where event objects are either objects or one or more people associated with the event, where people are either participants in the event or spectators;

outputting the tracking data as a multiple data source of event live-content; and

assembling new blended content in real time from the multiple data sources of event live-content using a blending algorithm, where the new blended content at least includes one or more new streams of video frames, where each new stream is unique in its composition of video frames with respect to any and all of the original video streams where each new stream contains at least two video frames taken from different original streams, and where the blending algorithm receives multi-camera video data and tracking data and selects each of the at least two video frames taken from different original streams at least in part using the tracking data,

thereby creating a broadcast with undetectable delay to the human viewer of the on-going event.