Method and system for low cost television production

• US 10,438,633 B2
• Filed: 05/26/2014
• Issued: 10/08/2019
• Est. Priority Date: 05/26/2013
• Status: Active Grant

• Current Assignee: Pixellot Ltd.
• Original Assignee: Pixellot Ltd.
• Inventors: Tamir, Miky, Oz, Gal
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Hasan, Syed Y
• Application Number:
  US14/893,077
• Publication Number:
  US 20160104510A1
• Time in Patent Office: 1,961 Days
• Field of Search: 386223, 348 36, 348 42, 348 47, 348 48, 348157, 348159, 348239, 348294, 382154
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

It is provided a method for producing a stream of desired image frames derived from a panoramic capturing of a scene by an array of video cameras. The production is conducted by a coordinated operation at a venue proximate to the scene and at a production suite located remotely from the venue. The method includes a venue server recording a stream of raw video images captured simultaneously by the video-cameras, and converting the stream of raw video images at a first data rate to a stream of panoramic views at a second data rate. The second data rate is less than one third of the first data rate. The method further includes transmitting the stream of panoramic views to a remote production suite, and there selecting a stream of desired frames including respective regions of interest. The method further includes transmitting data on the stream of desired frames to the venue server, and the venue server producing out of the recorded stream of raw video images, a stream of high resolution frames. The high resolution frames are characterized by non perceptible boundaries between neighboring images derived from respective neighboring raw video images.

Claims

1. A method for producing, by a coordinated operation conducted at a venue proximate to a scene and at a production suite located remotely from the venue, a stream of desired image frames derived from a panoramic capturing of the scene by an array of video cameras, the method comprising:

(a) a venue server recording a stream of raw video frames associated with images captured simultaneously by the video cameras, decomposing said raw video frames into pixel tiles, associating said pixel tiles with time codes, and recording said pixel tiles;

(b) converting said stream of raw video frames at a first data rate to a stream of panoramic views at a second data rate, said second data rate being less than one third of said first data rate;

(c) transmitting said stream of panoramic views to at least one remote production suite;

(d) in the remote production suite, selecting a stream of desired frames including respective regions of interest by a plurality of operators, the selected frames being portions of respective panoramic views;

(e) transmitting data on said stream of desired frames to said venue server, retrieving recorded pixel tiles associated with said stream of desired frames, transforming stitching lines from world coordinates to camera coordinates, and specifying lines in the stream of desired frames or close thereof; and

(f) said venue server geometrically aligning pixels belonging to neighboring video cameras using calibration parameters to produce out of the recorded pixel tiles, a stream of high resolution frames being characterized by non perceptible boundaries between raw video images recorded by neighboring video cameras.

2. The method of claim 1 wherein the method includes determining a plurality of preset fixed frames as desired frames for presenting a respective plurality of fixed regions of interest.

3. The method of claim 1 wherein the selecting a stream of desired frames includes selecting a frame from a group of frames consisting of:

(i) a preset fixed frame determined in advance for capturing a fixed region of interest;

(ii) a virtual moving camera following an object in the scene;

(iii) a virtual moving camera performing a pre-defined motion;

(iv) a frame determined by an algorithm for automatic detection and inclusion of a predefined event; and

(v) a frame associated with an event which occurred in a certain location in the scene previously.

4. The method of claim 1, wherein the method includes transmitting to the venue server for appending to a desired frame instructions regarding at least one data item selected from of a group consisting of:

(a) a graphical item;

(b) a desired artistic transition between frames;

(c) an audio item; and

(d) statistical data regarding identified events occurring in the scene.

5. The method of claim 1, wherein two or more arrays of video cameras are capturing the scene from two or more venue locations, and respective streams of panoramic views are transmitted to one or more production suites.

6. The method of claim 1, wherein at least one moving video camera is used in conjunction with the array of video cameras to capture an event.

7. The method of claim 1, wherein at least one of said transmitting to said production suite and said transmitting to said venue server is done using an internet link.

8. The method of claim 1, wherein a client module is selecting said stream of desired frames, said client module comprising a displaying interface and a desired frame selector and controller.

9. The method of claim 8, wherein:

(a) a cloud server receives a sequence of panoramic views from said venue server; and

,

(b) said cloud server streams said sequence of panoramic views to said client module using the world wide web.

10. The method of claim 1, wherein the method includes at least one step conducted at the venue server and selected from a group of steps consisting of:

(i) low quality blending and aligning of frames captured by neighboring video cameras;

(ii) generating of low resolution panoramic view from the blended and aligned frames; and

,

(iii) compressing and encoding the panoramic views for transmitting to the remote production suite.

11. The method of claim 1 wherein the method includes at least one step conducted at the remote production suite and selected from a group of steps consisting of:

(i) generating and distributing to the operators partial frames of the panoramic view in accordance with respective region of interest;

(ii) said operators moving across and zooming in and out said partial frames to get actual frames;

(iii) presenting to a director said actual frames on a split screen; and

(iv) said director selecting an actual frame to go on air.

12. The method of claim 1 wherein the method includes:

(i) encoding said stream of high resolution frames into a required output video format; and

,

(ii) broadcasting or webcasting the formatted stream of high resolution frames.