METHOD AND SYSTEM FOR LOW COST TELEVISION PRODUCTION

• US 20160104510A1
• Filed: 05/26/2014
• Published: 04/14/2016
• Est. Priority Date: 05/26/2013
• Status: Active Grant

• Current Assignee: Pixellot Ltd.
• Original Assignee: Pixellot Ltd.
• Inventors: TAMIR, Miky, OZ, Gal
• Granted Patent: US 10,438,633 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

It is provided a method for producing a stream of desired image frames derived from a panoramic capturing of a scene by an array of video cameras. The production is conducted by a coordinated operation at a venue proximate to the scene and at a production suite located remotely from the venue. The method includes a venue server recording a stream of raw video images captured simultaneously by the video-cameras, and converting the stream of raw video images at a first data rate to a stream of panoramic views at a second data rate. The second data rate is less than one third of the first data rate. The method further includes transmitting the stream of panoramic views to a remote production suite, and there selecting a stream of desired frames including respective regions of interest. The method further includes transmitting data on the stream of desired frames to the venue server, and the venue server producing out of the recorded stream of raw video images, a stream of high resolution frames. The high resolution frames are characterized by non perceptible boundaries between neighboring images derived from respective neighboring raw video images.

Claims

1. A method for producing, by a coordinated operation conducted at a venue proximate to a scene and at a production suite located remotely from the venue, a stream of desired image frames derived from a panoramic capturing of the scene by an array of video cameras, the method comprising:

(a) a venue server recording a stream of raw video images associated with images captured simultaneously by the video cameras;

(b) converting said stream of raw video images at a first data rate to a stream of panoramic views at a second data rate, said second data rate being less than one third of said first data rate;

(c) transmitting said stream of panoramic views to at least one remote production suite;

(d) in the remote production suite, selecting a stream of desired frames including respective regions of interest, the selected frames being portions of respective panoramic views;

(e) transmitting data on said stream of desired frames to said venue server; and

,

(f) said venue server producing out of the recorded stream of raw video images, a stream of high resolution frames in accordance with said stream of desired frames, said high resolution frames being characterized by non perceptible boundaries between neighboring images derived from respective neighboring raw video images.

2. The method of claim 1, wherein the method includes determining a plurality of preset fixed frames for presenting a respective plurality of fixed regions of interest.

3. The method of claim 1, wherein the method includes selecting a frame from a group of frames consisting of:

(i) a preset fixed frame determined in advance for capturing a fixed region of interest;

(ii) a virtual moving camera following an object in the scene;

(iii) a virtual moving camera performing a pre-defined motion;

(iv) a frame determined by an algorithm for automatic detection and inclusion of a predefined event; and

,

(v) a frame associated with an event which occurred in a certain location in the scene previously.

4. The method of claim 1, wherein the method includes transmitting to the venue server for appending to a desired frame instructions regarding at least one data item selected from a group consisting of:

(a) a graphical item;

(b) a desired artistic transition between frames;

(b) an audio item; and

(c) statistical data regarding identified events occurring in the scene.

5. The method of claim 1, wherein two or more arrays of video cameras are capturing the scene from two or more venue locations, and respective streams of panoramic views are transmitted to one or more production suites.

6. The method of claim 1, wherein at least one moving video camera is used in conjunction with the array of video cameras to capture the event.

7. The method of claim 1, wherein at least one of said transmitting to said production suite and said transmitting to said venue server is done using an Internet link.

8. The method of claim 1, wherein a client module is selecting said stream of desired frames, said client module comprising a displaying interface and a desired frame selector and controller.

9. The method of claim 8, wherein:

(A) a cloud server receives a sequence of panoramic views from said venue server; and

,

(B) said cloud server streams said sequence of panoramic views to said client module using the world wide web.

10. The method of claim 1, wherein the method further includes a step selected from a group of steps consisting of:

(i) preliminary calibrating the video cameras of the array based on mechanical design and lens specifications; and

(ii) accurately calibrating of said video cameras based on said scene to get calibration parameters.

11. The method of claim 1, wherein the method includes steps conducted at the venue server, the steps including:

(i) decomposing frames captured by the video cameras into pixel tiles, said tiles are rectangular arrays of pixels whose size is smaller than a single camera frame size;

(ii) associating said pixel tiles with time codes; and

,

(iii) recording said pixel tiles.

12. The method of claim 11, wherein the method includes at least one action selected from a group of actions consisting of:

(i) retrieving recorded pixel tiles associated with a selected frame;

(ii) transforming stitching lines from world coordinates to camera coordinates and specifying the lines in the selected frame or close thereof;

(iii) geometrically aligning pixels belonging to neighboring video cameras using said calibration parameters;

(iv) blending pixel pairs in stitching zones belonging to neighboring video cameras in accordance with a distance from a stitching line;

(v) correcting color intensities of pixels belonging to different video cameras; and

,

(vi) adjusting hue, saturation, intensity and gamma value of the entire frame.

13. The method of claim 1, wherein the method includes at least one step conducted at the venue server and selected from a group of steps consisting of:

(i) low quality blending and aligning of neighboring frames captured by said array of video cameras;

(ii) generating of low resolution panoramic view from the blended and aligned frames; and

,

(iii) compressing and encoding the panoramic views for transmitting to the remote production suite.

14. The method of claim 1, wherein the method includes at least one step conducted at the remote production suite and selected from a group of steps consisting of:

(i) generating and distributing to operators partial frames of the panoramic view in accordance with respective region of interest;

(ii) said operators moving across and zooming in and out said partial frames to get actual frames;

(iii) presenting to a director said actual frames; and

,

(iv) said director selecting an actual frame to go on air.

15. The method of claim 1, wherein the method includes:

(i) encoding said stream of high resolution frames into a required output video format; and

,

(ii) broadcasting or webcasting the formatted stream of high resolution frames.

16. A method for a production suite located remotely from a venue proximate to a scene, the production suite operating in a coordinated way with a venue server in the venue for producing a stream of desired image frames derived from a panoramic capturing of the scene by an array of video cameras, the production suite method comprising:

(a) receiving from the venue server a stream of low-resolution panoramic views derived from a stream of raw video images captured simultaneously by the video cameras;

(b) selecting a stream of desired frames including respective regions of interest, the selected frames being portions of said low resolution panoramic views; and

,

(c) transmitting data on said stream of desired frames to said venue server.

17. The method of claim 16, wherein the method includes at least one step selected from a group of steps consisting of:

(i) generating and distributing to operators in the production suite partial frames of the panoramic view in accordance with respective region of interest;

(ii) said operators moving across and zooming in and out said partial frames to get actual frames;

(iii) presenting to a director said actual frames; and

,

(iv) said director selecting an actual frame to go on air.

18. A method for a venue server producing, by a coordinated operation with a production suite, a stream of desired image frames derived from a panoramic capturing of a scene by an array of video cameras, the method comprising:

(a) recording a stream of raw video images captured simultaneously by the video cameras;

(b) converting said stream of raw video images to a stream of panoramic views at a reduced data rate, the data rate reduction being by a factor of at least three;

(c) transmitting said stream of panoramic views to the production suite for selecting thereof a stream of desired frames including respective regions of interest, the selected frames being portions of the panoramic views;

(d) receiving from said production suite data on said stream of desired frames; and

,

(e) producing out of the recorded stream of raw video images, a stream of high resolution frames in accordance with said stream of desired frames, said high resolution frames being characterized by non perceptible boundaries between neighboring images derived from respective neighboring raw video images.

19. The method of claim 18, wherein the method includes:

(i) decomposing frames captured by the video cameras into pixel tiles;

(ii) associating said pixel tiles with time codes; and

,

(iii) recording said pixel tiles.

20. The method of claim 19, wherein the method includes at least one action selected from a group of actions consisting of:

(i) retrieving recorded pixel tiles associated with a selected frame;

(ii) transforming stitching lines from world coordinates to camera coordinates and specifying the lines in the selected frame or close thereof;

(iii) geometrically aligning pixels belonging to neighboring video cameras using said calibration parameters;

(iv) blending pixel pairs in stitching zones belonging to neighboring video cameras in accordance with a distance from a stitching line;

(v) correcting color intensities of pixels belonging to different video cameras; and

,

(vi) adjusting hue, saturation, intensity and gamma value of the entire frame.