Dynamic encoding of multiple video image streams to a single video stream based on user input

• US 9,392,303 B2
• Filed: 10/26/2011
• Issued: 07/12/2016
• Est. Priority Date: 10/26/2011
• Status: Expired due to Fees

• Current Assignee: Ronnie Yaron, Tuvia Barak
• Original Assignee: Ronnie Yaron, Tuvia Barak
• Inventors: Yaron, Ronnie, Barak, Tuvia
• Primary Examiner(s): Bengzon, Greg C
• Application Number:
  US13/281,522
• Publication Number:
  US 20130111051A1
• Time in Patent Office: 1,721 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand

Abstract

A system and method for combining multiple video bit streams on a server using low latency encoding and stream them to a user based on user input over the network. Each frame of the resulting single video stream can include anything from a single video window at full size to multiple video windows at a variety of sizes. This allows apparently instantaneous video switching by the user without the buffering start-up delay normally suffered by a user when a new video stream is selected. User browsing can be done by scrolling through smaller scale thumbnail videos and zooming of one or more of the videos. The user can also browse video based on geospatial context.

Claims

1. A system for streaming multiple video images comprising:

a server adapted to service one or more remote users over a network receiving a plurality of input video bit streams;

said server having hardware that includes a plurality of frame decoders, a video frame selector, one or more low latency encoders each having a latency of less than 10 milliseconds, and a combiner, the server configured to present content from at least some of the input video bit streams to regions of a remote user'"'"'s screen display;

the plurality of frame decoders, one assigned to each of said input video bit streams, being adapted to produce sequential output video frames from the input video bit stream at a predetermined frame rate;

the video frame selector, under control of the server, being configured to dynamically select some of said sequential output video frames to produce selected video output frames for combining into a single video stream containing said regions of the remote user'"'"'s screen display for transmission over the network in immediate response to input received from the remote user over the network, wherein said input received from the user can dynamically modify which video input streams are combined into a single output stream for transmission to that particular user and dynamically select a display screen position, zoom level and size for each video input stream combined;

the one or more low latency encoders, each assigned to a different remote user, configured to digitally encode the selected output video frames for that particular user along with metadata information related to chosen input video bit streams including one or more of their content, screen position, size and zoom level into a single video stream for transmission over a network;

the single video stream having the content, screen position, zoom level and size dynamically selected by the particular user over the network for each of the selected video output frames;

the combiner configured to combine outputs from different low latency encoders assigned to different remote users into output streams for streaming over the network to the different remote users;

wherein said metadata includes at least one of location, direction of view, field of view, content type, or artist information.

2. The system of claim 1 wherein said plurality of input video streams originate from different sources.

3. The system of claim 2 wherein said different sources can include mobile devices, security cameras, aerial drones or pre-recorded video.

4. The system of claim 1 wherein said input received from a particular user is received over the network concurrently with transmission of the coded output stream to that user.

5. The system of claim 1 wherein said input received from the particular user can include a requested location, date, time, specific video streams to view, or a size for each input video stream to view.

6. The system of claim 1 wherein there is a plurality of low latency encoders servicing a plurality of remote users.

7. A method of transmitting a plurality of input video streams to a user in a single video output stream under dynamic user control comprising:

receiving at a server a plurality of video input bit streams;

receiving at the server dynamic input from a remote user over a network specifying instantaneous format and content of a desired output video stream, including one or more chosen video streams and a zoom level and size for each chosen video stream;

encoding the output frames into coded output frames for digital streaming along with metadata information related to each of the input frames including their position and resolution into a single video stream for transmission over a network,

combining at the server the coded output frames from each of the selected input video streams at the dynamically selected display zoom level, screen position and size into single output frames and streaming the single output frames to the remote user over the network;

wherein said metadata includes at least one of location, direction of view, field of view, content type, or artist information.

8. The method of claim 7 wherein said plurality of video input streams can include video streams from mobile devices, security cameras, aerial drones or pre-recorded video.

9. The method of claim 7 decoding each of said video input streams into incoming image frames and then combining some of said incoming image frames into outgoing image frames for transmission over the network.

10. The method of claim 7 wherein each output frame of said desired output video stream contains input frames from one or more of said video input streams, each of said input frames in a particular output frame having a content, size, zoom level and screen position in said desired output frame dynamically determined by the input from the remote user.

11. The method of claim 7 wherein said input from the remote user can include a requested location, date, time, specific video streams to view, or a size for each video input stream to view.

12. A method of transmitting a plurality of input video streams to a user in a single video output stream under dynamic user control comprising:

receiving at a server a plurality of video input streams;

receiving at the server dynamic commands from a remote user over a network containing instructions that dynamically specify format and content of a desired output video stream including a choice of video input streams and a zoom level and size for each chosen video input stream;

combining at the server from each of the selected input video streams at the dynamically selected zoom level and size into one single output stream;

encoding output frames along with metadata information related to each of the input frames combined within them including their position, size and zoom level, into a single video stream for transmission over a network;

transmitting said desired output video stream over the network to the user;

wherein said metadata includes at least one of location, direction of view, field of view, content type, or artist information.

13. The method of claim 12 wherein each output frame of said output video stream contains input frames from one or more of said video input streams, each of said input frames in a particular output frame having a size and position in said output frame dynamically determined by the input from the remote user.