Multipath video streaming over a wireless network

• US 8,873,560 B2
• Filed: 07/08/2009
• Issued: 10/28/2014
• Est. Priority Date: 07/08/2009
• Status: Active Grant

• Current Assignee: Dejero Labs, Inc.
• Original Assignee: Dejero Labs, Inc.
• Inventors: Frusina, Bogdan, Horvath, Akos
• Primary Examiner(s): Zhu, Alvin
• Application Number:
  US12/499,151
• Publication Number:
  US 20110007693A1
• Time in Patent Office: 1,938 Days
• Field of Search: 370/230, 370/230.1, 370/231, 370/232, 370/233, 370/234, 370/235, 370/236, 370/236.2, 370/237, 370/238
• US Class Current: 370/394
• CPC Class Codes: H04L 5/003 Arrangements for allocating sub-channels of the transmission path
• CPC Class Codes: H04L 5/0053 Allocation of signaling, i.e. of overhead other than pilot signals
• CPC Class Codes: H04L 5/0058 Allocation criteria
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/6131 involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server

Abstract

The method and system as disclosed relates to streaming of large quantities of time critical data over multiple distinct channels from a wireless communications device to a central receiver. More specifically the disclosure deals with the challenges and problems of maintaining consistent data reception quality when faced with the anomalies of a moving sender that is sending data using a relatively unstable radio frequency (RF) method. This is achieved by converting single source data into multiple data streams, placing them in transport buffers and storing them for forwarding. A plurality of radio frequency modules provide wireless connectivity to a plurality of wireless network. Links are maintained to provide feedback on network connections to allow for the transfer of data from one network to another and to adjust the amount of data being transmitted.

Claims

1. A system comprising:

a buffer management and transport controller configured to divide encoded data streams into data portions for buffering into multiple transport buffers;

a plurality of radio frequency modules configured to wirelessly transmit the data portions simultaneously to a plurality of different wireless networks to aggregate throughput across the plurality of different wireless networks, wherein the plurality of radio frequency modules are further configured to provide network feedback to the buffer management and transport controller regarding network conditions for the plurality of different wireless networks;

a data transmission control and reassembly configured to receive the data portions from the plurality of different wireless networks for reassembly and to provide receiver feedback to the buffer management and transport controller regarding additional network conditions for the plurality of different wireless networks;

wherein the buffer management and transport controller is further configured to use the network feedback and the receiver feedback to, for each data portion and prior to transmission of the data portion over at least one of the plurality of different wireless networks, withdraw the data portion from a transport buffer and dynamically select a radio frequency module of the plurality of radio frequency modules for the transmission of the data portion over the at least one of the plurality of different wireless networks;

wherein the buffer management and transport controller is configured to dynamically select a first wireless network for transmission of a first data portion, and a second different wireless network for transmission of a second data portion such that different data portions are transmitted to different wireless networks; and

wherein the buffer management and transport controller is configured to match a buffering rate for the data portions with a withdraw rate for the data portions using the network feedback and the receiver feedback.

2. The system of claim 1, wherein the network feedback and the receiver feedback comprises an error rate for each radio frequency module based on data portions that are not delivered at the destination for re-assembly;

and a capacity condition for each radio frequency module based on at least one of a latency for transmission of the data portions and data portions that are delivered at the destination for re-assembly.

3. The system of claim 1, wherein:

the data portions withdrawn from the buffer management and transport controller are assembled packets of datagrams having respective sequence numbers that are delivered to the dynamically selected radio frequency module; and

the delivered assembled packets from each radio frequency module are re-assembled at the destination using the sequence numbers in the corresponding datagrams.

4. The system of claim 1, further comprising a data source configured to:

capture audio-visual data for the encoded data streams; and

use the network feedback and the receiver feedback to control at least one of the quantity and the quality of the captured audio-visual data.

5. The system of claim 1, wherein the buffer management and transport controller is controlled by a second buffer management and transport controller that is configured to control:

the withdrawal by the buffer management and transport controller of the data portions;

the dynamic selection of the radio frequency modules for the transmission of the data portions; and

the transmission by selected radio frequency modules to simultaneously available different wireless networks.

6. The system of claim 1, further comprising an encoder for encoding data streams, wherein the encoded data streams comprise redundant information to be delivered for increased reliability as the data portions from each radio frequency module for the wireless radio transmission to the plurality of different wireless networks for re-assembly at a destination.

7. The system of claim 1, wherein the network feedback and the receiver feedback is selected from a group consisting of:

latency for transmission of data portions;

transmit collisions at the plurality of different wireless networks;

the number of the radio frequency modules in the wireless radio transmission with each of the plurality of different wireless networks;

throughput requirements of the plurality of radio frequency modules for the wireless radio transmission on each of the plurality of different wireless networks;

coverage drop-off indicators;

time between successful transmission attempts;

status values;

Received Signal Strength Indicator (RSSI);

packet retransmissions;

average packet loss per link;

sender bandwidth;

destination bandwidth;

current available bandwidth; and

combinations thereof.

8. The system of claim 1, wherein:

the withdrawal of encoded data portions from the buffer management and transport controller are assembled packets of datagrams having respective sequence numbers that are delivered to the selected radio frequency module;

the delivered data portions from each radio frequency module are re-assembled at the destination using the sequence numbers in the corresponding datagrams; and

the quantity at which the delivered data portions from each radio frequency modules are being re-assembled at the destination includes;

a rate at which the delivered data portions from each radio frequency modules are being re-assembled at the destination; and

an amount at which the delivered data portions from each radio frequency modules are being re-assembled at the destination.

9. The system of claim 1, wherein when the buffer management and transport controller is intermittently stationary and in motion relative to the plurality of different wireless networks, to use the network feedback and the receiver feedback, to control:

original data streams that are encoded into the buffer management and transport controller;

the withdrawal by the buffer management and transport controller of the data portions;

the dynamic selection of the radio frequency modules for the transmission of the data portions; and

the transmission by selected radio frequency modules to simultaneously available different wireless networks.

10. The system of claim 9, wherein the network feedback and the receiver feedback is selected from a group consisting of:

latency for transmission of data portions;

transmit collisions at the plurality of different wireless networks;

the number of the radio frequency modules in the wireless radio transmission with each of the plurality of different wireless networks;

throughput requirements of the plurality of radio frequency modules for the wireless radio transmission on each of the plurality of different wireless networks;

coverage drop-off indicators;

time between successful transmission attempts;

status values;

Received Signal Strength Indicator (RSSI);

packet retransmissions;

average packet loss per link;

sender bandwidth;

destination bandwidth;

current available bandwidth; and

combinations thereof.

11. The system of claim 1, wherein encoded data streams are generated from a continuous real-time stream of audio-visual data by a wireless device.

12. The system of claim 1 further comprising:

a data source configured to capture a continuous real-time stream of audio-visual data;

a converter configured to convert the captured audio-visual data into original data streams; and

an encoder configured to encode the original data streams into the encoded data streams.

13. The system of claim 12 wherein the buffer management and transport controller is configured to control the encoder based on the network feedback and the receiver feedback.

14. The system of claim 1, wherein two or more the radio frequency modules are further configured to enable said wireless radio transmission of the data portions to one of the plurality of different wireless networks.

15. The system of claim 1, wherein the buffer management and transport controller is further configured, using the network feedback and the receiver feedback, to control delivery rate of data portions to selected radio frequency modules.

16. The system of claim 1, wherein the buffer management and transport controller is further configured, using the network feedback and the receiver feedback, to dynamically and selectively redirect the data portions from radio frequency module to another radio frequency module, and from one the different wireless networks to another of the different wireless networks, whereby data throughput to the destination is maintained.

17. The system of claim 1, wherein each of the radio frequency modules is further configured to monitor and control the wireless radio transmission of the data portions, whereby data throughput to the destination is maintained.