High quality multimedia transmission from a mobile device for live and on-demand viewing

• US 8,875,208 B1
• Filed: 11/21/2008
• Issued: 10/28/2014
• Est. Priority Date: 11/21/2007
• Status: Active Grant

• Current Assignee: Skype Ireland Technologies Holdings Limited (Microsoft Corporation)
• Original Assignee: Skype S.Ã R.L. (Microsoft Corporation)
• Inventors: Abkairov, Nikolay, Timanovsky, Alexey
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US12/276,171
• Time in Patent Office: 2,167 Days
• Field of Search: 375/240, 37524001-24007, 375/240.1, 375/240.11, 37524012-24024, 341 50- 51, 370/229, 370/232, 370/235, 370/236, 348/192, 348/400.1, 348/401.1, 348/402.1, 348/409.1, 348/410.1, 348/413.1, 348/416.1, 709233-235, 725/105, 725/115
• US Class Current: 725/115
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/440227 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/44029 for generating different versions
• CPC Class Codes: H04N 21/44245 Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Techniques are provided for transmitting high quality multimedia data from a mobile device for live and on-demand viewing. The mobile device adjusts the quality of a single multimedia data stream based on one or more factors, such as the available bandwidth of a wireless link with a server, whether there are any “live” subscribers of the multimedia data stream, whether any “live” subscribers are interacting with the mobile device, and a change in the privacy settings of the user of the mobile device. Any data that was removed from the multimedia data stream before it was transmitted to a server is subsequently sent to the server. The server generates multiple versions of the multimedia data stream that vary in terms of quality. The server selects which version of the multimedia data stream to transmit to a subscriber of the multimedia data stream based on one or more criteria.

Claims

1. A machine-implemented method comprising:

capturing a multimedia data stream on a device;

while the device is in a real-time streaming mode;

performing one or more operations on a first portion of the multimedia data stream to remove data from the first portion;

tracking the removed data from the first portion at the device;

sending, to a server via a network, data from the first portion that was not removed by performing the one or more operations to enable real-time streaming of a first version of the multimedia data stream for live viewing at one or more subscriber devices; and

sending, to the server via the network, the removed data from the first portion after said sending, to the server via the network, the data from the first portion that was not removed, the sending the removed data effective to cause the server to generate and store a second version of the multimedia data stream for on-demand viewing by combining the data from the first portion that was not removed and the removed data, a quality of the second version of the multimedia data stream for on-demand viewing higher relative to the first version of the multimedia data stream for live viewing.

2. The method of claim 1, wherein the sending the data from the first portion that has been removed as a result of performing the one or more operations is performed while the device is streaming a second portion of the multimedia data to the server in a real-time manner.

3. The method of claim 1, further comprising before performing the one or more operations:

determining that no subscriber has subscribed for the live viewing of the multimedia data stream;

in response to determining that no subscriber has subscribed for the live viewing of the multimedia data stream, processing a particular portion of the multimedia data stream without regard to a delay between (a) capturing the particular portion of the multimedia data stream at the device and (b) transmitting the particular portion of the multimedia data stream to the server;

determining that a subscriber has subscribed for the live viewing of the multimedia data stream;

in response to determining that a subscriber has subscribed for the live viewing of the multimedia data stream, processing a second portion of the multimedia data stream with regard to an additional delay between (c) capturing the second portion of the multimedia data stream at the device and (d) transmitting the second portion of the multimedia data stream to the server.

4. The method of claim 1, further comprising before performing the one or more operations:

while no subscribers of the live viewing of the multimedia data stream are sending data to the device, processing a particular portion of the multimedia data stream without regard to a delay between (a) capturing the particular portion of the multimedia data stream at the device and (b) transmitting the particular portion of the multimedia data stream to the server;

determining that one or more subscribers of the live viewing of the multimedia data stream send data to the device;

in response to determining that one or more subscribers of the live viewing of the multimedia data stream send data to the device, processing a second portion of the multimedia data stream with regard to an additional delay between (c) capturing the second portion of the multimedia data stream at the device and (d) transmitting the second portion of the multimedia data stream to the server.

5. The method of claim 1, further comprising:

encoding the multimedia data stream to generate an encoded multimedia data;

determining whether a volatile medium on the device has enough storage space to store the encoded multimedia data;

in response to determining that the volatile medium has enough storage space;

storing the encoded multimedia data in the volatile medium; and

storing the encoded multimedia data in a non-volatile medium on the device;

determining that a portion of the encoded multimedia data was deleted from the volatile medium;

in response to determining that the portion of the encoded multimedia data was deleted from the volatile medium;

reading the portion of the encoded multimedia data from the non-volatile medium; and

sending the portion of the encoded multimedia data to the server via the network.

6. The method of claim 1, wherein prior to performing the one or more operations, the method further comprising:

determining whether the multimedia data stream can be streamed in real-time at a highest quality; and

responsive to determining that the multimedia data stream cannot be streamed in real-time at the highest quality, determining to remove the data from the first portion to enable the real-time streaming of the first version of the multimedia data stream for live viewing.

7. The method of claim 6, wherein the determining whether the multimedia data stream can be streamed in real-time at a highest quality further comprises determining whether delays in the network will prevent the real-time streaming of the multimedia data stream at the highest quality for live viewing.

8. The method of claim 1, wherein the second version of the multimedia data stream for the on-demand viewing is generated and stored for non-live viewing.

9. The method of claim 1, wherein the sending the removed data is further effective to cause the server to generate the second version of the multimedia data stream by enhancing the first version of the multimedia data stream for the on-demand viewing.

10. A machine-readable storage device having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform a method comprising:

capturing a multimedia data stream on a device;

while the device is in a real-time streaming mode;

performing one or more operations on a first portion of the multimedia data stream to remove data from the first portion;

tracking the removed data from the first portion;

sending, to a server via a network, data from the first portion that was not removed by performing the one or more operations to enable real-time streaming of a first version of the multimedia data stream for live viewing at one or more subscriber devices; and

sending, to the server via the network, the removed data from the first portion after said sending, to the server via the network, the data from the first portion that was not removed, the sending the removed data effective to cause the server to generate and store a second version of the multimedia data stream for on-demand viewing by combining the data from the first portion that was not removed and the removed data, a quality of the second version of the multimedia data stream for on-demand viewing higher relative to the first version of the multimedia data stream for live viewing.

11. The machine-readable storage device of claim 10, wherein the sending the data from the first portion that has been removed as a result of performing the one or more operations is performed while the device is streaming a second portion of the multimedia data to the server in a real-time manner.

12. The machine-readable storage device of claim 10, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform a method further comprising, before performing the one or more operations:

determining that no subscriber has subscribed for the live viewing of the multimedia data stream;

in response to determining that no subscriber has subscribed for the live viewing of the multimedia data stream, processing a particular portion of the multimedia data stream without regard to a delay between (a) capturing the particular portion of the multimedia data stream at the device and (b) transmitting the particular portion of the multimedia data stream to the server;

determining that a subscriber has subscribed for the live viewing of the multimedia data stream;

in response to determining that a subscriber has subscribed for the live viewing of the multimedia data stream, processing a second portion of the multimedia data stream with regard to an additional delay between (c) capturing the second portion of the multimedia data stream at the device and (d) transmitting the second portion of the multimedia data stream to the server.

13. The machine-readable storage device of claim 10, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform a method further comprising, before performing the one or more operations:

while no subscribers of the live viewing of the multimedia data stream are sending data to the device, processing a particular portion of the multimedia data stream without regard to a delay between (a) capturing the particular portion of the multimedia data stream at the device and (b) transmitting the particular portion of the multimedia data stream to the server;

determining that one or more subscribers of the live viewing of the multimedia data stream send data to the device;

in response to determining that one or more subscribers of the live viewing of the multimedia data stream send data to the device, processing a second portion of the multimedia data stream with regard to an additional delay between (c) capturing the second portion of the multimedia data stream at the device and (d) transmitting the second portion of the multimedia data stream to the server.

14. The machine-readable storage device of claim 10, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform a method further comprising:

encoding the multimedia data stream to generate an encoded multimedia data;

determining whether a volatile medium on the device has enough storage space to store the encoded multimedia data;

in response to determining that the volatile medium has enough storage space;

storing the encoded multimedia data in the volatile medium; and

storing the encoded multimedia data in a non-volatile medium on the device;

determining that a portion of the encoded multimedia data was deleted from the volatile medium;

in response to determining that the portion of the encoded multimedia data was deleted from the volatile medium;

reading the portion of the encoded multimedia data from the non-volatile medium; and

sending the portion of the encoded multimedia data to the server via the network.

15. The machine-readable storage device of claim 10, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform a method further comprising, prior to performing the one or more operations:

determining whether the multimedia data stream can be streamed in real-time at a highest quality; and

responsive to determining that the multimedia data stream cannot be streamed in real-time at the highest quality, determining to remove the data from the first portion to enable the real-time streaming of the first version of the multimedia data stream for live viewing.

16. The machine-readable storage device of claim 15, wherein the determining whether the multimedia data stream can be streamed in real-time at a highest quality further comprises determining whether delays in the network will prevent the real-time streaming of the multimedia data stream at the highest quality for live viewing.

17. The machine-readable storage device of claim 10, wherein the second version of the multimedia data stream for the on-demand viewing is generated and stored for non-live viewing.

18. The machine-readable storage device of claim 10, wherein the sending the removed data is further effective to cause the server to generate the second version of the multimedia data stream by enhancing the first version of the multimedia data stream for the on-demand viewing.

19. A method comprising:

receiving, at a server, a multimedia data stream from a device that is in a real-time streaming mode, the multimedia data stream having a portion of data removed by the device;

streaming in real-time a live version of the multimedia data stream to one or more subscriber devices for live viewing;

after said receiving, at the server, the multimedia data stream having the portion of data removed by the device, receiving, at the server, the portion of the data that was removed from the multimedia data stream; and

combining the portion of the data that was removed with the multimedia data stream to generate an on-demand version of the multimedia data stream and storing the on-demand version of the multimedia data stream at the server for on-demand viewing by one or more subscriber devices, the on-demand version of the multimedia data stream having a higher quality relative to the live version of the multimedia data stream.

20. The method of claim 19, wherein the on-demand version of the multimedia data stream is generated and stored for non-live viewing.