High quality multimedia transmission from a mobile device for live and on-demand viewing

• US 10,027,927 B2
• Filed: 07/13/2016
• Issued: 07/17/2018
• Est. Priority Date: 11/21/2007
• Status: Active Grant

• Current Assignee: Skype Ireland Technologies Holdings Limited (Microsoft Corporation)
• Original Assignee: Skype Ireland Technologies Holdings Limited (Microsoft Corporation)
• Inventors: Abkairov, Nikolay, Timanovsky, Alexey
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US15/209,589
• Publication Number:
  US 20160323547A1
• Time in Patent Office: 734 Days
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/440227 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/44029 for generating different versions
• CPC Class Codes: H04N 21/44245 Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Techniques are provided for transmitting high quality multimedia data from a mobile device for live and on-demand viewing. The mobile device adjusts the quality of a single multimedia data stream based on one or more factors, such as the available bandwidth of a wireless link with a server, whether there are any “live” subscribers of the multimedia data stream, whether any “live” subscribers are interacting with the mobile device, and a change in the privacy settings of the user of the mobile device. Any data that was removed from the multimedia data stream before it was transmitted to a server is subsequently sent to the server. The server generates multiple versions of the multimedia data stream that vary in terms of quality. The server selects which version of the multimedia data stream to transmit to a subscriber of the multimedia data stream based on one or more criteria.

Claims

1. A method comprising:

receiving, at a server, a first portion of a multimedia data stream from a device that is in a real-time streaming mode, the received first portion of the multimedia data stream including first sequencing information, the first portion missing a second portion of the multimedia data stream that was removed by the device;

streaming, by the server, in real-time a live version of the first portion of the multimedia data stream to one or more other devices for live viewing;

receiving, at the server, from the device, the second portion of the multimedia data stream, the portion having second sequencing information;

combining, at the server, the first and second portions of the multimedia data stream based on the first and second sequencing information to generate a different version of the multimedia data stream; and

causing the different version of the multimedia data stream to be stored for non-live viewing.

2. The method of claim 1, wherein the different version of the multimedia data stream comprises an on-demand version of the multimedia data stream.

3. The method of claim 1, wherein the different version of the multimedia data stream has a higher quality relative to the live version of the multimedia data stream.

4. The method of claim 1, wherein the second portion that was removed from the multimedia data stream is received after receiving the multimedia data stream from the device that is in the real-time streaming mode.

5. The method of claim 1, wherein the streaming in real-time the live version of the multimedia data stream is performed in response to determining that the one or more devices have subscribed to a live viewing of the multimedia data stream.

6. The method of claim 1, wherein the portion of the data that was removed from the multimedia data stream is received during the streaming of the live version of the multimedia data stream.

7. The method of claim 1, wherein the portion of data removed by the device comprises key frames and non-key frames of the multimedia data stream.

8. The method of claim 1, wherein the portion of data removed by the device comprises portions of frames removed by the device as a result of bit rate scaling.

10. The method of claim 1, further comprising:

receiving, from a client device, a request for the different version of the multimedia data stream, wherein the request is received during the streaming of the live version of the multimedia data stream, and the different version of the multimedia data stream is streamed to the client device before streaming the live version has ended.

11. The method of claim 1, wherein the portion of the data is received at the server from the device with a delay relative to the streaming, by the server, of a corresponding portion of the live version of the multimedia data stream to the one or more devices for live viewing.

12. The method of claim 1, wherein the first sequencing information includes sequence numbers for frames included in the multimedia data stream and the second sequencing information include sequence numbers for frames included in the portion, and the combining is based on the first and second sequence numbers.

13. The method of claim 1, further comprising keeping track, by the server, of missing sequence numbers in the first sequencing information, and requesting, from the device, the portion based on one or more of the tracked missing sequence numbers.

9. A system comprising:

a storage device; and

at least a memory and one or more hardware processors to perform operations comprising;

receiving, by a server, a first portion of a multimedia data stream from a device that is in a real-time streaming mode, the received first portion of the multimedia data stream including first sequencing information, the first portion missing a second portion of the multimedia data stream that was removed by the device;

streaming, by the server, in real-time a live version of the first portion of the multimedia data stream to one or more other devices for live viewing;

receiving, at the server and from the device, the second portion of the multimedia data stream, the second portion including second sequencing information;

generating, by the server, a different version of the multimedia data stream y combining the first and second portions of the multimedia data stream based on the first and second sequencing information; and

storing the different version of the multimedia data stream in the storage device for non-live viewing.

14. The system of claim 9, wherein the portion of the data that was removed from the multimedia data stream is received during the streaming of the live version of the multimedia data stream.

15. The system of claim 9, wherein the different version of the multimedia data stream comprises an on-demand version of the multimedia data stream.

16. The system of claim 15, wherein the storing comprises storing the on-demand version of the multimedia data stream in the storage device for viewing by one or more subscriber devices.

17. The system of claim 9, wherein the different version of the multimedia data stream has a higher quality relative to the live version of the multimedia data stream.

18. The system of claim 9, wherein the second portion that was removed from the multimedia data stream is received after receiving the multimedia data stream from the device that is in the real-time streaming mode.

19. The system of claim 9, wherein the streaming in real-time the live version of the multimedia data stream is performed in response to determining that the one or more devices have subscribed to a live viewing of the multimedia data stream.

20. The system of claim 9, wherein the portion of data removed by the device comprises key frames and non-key frames of the multimedia data stream.

21. The system of claim 9, wherein the portion of data removed by the device comprises portions of frames removed by the device as a result of bit rate scaling.

22. The system of claim 9, wherein the operations further comprise:

receiving, from a client device, a request for the different version of the multimedia data stream; and

streaming the different version of the multimedia data stream to the client device.

23. The system of claim 22, wherein the request is received after the streaming of the live version of the multimedia data stream has ended.

24. The system of claim 22, wherein the request is received during the streaming of the live version of the multimedia data stream, and the different version of the multimedia data stream is streamed to the client device before streaming the live version has ended.

25. One or more machine-readable storage memories having instructions stored thereon that, responsive to execution by one or more hardware processors, cause the one or more hardware processors to perform operations comprising:

receiving, at a server, a first portion of a multimedia data stream from a device that is in a real-time streaming mode, the received first portion of the multimedia data stream including first sequencing information, the first portion missing a second portion of the multimedia data stream that was removed by the device;

streaming, by the server, in real-time a live version of the first portion of the multimedia data stream to one or more other devices for live viewing;

receiving, at the server and from the device, the second portion of the multimedia data stream, the second portion including second sequencing information;

combining, at the server, the first and second portions of the multimedia data stream based on the first and second sequencing information to generate a different version of the multimedia data stream; and

causing the different version of the multimedia data stream to be stored for non-live viewing.

26. The one or more machine-readable storage memories of claim 25, wherein the different version of the multimedia data stream comprises an on-demand version of the multimedia data stream.

27. The one or more machine-readable storage memories of claim 25, wherein the different version of the multimedia data stream has a higher quality relative to the live version of the multimedia data stream.

28. The one or more machine-readable storage memories of claim 25, wherein the streaming in real-time the live version of the multimedia data stream is performed in response to determining that the one or more devices have subscribed to a live viewing of the multimedia data stream.

29. The one or more machine-readable storage memories of claim 25, wherein the portion of the data that was removed from the multimedia data stream is received after receiving the multimedia data stream from the device that is in the real-time streaming mode.

30. The one or more machine-readable storage memories of claim 25, wherein the portion of the data that was removed from the multimedia data stream is received during the streaming of the live version of the multimedia data stream.

31. The one or more machine-readable storage memories of claim 25, wherein the portion of data removed by the device comprises key frames and non-key frames of the multimedia data stream.

32. The one or more machine-readable storage memories of claim 25, wherein the portion of data removed by the device comprises portions of frames removed by the device as a result of bit rate scaling.