Packet stream arrangement in multimedia transmission

• US 20060109805A1
• Filed: 11/19/2004
• Published: 05/25/2006
• Est. Priority Date: 11/19/2004
• Status: Active Grant

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Hannuksela, Miska, Wenger, Stephan, Malamal Vadakital, Vinod Kumar
• Granted Patent: US 7,751,324 B2
• Time in Patent Office: Days
• US Class Current: 370/299
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4382 Demodulation or channel decoding, e.g. QPSK demodulation analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/6131 involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]

Abstract

Transmitting media information from a transmitting device to a receiving device. To perform the transmission a media stream is encoded and packets are formed from the encoded media stream, the packets containing application data units, the application data units having a decoding order. At least two blocks are generated from the packets. The packets of the at least two blocks are organized into a first group and a second group, the first group being decodable without reception of said second group. The transmission order of the packets of said first group is arranged succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

Claims

1. A system for transmitting media information comprising:

an encoder for encoding at least one media stream;

a packetizer for forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

a packet block generator for generating at least two blocks from the packets;

an organizer for organizing the packets within a block of the at least two blocks into a first group and a second group, the first group being decodable without reception of said second group;

an orderer for arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order; and

a transmitter for transmitting the packets according to the transmission order;

a receiver for receiving the packets;

a buffer for buffering received packets;

a reorderer for reordering the received packets into decoding order;

a depacketizer for reconstructing the encoded media information from the packets; and

a decoder for decoding media information of the packets.

2. A system according to claim 1 wherein said media stream comprises at least a video stream, wherein said encoder is adapted to encode the video stream to form coded pictures, and said organizer is adapted to organize the coded pictures into a first group and a second group, the first group being decodable without reception of said second group.

3. A system according to claim 1 comprising

a transmitting device which comprises said encoder, packetizer, packet block generator, organizer, orderer and transmitter;

a receiving device which comprises said receiver, buffer, reordered, depacketizer and decoder.

4. A system according to claim 1 comprising

an encoding device which comprises said encoder;

a transmitting device which comprises said packetizer, packet block generator, organizer, orderer and transmitter;

a receiving device which comprises said receiver, buffer, reorderer, depacketizer and decoder.

5. A system according to claim 1 wherein said media stream comprises at least a video stream and an audio stream, said encoder comprising a video encoder and an audio encoder;

wherein said video encoder is adapted to encode the video stream to form coded pictures, said audio encoder is adapted to encode the audio stream to form encoded audio, and said organizer is adapted to organize the encoded audio into said first group and the coded pictures into said second group.

6. A system according to claim 1 wherein said reorderer and depacketizer are combined.

7. A transmitting device comprising:

an encoder for encoding at least one media stream;

a packetizer for forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

a packet block generator for generating at least two blocks from the packets;

an organizer for organizing the packets within a block into a first group and a second group, the first group being decodable without reception of said second group; and

an orderer for arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

8. A transmitting device according to claim 7 comprising

an error correction data definer for defining first error correction data for said first group on the basis of the packets of said first group, and a second error correction data for said second group on the basis of the packets of said second group,

a packet selector for selecting a first integral group of packets on the basis of said first group and said first error correction data and for selecting a second integral group on the basis of said second group and said second error correction data wherein said first group of packets can be restored from said first integral group and said second group of packet can be restored from said second integral group,

and wherein the orderer is adapted to order the transmission of said first integral group after the packets of said second integral group are ordered to be transmitted.

9. A transmitting device according to claim 8 wherein the packet selector is adapted to select said first integral group to comprise said first group and said second integral group to comprise said second group.

10. A transmitting device according to claim 8 wherein the encoder is adapted to form reference data units and non-reference data units wherein the organizer is adapted to organize the reference data units into said first group and the non-reference data units into said second group.

11. A transmitting device according to claim 10 wherein the reference data units and non-reference data units form more than one IDR period, wherein the orderer is adapted to order the transmission of packets formed from the IDR periods in decoding order of the IDR periods.

12. A transmitting device according to claim 9 wherein the orderer is adapted to order the packets in at least one of the groups among said first integral group and said second integral group in reverse decoding order.

13. A transmitting device according to claim 7 wherein the encoder is adapted to encode at least two different media types, and the organizer is adapted to organize the packets of one media type into said first group and the packets of another media type into said second group.

14. A transmitting device according to claim 13 wherein said one media type is a video stream comprising reference pictures and non-reference pictures, and the another media type is an audio stream, wherein the organizer is adapted to organize the packets of non-reference pictures and the packets of reference pictures into said first group such that the packets of reference pictures succeed the packets of non-reference pictures.

15. A transmitting device according to claim 7 wherein the packetizer is adapted to select a forward error correction code rate for said first group different from a forward error correction code rate for said second group.

16. A transmitting device comprising:

an encoder for encoding at least one media stream;

a packetizer for forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

a packet block generator for generating at least two blocks from the packets;

an organizer for organizing the packets within a block into a first group and a second group, the first group being decodable without reception of said second group; and

an orderer for arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order,

wherein the encoder is adapted to encode at least two different media types, the media types having been classified according to their importance, wherein the orderer is adapted to order the transmission of packets of the media types in ascending order of importance.

17. A device comprising:

a packetizer for forming packets from at least one encoded media stream, the packets containing application data units, the application data units having a decoding order;

a packet block generator for generating at least two blocks from the packets;

an organizer for organizing the packets within a block into a first group and a second group, the first group being decodable without reception of said second group; and

an orderer for arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

18. A receiving device comprising:

a receiver for receiving packets including at least one encoded media stream and which packets are at least partly in a different order than the decoding order;

a buffer for buffering received packets;

a reorderer for reordering received packets into decoding order;

a depacketizer for reconstructing the encoded media information from the packets; and

a decoder for decoding media information of the packets.

19. A receiving device according to claim 18 further comprising an error correction data decoder producing packets of the encoded media stream on the basis of received packets.

20. A receiving device according to claim 19 comprising a common memory for the error correction data decoder and the reorderer.

21. A wireless communication device comprising:

an encoder for encoding at least one media stream;

a packetizer for forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

a packet block generator for generating at least two blocks from the packets;

an organizer for organizing the packets within a block of the at least two blocks into a first group and a second group, the first group being decodable without reception of said second group; and

an orderer for arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

22. A wireless communication device comprising:

a receiver for receiving packets including at least one encoded media stream and which packets are at least partly in a different order than the decoding order;

a buffer for buffering received packets;

a reorderer for reordering received packets into decoding order;

a depacketizer for reconstructing the encoded media information from the packets; and

a decoder for decoding media information of the packets.

23. A method for transmitting media information from a transmitting device to a receiving device comprising:

encoding at least one media stream;

forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

generating at least two blocks from the packets;

organizing the packets of the at least two blocks into a first group and a second group, the first group being decodable without reception of said second group; and

arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

24. The method according to claim 23 further comprising

transmitting the packets to the receiving device according to the transmission order;

receiving the packets;

buffering received packets;

reordering the received packets into decoding order;

reconstructing the encoded media information from the packets; and

decoding media information of the packets.

25. A signal for carrying encoded media information from a transmitting device to a receiving device comprising packets from at least one encoded media stream, the packets containing application data units, the application data units having a decoding order;

and at least two blocks are generated from the packets, wherein the packets of the signal are ordered by;

organizing packets within a block of the at least two blocks into a first group and a second group, the first group being decodable without reception of said second group; and

arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

26. A signal according to claim 25 comprising packets of at least two different media types, wherein the packets of one media type are in said first group and the packets of another media type are in said second group.

27. A signal according to claim 26 wherein said one media type is a video stream comprising reference pictures and non-reference pictures, and the another media type is an audio stream, wherein the the packets of non-reference pictures and the packets of reference pictures are in said first group such that the packets of reference pictures succeed the packets of non-reference pictures.

28. A signal according to claim 25 wherein a forward error correction code rate have been selected for said first group different from a forward error correction code rate for said second group.

29. A computer program product comprising a software for transmitting media information from a transmitting device to a receiving device, the software comprising machine executable steps for:

encoding a media stream;

forming packets from the encoded media stream, the packets containing application data units, the application data units having a decoding order;

generating at least two blocks from the packets;

organizing the packets within a block of the at least two blocks into a first group and a second group, the first group being decodable without reception of said second group; and

arranging the transmission order of the packets of said first group succeeding packets of said second group such that the application data units in the transmission order are at least partly in a different order than the decoding order.

30. A signal

comprising a first integral group of packets and a second integral group of packets;

said first integral group of packets being transmitted after said second integral group of packets;

said first integral group of packets being decodable to first packets of a coded media stream;

a second integral group of packets being decodable to second packets of a coded media stream;

said first packets of a coded media stream being decodable to uncompressed media samples without decoding of said second packets of a coded media stream;

at least one packet in said second packets of a coded media stream is succeeding in decoding order at least one packet in said first packets of a coded media stream.

31. A signal according to claim 30 in which decoding of both said first packets of a coded media stream and said second packets of a coded media stream improve the subjective quality compared to decoding of said first packets of coded media stream.

32. A signal according to claim 30 in which decoding of both said first packets of a coded media stream and said second packets of a coded media stream increases the output sample rate compared to decoding of said first packets of coded media stream

33. A signal according to claim 30 wherein the packets within at least one of said first integral group of packets and said second integral group of packets are in reverse order.

34. A signal comprising at least two coded media streams wherein

each media stream is organized into super blocks of a media data;

earliest and latest media decoding time is derivative for super blocks;

super blocks of a media stream do not overlap in decoding time;

super blocks of a media stream are ordered within the signal in ascending order of decoding time;

super block boundaries in decoding time of one media stream are within a defined range compared to super block boundaries of another media stream; and

at least one inter-stream frame is comprised of super blocks of different stream whose boundaries are within said defined range.

35. A signal according to claim 34 wherein

there is at least a first group of data of first media type and a second group of data of second media type;

said first group of data is not interleaved with said second group of data in the signal.

36. A signal according to claim 35 wherein said first group of data precedes said second group of data if the decoded output of said first group of data is subjectively less important than the decoded output of said second group of data.