System and method for generating multiple synchronized encoded representations of media data

• US 7,885,340 B2
• Filed: 12/22/2004
• Issued: 02/08/2011
• Est. Priority Date: 04/27/1999
• Status: Active Grant

• Current Assignee: Cisco Systems, Inc.
• Original Assignee: RealNetworks, Inc.
• Inventors: Stammen, Dale R., Hefta-Gaub, Bradley D., Lippman, Alan F., Conklin, Gregory J., Ayars, Jeffrey M., Greenbaum, Gary S.
• Primary Examiner(s): Senfi, Behrooz
• Application Number:
  US11/022,474
• Publication Number:
  US 20050123058A1
• Time in Patent Office: 2,239 Days
• Field of Search: 348/425.1, 348/425.3, 348/425.4, 348/500, 348/516, 348/577, 348/521, 348/522, 348/142, 348/565, 348/563, 348/564, 348/416, 348/512, 348/513, 348/518, 348/536, 348/14.1, 348/14.13, 348/14.14, 348/14.15, 348/14.12, 348/154, 348/155, 348/431, 348/452, 348/399, 348/397, 348/469, 375/240.28, 375/240.26, 375/240.12, 375/240.16, 375/240.01, 375/240.1, 375/240.13, 375/240.14, 375/240, 375/240.2, 382/239, 382/232, 382/236
• US Class Current: 375/240.28
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 21/23439 for generating different versions

Abstract

The present invention provides a system and methods for producing multiple encoded representations of a video input sequence. The multiple representations produced each contain identified synchronization frames that allow a server and a client to switch between streamed representations in real time without interruption. Synchronization frames are frames of encoded video that can be independently decoded. A representation can thus be decoded starting at a synchronization frame. Each synchronization frame in one representation has a corresponding synchronization frame at a substantially similar temporal location in any other generated representation of the same video input sequence. The temporal co-location of synchronization frames in all representations facilitates the dynamic switching between representations during the streaming process. The present invention also provides a video encoder application that shares data during the encoding of multiple representations of a video input sequence by reusing data calculated in the encoding of one representation to encode other representations. The application can also generate the multiple encoded representations simultaneously.

Claims

1. A method of producing a plurality of encoded representations of an input media signal, the method comprising:

providing the input media signal to an encoder;

encoding said input media signal with said encoder to generate the plurality of encoded representations of the input media signal, wherein at least a portion of the media signal is included in a first encoded representation of said plurality of encoded representations and the same portion is also included in a second encoded representation of said plurality of encoded representations, and said first encoded representation is encoded according to a different set of encoding parameters than is said second encoded representation; and

designating and identifying a plurality of synchronization points within each of said encoded representations of the input media signal such that switching between a decoding of one of said encoded representations and another of said encoded representations can be performed without any substantial discontinuity.

2. The method of claim 1, wherein each of said encoded representations can be decoded independently of any other encoded representation.

3. The method of claim 1, wherein each of said encoded representations can be decoded starting at said synchronization points.

4. The method of claim 1, wherein a temporal period between any two adjacent synchronization points does not exceed a specified maximum temporal period.

5. The method of claim 1, wherein each synchronization points corresponds to a substantially similar temporal location within each of the encoded representations.

6. The method of claim 1, wherein said input media signal comprises a video input sequence, wherein said video input sequence comprises frames of digital video, and wherein said synchronization points correspond to encoded frames of digital video.

7. The method of claim 6 further comprising:

identifying a frame in said input sequence;

encoding said identified frame to produce an encoded frame of a first encoded representation, wherein a decoding of said encoded frame of said first encoded representation does not require a decoded version of another frame; and

identifying said encoded frame of said first encoded representation as corresponding to a synchronization point.

8. The method of claim 1, wherein at least a portion of each of said encoded representations is generated before any encoded representation is completely generated.

36. The method of claim 1, wherein said plurality of encoded representations are interleaved in an output file or output stream.

37. The method of claim 1, wherein the input media signal comprises a plurality of different media sources.

38. The method of claim 37, wherein the media sources comprise at least one from the group consisting of:

audio segments, video frames, graphics, and still images.

39. The method of claim 1, wherein said input media signal comprises video and audio.

40. The method of claim 1, wherein each of said encoded representations is a representation of a portion of the input media signal.

41. The method of claim 1, wherein each of said encoded representations is a representation of the entire input media signal.

42. The method of claim 1, further comprising storing the plurality of encoded representations of the input media signal in a memory, wherein the memory comprises at least one from the group consisting of:

a media server, download server, a video server, a hard disk drive, a CD rewriteable drive, and a read/write DVD drive.

43. The method of claim 1, further comprising storing the plurality of encoded representations of the input media signal in a memory for streaming at least one of the encoded representations to a decoder.

57. The method of claim 1 wherein designating a plurality of synchronization points includes:

determining whether each frame of the plurality of encoded representations is intra-encoded, and

in response to each frame being intra-coded, designating each frame as a synchronization frame.

9. A series of executable instructions embodied in a tangible computer-readable medium which, when executed by a computer processor, causes the processor to:

accept an input media signal;

encode said input media signal to generate a plurality of encoded representations of the input media signal, wherein at least a portion of the media signal is included in in a first of the plurality of encoded representations and said portion is included in a second of the plurality of the encoded representations and the first of the plurality of encoded representations is encoded according to a different set of encoding parameters than the second of the plurality of encoded representations; and

indicate a plurality of synchronization points within each of the encoded representations of the input media signal such that switching between a decoding of the first of said encoded representations and the second of said encoded representations can be performed without any substantial discontinuity.

44. The series of claim 9, wherein each of said encoded representations can be decoded independently of any other encoded representation.

10. A system for producing a plurality of encoded representations of a media presentation, comprising:

a source, operable to provide a media presentation; and

an encoder coupled to the source, the encoder configured to receive said media presentation and to generate a plurality of encoded representations of said media presentation, wherein at least a portion of the media presentation is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations and said encoder encodes at least two of the representations according to a different set of encoding parameters, and wherein the encoder is further configured to designate a plurality of synchronization points within each of said encoded representations of the media presentation such that switching between a decoding of one of said encoded representations to another of said encoded representations can be performed without any substantial discontinuity.

11. The system of claim 10, wherein each synchronization point corresponds to a substantially similar temporal location within each of the encoded representations.

12. The system of claim 10, further comprising a storage device configured to store said encoded representations.

13. The system of claim 10, further comprising a server configured to transmit at least one of said encoded representations over a communications network for presentation, said server responsive to a transition signal to switch from transmitting one of said encoded representations to transmitting another of said encoded representations to a client without a substantial interruption in said presentation.

14. The system of claim 10, further comprising a decoder configured to decode a frame preceding a synchronization point in one of said encoded representations, then to decode a frame corresponding to the synchronization point in another of said encoded representations.

45. The system of claim 10, wherein each of said encoded representations can be decoded independently of any other encoded representation.

15. A video encoding system comprising:

a host computer;

a digital video input sequence;

output comprising a plurality of independent encoded representations of said digital video input sequence, wherein each representation is encoded according to a different set of encoding parameters, wherein at least a portion of the media signal is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations, and wherein each encoded representation contains synchronization frames identifying locations at which a switch from a decoding of one of said encoded representations to another of said encoded representations can be performed without any substantial discontinuity; and

a video encoding application operating on said host computer, wherein said video encoding application generates said output from said digital video input sequence.

16. The system of claim 15, wherein said video encoding application is configured to generate a set of data from said digital video input sequence, said video encoding application using said set of data to generate said plurality of encoded representations of said digital video input sequence.

17. The system of claim 15, further comprising a storage device used to store said encoded representations.

18. The system of claim 15, further comprising a server configured to transmit at least one of said encoded representations over a communications network for presentation, said server responsive to a transition signal to switch from transmitting one of said encoded representations to transmitting another of said encoded representations to a client without a substantial interruption in said presentation.

19. The system of claim 15, further comprising a decoder configured to decode a frame preceding a first synchronization frame in one of said encoded representations, then to decode a second synchronization frame in another of said encoded representations, said second synchronization frame having substantially the same temporal location as said first synchronization frame.

20. A data file containing a plurality of independent encoded representations of a video sequence comprising:

a first of said encoded representations having a first set of synchronization frames; and

a second of said encoded representations having a second set of designated synchronization frames, wherein each of said second set of synchronization frames is associated with one of the first set of synchronization frames, and wherein the associated synchronization frames have a substantially similar temporal location in the video sequence such that switching between a decoding of the first of said encoded representations and the second of said encoded representations can be performed without any substantial discontinuity.

21. The data file of claim 20, wherein said encoded representations comprise segments, and wherein segments of said first encoded representation are interleaved in said data file with segments of said second encoded representation.

22. The data file of claim 20, wherein each of said encoded representations is included contiguously within said data file.

23. The data file of claim 20, further comprising synchronization information, said synchronization information comprising locations of synchronization points within said data file.

24. A method of producing a plurality of encoded representations of an input media signal comprising:

receiving the input media signal with an encoder; and

generating the plurality of encoded representations of said input media signal with the encoder, wherein at least a same portion of the media signal is included in each of a first encoded representation and a second encoded representation, wherein the first and second encoded representations are encoded according to a different set of encoding parameters, and wherein the first and second encoded representations each include a respective identified synchronization frame that enables switching between the first and second encoded representations.

25. The method of claim 24, wherein each of said encoded representations is a complete and separate representation of said input media signal.

26. The method of claim 24, wherein any one encoded representation can be decoded without reference to another encoded representation.

27. The method of claim 24, wherein said set of data comprises intermediate encoding data.

28. The method of claim 24, wherein said input media signal is a video input sequence comprising frames of digitized video.

29. The method of claim 28, wherein said set of data comprises transform data.

30. The method of claim 28, wherein said set of data comprises discrete cosine transform data.

31. The method of claim 28, wherein said set of data comprises motion vector data.

32. The method of claim 28, wherein said set of data comprises color converted frame data.

33. The method of claim 28, wherein said set of data comprises resampled frame data.

34. A series of executable instructions embodied in a tangible computer-readable medium which, when executed by a processor, causes the processor to:

generate a plurality of encoded representations of said input media signal, wherein at least a same portion of the media signal is included in each of a first and second encoded representation, wherein the first and second encoded representations are encoded according to a different set of encoding parameters, and wherein the first and second encoded representations respectively include a designated synchronization frame that enables switching between the first and second encoded representations; and

store at least one of the plurality of said encoded representations in a tangible computer-readable medium.

35. A system for producing a plurality of encoded representations of a video input sequence comprising:

a video encoder configured to generate a set of intermediate encoding data from said video input sequence, said video encoder using said set of intermediate encoding data to generate said plurality of independent encoded representations of said video input sequence, wherein at least a portion of the video input sequence is included in one of the plurality of the encoded representations and said portion is included in another one of the plurality of the encoded representations, wherein each encoded representation is encoded according to a different set of encoding parameters, and wherein each encoded representation respectively includes a designated synchronization frame that enables switching between encoded representations without substantial discontinuity; and

an output module configured to output said encoded representations.

46. A method of encoding, the method comprising:

receiving a media signal;

generating with an encoder a first encoded representation of at least a selected part of the media signal using a first set of data and a first set of encoding parameters;

generating with the encoder a second encoded representation of the at least a selected part of the media signal using the first set of data and a second set of encoding parameters that is different than the first set of encoding parameters;

identifying corresponding synchronization points in the first and second encoded representations; and

storing at least one of said plurality of encoded representations in a tangible computer-readable medium.

47. The method of claim 46, wherein said plurality of encoded representations are interleaved in an output file or output stream.

48. The method of claim 47, wherein the input media signal comprises a plurality of different media sources.

49. The method of claim 48, wherein the media sources comprise at least one from the group consisting of:

audio segments, video frames, graphics, and still images.

50. The method of claim 47, wherein said input media signal comprises video and audio.

51. The method of claim 47, wherein each of said encoded representations is a representation a portion of the input media signal.

52. The method of claim 47, wherein each of said encoded representations is a representation of the entire input media signal.

53. The method of claim 47, further comprising storing the plurality of encoded representations of the input media signal in a memory, wherein the memory comprises at least one from the group consisting of:

a media server, download server, a video server, a hard disk drive, a CD rewriteable drive, and a read/write DVD drive.

54. The method of claim 47, further comprising storing the plurality of encoded representations of the input media signal in a memory for streaming at least one of the encoded representations to a decoder.

58. The method of claim 46 wherein the first set of data includes one or more of color data, motion vector data, and discrete cosine coefficients.

55. A method of producing a plurality of encoded representations of an input media signal, the method comprising:

providing the input media signal to an encoder;

encoding said input media signal with said encoder to generate the plurality of encoded representations of the input media signal, wherein at least a portion of the media signal is included in a first encoded representation of said plurality of encoded representations and the same portion is also included in a second encoded representation of said plurality of encoded representations, and said first encoded representation is encoded according to a different set of encoding parameters than is said second encoded representation; and

designating a plurality of synchronization points within each of said encoded representations of the input media signal such that switching between a decoding of one of said encoded representations and another of said encoded representations can be performed without any substantial discontinuity;

wherein designating a plurality of synchronization points includes;

determining whether a frame in a master representation of the media presentation is intra-encoded,

comparing a temporal period since the previous synchronization point to a maximum temporal period between adjacent synchronization points, and

in response to the temporal period being equal to or greater than a predetermined percent of the maximum temporal period, and to the frame being intra-coded, designating the frame as a synchronization frame.

56. The method of claim 55 wherein the predetermined percent includes 90 percent.