System and method for generating multiple synchronized encoded representations of media data

• US 20050123058A1
• Filed: 12/22/2004
• Published: 06/09/2005
• Est. Priority Date: 04/27/1999
• Status: Active Grant

• Current Assignee: Cisco Systems, Inc.
• Original Assignee: RealNetworks, Inc.
• Inventors: Hefta-Daub, Bradley D., Greenbaum, Gary S., Ayars, Jeffrey M., Stammen, Dale R., Conklin, Gregory J., Lippman, Alan F.
• Granted Patent: US 7,885,340 B2
• Time in Patent Office: Days
• US Class Current: 375/240.280
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 21/23439 for generating different versions

Abstract

The present invention provides a system and methods for producing multiple encoded representations of a video input sequence. The multiple representations produced each contain identified synchronization frames that allow a server and a client to switch between streamed representations in real time without interruption. Synchronization frames are frames of encoded video that can be independently decoded. A representation can thus be decoded starting at a synchronization frame. Each synchronization frame in one representation has a corresponding synchronization frame at a substantially similar temporal location in any other generated representation of the same video input sequence. The temporal co-location of synchronization frames in all representations facilitates the dynamic switching between representations during the streaming process. The present invention also provides a video encoder application that shares data during the encoding of multiple representations of a video input sequence by reusing data calculated in the encoding of one representation to encode other representations. The application can also generate the multiple encoded representations simultaneously.

Claims

1. A method of producing a plurality of encoded representations of an input media signal, the method comprising:

providing the input media signal;

encoding said input media signal to generate the plurality of encoded representations, wherein at least a portion of the media signal is included in a first encoded representation of said plurality of encoded representations and the same portion is also included in a second encoded representation of said plurality of encoded representations, and said first encoded representation is encoded according to a different set of encoding parameters than said second encoded representation; and

designating a plurality of synchronization points such that switching between a decoding of one of said encoded representations and another of said encoded representations can be performed without any substantial discontinuity.

2. The method of claim 1, wherein each of said encoded representations can be decoded independently of any other encoded representation.

3. The method of claim 1, wherein each of said encoded representations can be decoded starting at said synchronization points.

4. The method of claim 1, wherein a temporal period between any two adjacent synchronization points does not exceed a specified maximum temporal period.

5. The method of claim 1, wherein each synchronization point corresponds to a substantially similar temporal location within each of the encoded representations.

6. The method of claim 1, wherein said input media signal comprises a video input sequence, wherein said video input sequence comprises frames of digital video, and wherein said synchronization points correspond to encoded frames of digital video.

7. The method of claim 6 further comprising:

identifying a frame in said input sequence;

encoding said identified frame to produce an encoded frame of a first encoded representation, wherein a decoding of said encoded frame of said first encoded representation does not require a decoded version of another frame;

identifying said encoded frame of said first encoded representation as corresponding to a synchronization point;

encoding said identified frame to produce an encoded frame of a second encoded representation, wherein a decoding of said encoded frame of said second encoded representation does not require a decoded version of another frame; and

identifying said encoded frame of said second encoded representation as corresponding to a synchronization point.

8. The method of claim 1, wherein at least a portion of each of said encoded representations is generated before any encoded re

36. The method of claim 2, wherein said plurality of encoded representations are interleaved in an output file or output stream.

37. The method of claim 2, wherein the input media signal comprises a plurality of different media sources.

38. The method of claim 37, wherein the media sources comprise at least one from the group consisting of:

audio segments, video frames, graphics, and still images.

39. The method of claim 2, wherein said input media signal comprises video and audio.

40. The method of claim 2, wherein each of said encoded representations is a representation of a portion of the input media signal.

41. The method of claim 2, wherein each of said encoded representations is a representation of the entire input media signal.

42. The method of claim 2, further comprising storing the plurality of encoded representations of the input media signal in a memory, wherein the memory comprises at least one from the group consisting of:

a media server, download server, a video server, a hard disk drive, a CD rewriteable drive, and a read/write DVD drive.

43. The method of claim 2, further comprising storing the plurality of encoded representations of the input media signal in a memory for streaming at least one of the encoded representations to a decoder.

9. A computer readable medium having stored thereon a plurality of instructions which, when executed by a processor in a computer system, cause the processor to perform the steps of:

accepting an input media signal;

encoding said input media signal to generate a plurality of encoded representations, wherein at least a portion of the media signal is included in a first of the plurality of encoded representations and said portion is included in a second of the plurality of the encoded representations and the first of the plurality of encoded representations is encoded according to a different set of encoding parameters than the second of the plurality of encoded representations; and

indicating a plurality of synchronization points such that switching between a decoding of one of said encoded representations and another of said encoded representations can be performed without any substantial discontinuity.

44. The medium of claim 9, wherein each of said encoded representations can be decoded independently of any other encoded representation.

10. A system for producing a plurality of encoded representations of a video input sequence comprising:

a video encoder configured to generate said plurality of encoded representations of said video input sequence, wherein at least a portion of the media signal is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations and said video encoder encodes at least two of the representations according to a different set of encoding parameters, and wherein the video encoder is further configured to designate a plurality of synchronization points such that switching between a decoding of one of said encoded representations to another of said encoded representations can be performed without any substantial discontinuity; and

an output module configured to output said encoded representations.

11. The system of claim 10, wherein each synchronization point corresponds to a substantially similar temporal location within each of the encoded representations.

12. The system of claim 10, further comprising a storage device configured to store said encoded representations.

13. The system of claim 10, further comprising a server configured to transmit at least one of said encoded representations over a communications network for a real-time presentation, said server responsive to a transition signal to switch from transmitting one of said encoded representations to transmitting another of said encoded representations to a client without a substantial interruption in said real-time presentation.

14. The system of claim 10, further comprising a decoder configured to decode a frame preceding a synchronization point in one of said encoded representations, then to decode a frame corresponding to the synchronization point in another of said encoded representations.

45. The system of claim 10, wherein each of said encoded representations can be decoded independently of any other encoded representation.

15. A video encoding system comprising:

a host computer;

a digital video input sequence;

output comprising a plurality of independent encoded representations of said digital video input sequence, wherein each representation is encoded according to a different set of encoding parameters, wherein at least a portion of the media signal is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations, and wherein each encoded representation contains synchronization frames identifying locations at which a switch from a decoding of one of said encoded representations to another of said encoded representations can be performed without any substantial discontinuity; and

a video encoding application operating on said host computer, wherein said video encoding application generates said output from said digital video input sequence.

16. The system of claim 15, wherein said video encoding application is configured to generate a set of data from said digital video input sequence, said video encoding application using said set of data to generate said plurality of encoded representations of said digital video input sequence.

17. The system of claim 15, further comprising a storage device used to store said encoded representations.

18. The system of claim 15, further comprising a server configured to transmit at least one of said encoded representations over a communications network for a real-time presentation, said server responsive to a transition signal to switch from transmitting one of said encoded representations to transmitting another of said encoded representations to a client without a substantial interruption in said real-time presentation.

19. The system of claim 15, further comprising a decoder configured to decode a frame preceding a first synchronization frame in one of said encoded representations, then to decode a second synchronization frame in another of said encoded representations, said second synchronization frame having substantially the same temporal location as said first synchronization frame.

20. A data file containing a plurality of independent encoded representations of a video sequence comprising:

a first of said encoded representations having a first set of synchronization frames; and

a second of said encoded representations having a second set of synchronization frames, wherein each of said second set of synchronization frames is associated with one of the first set of synchronization frames having a substantially similar temporal location in the video sequence.

21. The data file of claim 20, wherein said encoded representations comprise segments, and wherein segments of said first encoded representation are interleaved in said data file with segments of said second encoded representation.

22. The data file of claim 20, wherein each of said encoded representations is included contiguously within said data file.

23. The data file of claim 20, further comprising synchronization information, said synchronization information comprising locations of synchronization points within said data file.

24. A method of producing a plurality of encoded representations of an input media signal comprising:

providing the input media signal; and

using said input media signal to generate the plurality of encoded representations of said input media signal, wherein at least a portion of the media signal is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations and at least two of the plurality of encoded representations are encoded according to a different set of encoding parameters.

25. The method of claim 24, wherein each of said encoded representations is a complete and separate representation of said input media signal.

26. The method of claim 24, wherein any one encoded representation can be decoded without reference to another encoded representation.

27. The method of claim 24, wherein said set of data comprises intermediate encoding data.

28. The method of claim 24, wherein said input media signal is a video input sequence comprising frames of digitized video.

29. The method of claim 28, wherein said set of data comprises transform data.

30. The method of claim 28, wherein said set of data comprises discrete cosine transform data.

31. The method of claim 28, wherein said set of data comprises motion vector data.

32. The method of claim 28, wherein said set of data comprises color converted frame data.

33. The method of claim 28, wherein said set of

34. A computer readable medium having stored thereon a plurality of instructions which, when executed by a processor in a computer system, cause the processor to perform the steps of:

accepting an input media signal; and

using said input media signal to generate a plurality of independently encoded representations of said input media signal, wherein at least a portion of the media signal is included in one of the plurality of encoded representations and said portion is included in another one of the plurality of encoded representations and at least two of the encoded representations are encoded according to a different set of encoding parameters.

35. A system for producing a plurality of encoded representations of a video input sequence comprising:

a video encoder configured to generate a set of intermediate encoding data from said video input sequence, said video encoder using said set of intermediate encoding data to generate said plurality of independent encoded representations of said video input sequence, wherein at least a portion of the video input sequence is included in one of the plurality of the encoded representations and said portion is included in another one of the plurality of the encoded representations, and wherein each encoded representation is encoded according to a different set of encoding parameters; and

an output module configured to output said encoded representations.

46. A method of encoding, the method comprising:

encoding a media signal to generate the plurality of encoded representations for at least a selected part of the media signal, wherein each of the plurality of encoded representations is encoded according to a different set of encoding parameters, wherein each of the plurality of the encoded representations is adapted to be decoded to play the selected part of the media signal, and wherein each of the plurality of encoded representations are encoded using shared color data, motion vector data, and/or discrete cosine coefficients.

47. The method of claim 46, wherein said plurality of encoded representations are interleaved in an output file or output stream.

48. The method of claim 47, wherein the input media signal comprises a plurality of different media sources.

49. The method of claim 48, wherein the media sources comprise at least one from the group consisting of:

audio segments, video frames, graphics, and still images.

50. The method of claim 47, wherein said input media signal comprises video and audio.

51. The method of claim 47, wherein each of said encoded representations is a representation of a portion of the input media signal.

52. The method of claim 47, wherein each of said encoded representations is a representation of the entire input media signal.

53. The method of claim 47, further comprising storing the plurality of encoded representations of the input media signal in a memory, wherein the memory comprises at least one from the group consisting of:

a media server, download server, a video server, a hard disk drive, a CD rewriteable drive, and a read/write DVD drive.

54. The method of claim 47, further comprising storing the plurality of encoded representations of the input media signal in a memory for streaming at least one of the encoded representations to a decoder.