INDICATING PICTURE USEFULNESS FOR PLAYBACK OPTIMIZATION

• US 20080115176A1
• Filed: 07/31/2007
• Published: 05/15/2008
• Est. Priority Date: 11/13/2006
• Status: Active Grant

• Current Assignee: Tech 5 SAS
• Original Assignee: Scientific-Atlanta Incorporated (Cisco Systems, Inc.)
• Inventors: Rodriguez, Arturo A.
• Granted Patent: US 8,875,199 B2
• Time in Patent Office: Days
• US Class Current: 725/89
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/440281 by altering the temporal resolution, e.g. by frame skipping television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783

Abstract

In one embodiment the invention provides a method for outputting auxiliary information for use in playing back a video sequence, the method comprising obtaining a measure of usefulness of a particular picture for playing back the video sequence, wherein the measure of usefulness indicates a performance of playing back the video sequence if the particular picture is available in a decoded state at a time of playback versus the particular picture not being available in a decoded state at a time of playback; determining a position in a data stream that includes the video sequence; and storing the auxiliary information at the determined position in the data stream.

Claims

1. A method for outputting auxiliary information for use in playing back a video sequence, the method comprising:

obtaining a measure of usefulness of a particular picture for playing back the video sequence, wherein the measure of usefulness indicates a performance of playing back the video sequence if the particular picture is available in a decoded state at a time of playback versus the particular picture not being available in a decoded state at a time of playback;

determining a position in a data stream that includes the video sequence; and

storing the auxiliary information at the determined position in the data stream.

2. The method of claim 1, wherein playback occurs from a storage device in a set-top box.

3. The method of claim 2, wherein the measure of usefulness includes an indication of the usefulness of the particular picture during playback at a slower speed than normal.

4. The method of claim 2, wherein the measure of usefulness includes an indication of the usefulness of the particular picture during playback at a higher speed than normal.

5. The method of claim 2, wherein the measure of usefulness includes an indication of the usefulness of the particular picture during playback while skipping frames.

6. The method of claim 2, wherein the measure of usefulness includes an indication of the usefulness of the particular picture during a pause operation.

7. The method of claim 1, wherein the measure of usefulness includes an indication of an output-delay-property.

8. The method of claim 1, wherein the measure of usefulness includes an indication of a complete-information-property.

9. The method of claim 1, wherein the measure of usefulness includes an indication of an importance-level-property.

10. The method of claim 1, wherein the measure of usefulness is only stored during a playback operation that uses a trick play.

20. The method of claim 1, wherein the step of determining includes:

reading a first data field in the first video sequence to obtain a first value, wherein the value indicates a number of consecutive pictures prior to the exit point.

21. The method of claim 2, further comprising:

reading a second data field in the first video sequence to obtain a second value, wherein the second value indicates a number of decoded pictures in the decoded picture buffer (DPB).

22. The method of claim 3, wherein the second value indicates a number of decoded pictures in the DPB that have successive picture output times.

23. The method of claim 4, wherein a first of the successive output times includes a picture output time immediately after the decompression time of a last consecutive picture prior to the exit point as indicated by the first value.

24. The method of claim 1, wherein the actions are performed at a video splicing device located in a headend of a digital video broadcast system.

25. The method of claim 1, further comprising:

assigning picture output times to the pictures in the picture decode buffer in accordance with an original order of the pictures in the picture decode buffer.

26. The method of claim 7, further comprising:

ensuring that all of the pictures from the first video sequence to be decoded are decoded prior to the exit point.

27. The method of claim 8, further comprising:

decoding a number of pictures from the second video sequence as indicated by the second value, wherein the decoding occurs prior to completion of a transition period.

28. The method of claim 9, further comprising:

setting an earliest picture output time of a picture from the second video sequence to be just after an end of the transition period.

29. The method of claim 1, wherein the first value is larger than the second value.

32. The method of claim 3, wherein the first and second data fields are provided in association with multiple pictures.

35. The method of claim 1, further comprising:

causing display of a first picture of the second video sequence to be equal to a presentation time stamp of a first picture in the DPB.

39. The apparatus of claim 20, wherein the machine-readable storage medium further includes one or more instructions for:

reading a first data field in the first video sequence to obtain a first value, wherein the value indicates a number of consecutive pictures prior to the exit point.

40. The apparatus of claim 20, wherein the machine-readable storage medium further includes one or more instructions for:

reading a second data field in the first video sequence to obtain a second value, wherein the second value indicates a number of decoded pictures in the decoded picture buffer (DPB).

42. The machine-readable medium of claim 23, further comprising one or more instructions for:

reading a first data field in the first video sequence to obtain a first value, wherein the value indicates a number of consecutive pictures prior to the exit point.

43. The machine-readable medium of claim 23, further comprising one or more instructions for:

reading a second data field in the first video sequence to obtain a second value, wherein the second value indicates a number of decoded pictures in the decoded picture buffer (DPB).

45. The method of claim 26, further comprising:

determining whether a picture includes an I-frame; and

in response to the determining, setting the measure of picture importance according to a predetermined criteria.

46. The method of claim 26, further comprising:

determining whether a picture includes a P-frame; and

in response to the determining, setting the measure of picture importance according to a predetermined criteria.

47. The method of claim 26, further comprising:

determining whether a picture includes a B-frame; and

in response to the determining, setting the measure of picture importance according to a predetermined criteria.

11. An apparatus for outputting auxiliary information for use in playing back a video sequence, the apparatus comprising:

a processor;

one or more instructions executable by the processor for obtaining a measure of usefulness of a particular picture for playing back the video sequence, wherein the measure of usefulness indicates a performance of playing back the video sequence if the particular picture is available in a decoded state at a time of playback versus the particular picture not being available in a decoded state at a time of playback;

determining a position in a data stream that includes the video sequence; and

storing the auxiliary information at the determined position in the data stream.

12. The apparatus of claim 11, wherein the apparatus is included as part of a set-top box.

13. The apparatus of claim 11, wherein the apparatus is included as part of a video encoder.

14. The apparatus of claim 11, wherein the apparatus is included as part of a video decoder.

15. The apparatus of claim 11, wherein the apparatus is included as part of a network device.

16. The apparatus of claim 11, wherein the apparatus is included as part of a display system.

30. The method of claim 11, wherein the first value is also greater than a predetermined threshold.

31. The method of claim 12, wherein the predetermined threshold is equal to three picture output times.

33. The method of claim 14, wherein the first and second data fields are provided in association with each of the determined number of pictures of the first video sequence.

34. The method of claim 15, wherein the first data field'"'"'s value is N for the first picture and decreases by one successively in each successive instance of the first data field in a subsequent picture.

49. The method of claim 30, further comprising:

predicting a measure of picture importance in the presentation operation, wherein the measure is based at least in part on picture usefulness to achieve a desired decode order of pictures.

17. A machine-readable storage medium including one or more instructions executable by a processor for:

obtaining a measure of usefulness of a particular picture for playing back the video sequence, wherein the measure of usefulness indicates a performance of playing back the video sequence if the particular picture is available in a decoded state at a time of playback versus the particular picture not being available in a decoded state at a time of playback;

determining a position in a data stream that includes the video sequence; and

storing the auxiliary information at the determined position in the data stream.

36. The method of claim 17, wherein the displayed first picture of the second video sequence includes an instantaneous decoding refresh picture.

18. A method for decoding a video sequence, the method comprising:

obtaining a measure of usefulness of a particular picture for decoding the video sequence, wherein the measure of usefulness indicates a performance of playing back the video sequence if the particular picture is available in a decoded state at a time of playback versus the particular picture not being available in a decoded state at a time of playback; and

decoding the video sequence in response to the measure of usefulness of the particular picture.

37. The method of claim 18, wherein the instantaneous decoding refresh picture includes an I-frame.

19. A method for splicing from a first video sequence to a second video sequence, wherein each video sequence includes encoded pictures that are decoded into decoded pictures in a decode order, wherein decoded pictures are displayed in a presentation order at picture output times, the method comprising:

identifying an exit point for the first video sequence, wherein the exit point determines a last frame to display of the first video sequence;

determining a number of pictures of the first video sequence that are already decoded into a decode picture buffer; and

initiating decoding of the second video sequence so that a sufficient number of pictures in the second video sequence are decoded at a time of displaying the last picture from the decode picture buffer so that displaying of pictures in the second video sequence can begin at a next picture output time.

38. An apparatus for splicing from a first video sequence to a second video sequence, wherein each video sequence includes encoded pictures that are decoded into decoded pictures in a decode order, wherein decoded pictures are displayed in a presentation order at picture output times, the apparatus comprising:

a processor;

a machine-readable storage medium including instructions for execution by the processor, including one or more instructions for;

identifying an exit point for the first video sequence, wherein the exit point determines a last frame to display of the first video sequence;

determining a number of pictures of the first video sequence that are already decoded into a decode picture buffer; and

initiating decoding of the second video sequence so that a sufficient number of pictures in the second video sequence are decoded at a time of displaying the last picture from the decode picture buffer so that displaying of pictures in the second video sequence can begin at a next picture output time.

41. A machine-readable storage medium including one or more instructions for splicing from a first video sequence to a second video sequence, wherein each video sequence includes encoded pictures that are decoded into decoded pictures in a decode order, wherein decoded pictures are displayed in a presentation order at picture output times, the machine-readable medium comprising one or more instructions for:

identifying an exit point for the first video sequence, wherein the exit point determines a last frame to display of the first video sequence;

determining a number of pictures of the first video sequence that are already decoded into a decode picture buffer; and

initiating decoding of the second video sequence so that a sufficient number of pictures in the second video sequence are decoded at a time of displaying the last picture from the decode picture buffer so that displaying of pictures in the second video sequence can begin at a next picture output time.

44. A method for providing a measure of picture importance in a decode operation, the method comprising:

predicting a measure of picture importance in the decode operation, wherein the measure is based at least in part on a picture type as defined by the Advanced Video Coding standard.

48. A method for providing a measure of picture importance in a presentation operation, the method comprising:

predicting a measure of picture importance in the presentation operation, wherein the measure is based at least in part on picture usefulness to achieve a desired presentation order of pictures.

50. A method for providing a measure of picture importance for decoding digital video information, wherein the digital video information includes a plurality of pictures, wherein the pictures include interdependencies wherein a dependent picture can not be decoded until a picture from which it depends is decoded, the method comprising:

identifying dependencies among the pictures;

generating a measure of usefulness for at least one of the pictures, wherein the measure of usefulness includes a value that is used at a time of decoding to determine whether to maintain the picture in the decoding system; and

including the generated measure of usefulness in the digital video information.