Capture, editing and encoding of motion pictures encoded with repeating fields or frames

• US 20060233248A1
• Filed: 02/28/2006
• Published: 10/19/2006
• Est. Priority Date: 04/15/2005
• Status: Active Grant

• Current Assignee: Avid Technology Incorporated
• Original Assignee: Avid Technology Incorporated
• Inventors: Rynderman, Michel, Phillips, Michael E.
• Granted Patent: US 8,023,568 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

An image sequence captured at a first image rate is encoded as an encoded image sequence at a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded. To assist in editing a video program using the encoded image sequence, the information in the encoded image sequence indicative of images that are repeated when decoded may be cleared. Further, image index information for the encoded image sequence may be generated. This image index information is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence. Temporal positions used with the image index information correspond to the first image rate. Editing a video program involves defining a sequence of clips that reference source encoded image sequences using edit points. Each clip has a start and end points in its corresponding source and a position for the clip in the edited video program. The temporal resolution of these edit points corresponds to the first image rate. The information in the encoded image sequence indicative of images that are repeated when decoded may be ignored or may have already been cleared. An edited video program may be output as an encoded image sequence, using the source encoded image sequences referenced by the sequence of clips. The encoded image sequence is constructed according to the edited sequence of clips, using the source encoded image sequences. In particular, the encoder clears any information in the source encoded image sequences indicative of images that are repeated when decoded. In the output encoded image sequence, information indicative of images that are repeated when decoded is set according to a cadence associated with a difference between the first image rate and the second image rate.

Claims

1. A computer-implemented method for processing data including an image sequence captured at a first image rate and encoded as an encoded image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

receiving the encoded image sequence; and

creating image index information of the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used with the image index information correspond to the first image rate.

2. The computer-implemented method of claim 1, wherein the first image rate is 23.976 images per second and the second image rate is 29.97 images per second.

3. The computer-implemented method of claim 1, wherein the first image rate is 24 images per second and the second image rate is 30 images per second.

4. The computer-implemented method of claim 1, wherein the first image rate is 24 images per second and the second image rate is 60 images per second.

5. The computer-implemented method of claim 1, wherein the first image rate is 23.976 images per second and the second image rate is 59.94 images per second.

6. The computer-implemented method of claim 1, wherein the image sequence is encoded using intraframe compression.

7. The computer-implemented method of claim 1, wherein the image sequence is encoded using both interframe and intraframe compression.

8. The computer-implemented method of claim 7, wherein the temporal positions of images in the image sequence are in a different order than the positions of corresponding data in the encoded image sequence.

9. The computer-implemented method of claim 1, further comprising clearing all information in the encoded image sequence indicative of images that are repeated when decoded.

10. The computer-implemented method of claim 9, wherein the information in the encoded image sequence indicative of images that are repeated when decoded includes a repeat field flag associated with each image that is repeated when decoded.

11. The computer-implemented method of claim 10, wherein the encoded image sequence is encoded according to the MPEG-2 standard.

12. The computer-implemented method of claim 9, wherein the encoded image sequence is encoded according to the H.264 standard.

13. The computer-implemented method of claim 1, wherein creating the image index information comprises:

receiving an original image index of the encoded image sequence that maps a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the image index correspond to the second image rate; and

converting the original image index into a new image index of the encoded image sequence that maps a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the new image index correspond to the first image rate.

14. The computer-implemented method of claim 1, wherein the data includes audio data associated with the encoded image sequence, the method further comprising:

separating the encoded image sequence and the audio data from the data into separate data files.

15. The computer-implemented method of claim 14, wherein the audio data and the encoded image sequence are multiplexed together in the data and wherein the step of separating includes demultiplexing the audio data from the encoded image sequence.

16. The computer-implemented method of claim 1, wherein the encoded image sequence is received and the method is performed during live recording of the data.

17. The computer-implemented method of claim 16, wherein the method is performed by circuitry in the housing of a portable video camera.

18. The computer-implemented method of claim 1, wherein the encoded image sequence is recorded and the method is performed after recording of the data.

19. The computer-implemented method of claim 9, wherein the first image rate is 23.976 images per second and the second image rate is 29.97 images per second.

20. The computer-implemented method of claim 9, wherein the first image rate is 24 images per second and the second image rate is 29.97 images per second.

21. The computer-implemented method of claim 9, wherein the first image rate is 24 images per second and the second image rate is 59.94 images per second.

22. The computer-implemented method of claim 9, wherein the first image rate is 23.976 images per second and the second image rate is 59.94 images per second.

23. The computer-implemented method of claim 9, wherein the image sequence is encoded using intraframe compression.

24. The computer-implemented method of claim 9, wherein the image sequence is encoded using both interframe and intraframe compression.

25. The computer-implemented method of claim 9, wherein the temporal positions of images in the image sequence are in a different order than the positions of corresponding data in the encoded image sequence.

26. The computer-implemented method of claim 9, wherein creating the image index information comprises:

receiving an original image index of the encoded image sequence that maps a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the image index correspond to the second image rate; and

converting the original image index into a new image index of the encoded image sequence that maps a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the new image index correspond to the first image rate.

27. The computer-implemented method of claim 9, wherein the data includes audio data associated with the encoded image sequence, the method further comprising:

separating the encoded image sequence and the audio data from the data into separate data files.

28. The computer-implemented method of claim 9, wherein the audio data and the encoded image sequence are multiplexed together in the data and wherein the step of separating includes demultiplexing the audio data from the encoded image sequence.

29. The computer-implemented method of claim 9, wherein the encoded image sequence is received and the method is performed during live recording of the data.

30. The computer-implemented method of claim 9, wherein the method is performed by circuitry in the housing of a portable video camera.

31. The computer-implemented method of claim 9, wherein the encoded image sequence is recorded and the method is performed after recording of the data.

32. An apparatus for processing data including an image sequence captured at a first image rate and encoded as an encoded image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the apparatus comprising:

means for receiving the encoded image sequence; and

means for creating image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used with the image index information correspond to the first image rate.

33. A computer program product, comprising:

a computer readable medium;

computer program instruction stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for processing data including an image sequence captured at a first image rate and encoded as an encoded image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising;

receiving the encoded image sequence; and

creating image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used with the image index information correspond to the first image rate.

34. A computer system for processing data including an image sequence captured at a first image rate and encoded as an encoded image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the computer system comprising:

an input for receiving the encoded image sequence; and

a processor programmed to create image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used with the image index information correspond to the first image rate.

35. A computer-implemented method for processing data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

receiving the encoded image sequence; and

clearing all information in the encoded image sequence indicative of images that are repeated when decoded.

36. The computer-implemented method of claim 35, wherein the first image rate is 23.976 images per second and the second image rate is 29.97 images per second.

37. The computer-implemented method of claim 35, wherein the first image rate is 24 images per second and the second image rate is 29.97 images per second.

38. The computer-implemented method of claim 35, wherein the first image rate is 24 images per second and the second image rate is 59.94 images per second.

39. The computer-implemented method of claim 35, wherein the first image rate is 23.976 images per second and the second image rate is 59.94 images per second.

40. The computer-implemented method of claim 35, wherein the image sequence is encoded using intraframe compression.

41. The computer-implemented method of claim 35, wherein the image sequence is encoded using both interframe and intraframe compression.

42. The computer-implemented method of claim 35, wherein the information in the encoded image sequence indicative of images that are repeated when decoded includes a repeat field flag associated with an image.

43. The computer implemented method of claim 42, wherein the encoded image sequence is encoded according to the MPEG-2 standard.

44. The computer-implemented method of claim 35, wherein the encoded image sequence in encoded according to the MPEG-2 standard.

45. The computer-implemented method of claim 35, wherein the data includes audio data associated with the encoded image sequence, the method further comprising:

separating the encoded image sequence and the audio data from the data into separate data files.

46. The computer-implemented method of claim 45, wherein the audio data and the encoded image sequence are multiplexed together in the data and wherein the step of separating includes demultiplexing the audio data from the encoded image sequence.

47. The computer-implemented method of claim 35, wherein the encoded image sequence is received and the method is performed during live recording of the data.

48. The computer-implemented method of claim 47, wherein the method is performed by circuitry in the housing of a portable video camera.

49. The computer-implemented method of claim 35, wherein the encoded image sequence is recorded and the method is performed after recording of the data.

50. An apparatus for processing data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

means for receiving the encoded image sequence; and

means for clearing all information in the encoded image sequence indicative of images that are repeated when decoded.

51. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for processing data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising;

receiving the encoded image sequence; and

clearing all information in the encoded image sequence indicative of images that are repeated when decoded.

52. A computer system for processing data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the computer system comprising:

an input for receiving the encoded image sequence; and

a processor programmed to clear all information in the encoded image sequence indicative of images that are repeated when decoded.

53. A method for editing encoded data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence;

permitting an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

during playback of the encoded image sequence, decoding the encoded image sequence, using the image index information, to produce a decoded image sequence at the first image rate by not repeating images.

54. The method of claim 53, wherein the temporal positions used with the image index information correspond to the first image rate.

55. The method of claim 53, wherein decoding includes ignoring the information in the encoded image sequences indicative of images that are repeated when decoded.

56. The method of claim 54, further comprising, prior to decoding, clearing the information in the encoded image sequences indicative of images that are repeated when decoded.

57. The method of claim 56, further comprising creating the image index information.

58. The method of claim 57, wherein creating the image index information comprises:

receiving an original image index of the encoded image sequence that maps a temporal position of an image in the image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the image index correspond to the second image rate.; and

converting the original image index into a new image index for the encoded image sequence that maps a temporal position of an image in the image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used in the new image index correspond to the first image rate.

59. The method of claim 53, wherein the temporal positions used with the image index information correspond to the second image rate.

60. The method of claim 59, wherein decoding includes ignoring the information in the encoded image sequences indicative of images are repeated when decoded.

61. The method of claim 59, further comprising, prior to decoding, clearing the information in the encoded image sequences indicative of images that are repeated when decoded.

62. The method of claim 53, wherein the encoded data includes audio data associated with the encoded image sequence, the method further comprising:

separating the encoded image sequence and the audio data from the encoded data into separate data files.

63. The method of claim 62, wherein the audio data and the encoded image sequences are multiplexed together in the encoded data and wherein separating includes demultiplexing the audio data from the encoded image sequence.

64. The method of claim 53, wherein the information in the encoded image sequences indicative of images that are repeated when decoded includes a repeat field flag associated with encoded data for each image that is to be repeated when decoded.

65. The method of claims 64, wherein the encoded image sequence is encoded according to the MPEG-2 standard.

66. The method of claim 53, wherein the encoded image sequence is encoded according to the H.264 standard.

67. An apparatus for editing encoded data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the apparatus comprising:

means for receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence;

means for permitting an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

means for decoding the encoded image sequence, during playback of the encoded image sequence, using the image index information, to produce a decoded image sequence at the first image rate by not repeating images.

68. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for editing encoded data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising;

receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence;

permitting an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

decoding the encoded image sequence, during playback of the encoded image sequence, using the image index information, to produce a decoded image sequence at the first image rate by not repeating images.

69. A computer system for editing encoded data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the computer system comprising:

a memory for receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence;

an input for receiving instructions from an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

a processor programmed to decode the encoded image sequence, during playback of the encoded image sequence, using the image index information, to produce a decoded image sequence at the first image rate by not repeating images.

70. A method for exporting an edited image sequence having a first image rate as an encoded image sequence having a second image rate different from the first image rate, the method comprising:

receiving data describing an edited sequence of clips that reference source encoded image sequences using edit points at a temporal resolution corresponding to the first image rate;

constructing the encoded image sequence according to the edited sequence of clips and using the source encoded image sequences, including;

clearing any information in the source encoded image sequences, used in the sequence of clips, indicative of images that are repeated when decoded; and

setting, in the encoded image sequence, information indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.

71. The method of claim 70, wherein the information in the encoded image sequence indicative of images that are repeated when decoded includes a repeat field flag associated with each image that is repeated when decoded.

72. The method of claim 71, wherein the encoded image sequence is encoded according to the MPEG-2 standard.

73. The method of claim 71, wherein the encoded image sequence is encoded according to the H.264 standard.

74. The method of claim 70, wherein constructing further comprises:

identifying rendered portions of the edited sequence;

rendering and encoding the rendered portions of the edited sequence;

setting information, in the encoded rendered portions in the encoded edited sequence, indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.

75. The method of claim 70, wherein constructing further comprises:

identifying rendered portions and unrendered portions of the edited sequence;

for each unrendered portion, inserting data from a corresponding source encoded image sequence into the encoded edited sequence; and

for each rendered portion,

rendering and encoding the rendered portions of the edited sequence; and

inserting data the encoded rendered portions into the encoded image sequence.

76. An apparatus for exporting an edited image sequence having a first image rate as an encoded image sequence having a second image rate different from the first image rate, the apparatus comprising:

means for receiving data describing an edited sequence of clips that reference source encoded image sequences using edit points at a temporal resolution corresponding to the first image rate;

means for constructing the encoded image sequence according to the edited sequence of clips and using the source encoded image sequences, including;

means for clearing any information in the source encoded image sequences, used in the sequence of clips, indicative of images that are repeated when decoded; and

means for setting, in the encoded image sequence, information indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.

77. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for exporting an edited image sequence having a first image rate as an encoded image sequence having a second image rate different from the first image rate, the method comprising;

receiving data describing an edited sequence of clips that reference source encoded image sequences using edit points at a temporal resolution corresponding to the first image rate;

constructing the encoded image sequence according to the edited sequence of clips and using the source encoded image sequences, including;

clearing any information in the source encoded image sequences, used in the sequence of clips, indicative of images that are repeated when decoded; and

setting, in the encoded image sequence, information indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.

78. A computer system for exporting an edited image sequence having a first image rate as an encoded image sequence having a second image rate different from the first image rate, the computer system comprising:

a memory for receiving data describing an edited sequence of clips that reference source encoded image sequences using edit points at a temporal resolution corresponding to the first image rate;

a processor programmed to construct the encoded image sequence according to the edited sequence of clips and using the source encoded image sequences, by;

clearing any information in the source encoded image sequences, used in the sequence of clips, indicative of images that are repeated when decoded; and

setting, in the encoded image sequence, information indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.

79. A computer-implemented method for processing data including an image sequence captured at a first image rate and encoded as an encoded image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

receiving the encoded image sequence;

receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence wherein the temporal positions used with the image index information correspond to the first image rate;

permitting an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

decoding the encoded image sequence, during playback of the encoded image sequence, using the image index information, to produce a decoded image sequence at the first image rate by not repeating images.

80. A computer-implemented method for processing data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate with information in the encoded image sequence indicative of images that are repeated when decoded, the method comprising:

receiving the encoded image sequence;

clearing all information in the encoded image sequence indicative of images that are repeated when decoded;

receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence wherein the temporal positions used with the image index information correspond to the first image rate;

permitting an editor to create a sequence of segments of encoded image sequences at a temporal resolution corresponding to the first image rate; and

decoding the encoded image sequence, during playback of the encoded image sequence, using the image index information, to produce a decoded image sequence at a second image rate different from the first image rate by repeating fields according to a cadenced based on the difference between the first image rate and the second image rate.

81. A computer-implemented method for editing encoded data including an image sequence captured at a first image rate and encoded as an image sequence having a second image rate different from the first image rate, with information in the encoded image sequence indicative of images that are repeated when decoded, to produce an encoded edited sequence, the method comprising:

receiving image index information for the encoded image sequence that is used to map a temporal position of an image in a decoded image sequence to a position of corresponding data in the encoded image sequence, wherein the temporal positions used with the image index information correspond to the first image rate;

permitting an editor to create an edited sequence of clips that reference source encoded image sequences using edit points at a temporal resolution corresponding to the first image rate; and

constructing the encoded edited sequence according to the edited sequence of clips and using the source encoded image sequences, including;

clearing any information in the source encoded image sequences, used in the sequence of clips, indicative of images that are repeated when decoded; and

setting, in the encoded image sequence, information indicative of images that are repeated when decoded according to a cadence associated with the difference between the first image rate and the second image rate.