Methods and apparatus for generating, including and using information relating to archived audio/video data

• US 6,947,598 B2
• Filed: 04/20/2001
• Issued: 09/20/2005
• Est. Priority Date: 04/20/2001
• Status: Expired due to Term

• Current Assignee: CDN Innovations, LLC (IP Investments Group LLC)
• Original Assignee: Front Porch Digital Inc. (Incentra Solutions, Inc.)
• Inventors: Gupta, Bobby R., Martucci, Stephen A., Fendt, Melanie R., Yogeshwar, Jay, Venzie, Frederick M. III
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US09/839,354
• Publication Number:
  US 20030210821A1
• Time in Patent Office: 1,614 Days
• Field of Search: 382/232, 382/239, 382/247, 382/298, 375/240.13, 375/240.01, 375/240.12, 358/426.03, 704/500, 341/51, 341/50, 341/60, 341/76
• US Class Current: 382/232
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence

Abstract

Archival storage and retrieval of audio/video information is described. Audio and/or video information is digitized, compressed and stored in an intermediate archive format (IAF), which preserves the content at a high-enough quality for subsequent retrieval and conversion into various formats required at the time of use or distribution. A single capture operation is performed with ancillary metadata being added to facilitate subsequent searching, indexing and format conversion. Captured data content is catalogued and indexed at or subsequent to the creation of an IAF file that includes the archived information. The IAF includes a family of audio-video digital encoding formats based on public standards. The encoding format used in any particular application is determined at encoding time from information provided by the archive system user. At encoding, the particular encoding scheme is selected to optimize a tradeoff between storage constraints and end use quality requirements.

Claims

1. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

analyzing said image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity;

encoding said image data according to a first encoding format to generate first encoded image data; and

storing with the first encoded image data encoding complexity level information indicating at least one determined level of encoding complexity associated with the first encoded image data.

2. The method of claim 1, further comprising the steps of:

performing an automated image content analysis operation on at least one image represented by said image data; and

storing, with the first encoded image data, image content description information generated by performing said content analysis operation.

3. The method of claim 2, further comprising the steps of:

receiving image content information from a user

of the system; and

storing, with the first encoded image data and the image content description information generated by performing said content analysis operation, the image content description information received from the system user.

4. The method of claim 1, further comprising the step of:

selecting, based on the determined encoding complexity level information, an image represented by the first encoded image data, to be viewed after decoding.

5. The method of claim 4, further comprising the step of:

decoding the encoded image data representing the selected image to generate decoded image data; and

displaying the decoded selected image on a display device.

6. The method of claim 7, wherein said step of decoding the encoded image data is performed as part of said encoding step.

7. The method of claim 1, further comprising the step of:

retrieving the first encoded image data and encoding complexity level information; and

using the retrieved encoding complexity level information to identify at least one data format suitable for distributing an image represented by the first encoded image data, the identified data format being different from the first encoding format.

8. The method of claim 6, further comprising the step of:

presenting to a user of the system a plurality of data formats suitable for distributing the image represented by the first encoded image data, the identified data format being one of said plurality of presented data formats;

receiving from the user information selecting one of the presented data formats for use in distributing the image; and

converting at least a portion of the first encoded image data from the first encoding format to the user selected data format.

9. The method of claim 1, wherein the step of analyzing image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity includes:

generating an activity measure for at least one image represented by said image data.

10. The method of claim 1, wherein the step of analyzing image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity includes:

generating a measure of the luminance variance throughout at least one image represented by said image data.

11. The method of claim 1, wherein the step of analyzing image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity includes:

generating a measure of the chrominance variance throughout at least one image represented by said image data.

12. The method of claim 1, wherein the step of analyzing image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity includes:

generating a measure of the motion between at least two complete frames, each frame corresponding to a different image.

13. The method of claim 1, further comprising:

generating true motion vectors indicating motion between a first image and a second image;

generating in accordance with the first encoding format, as part of said step of encoding said image data, a set of motion vectors indicating motion between said first image and said second image, said set of motion vectors including at least some motion vectors which are different form said true motion vectors; and

storing the true motion vectors with the first encoded image data which includes said set of motion vectors.

14. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

analyzing said image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity;

encoding said image data according to a first encoding format to generate first encoded image data;

storing with the first encoded image data encoding complexity level information indicating at least one determined level of encoding complexity associated with the first encoded image data;

retrieving the first encoded image data and encoding complexity level information from the storage device;

decoding the first encoded image data to generate decoded image data;

determining at least one encoding parameter to be used to re-encode the decoded image data as a function of the retrieved encoding complexity level information; and

re-encoding the generated decoded image data using the encoding parameter determined as a function of the retrieved encoding complexity level information.

15. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

analyzing said image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity;

encoding said image data according to a first encoding format to generate first encoded image data;

storing with the first encoded image data encoding complexity level information indicating at least one determined level of encoding complexity associated with the first encoded image data;

retrieving the first encoded image data and image content description information from the storage device;

decoding the first encoded image data to generate decoded image data;

determining at least one encoding parameter to be used to re-encode the decoded image data as a function of the retrieved image content description information; and

re-encoding the generated decoded image data using the encoding parameter determined as a function of the retrieved image content description information.

16. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

analyzing said image data to be encoded to determine, for each image represented by the image data, a level of encoding complexity;

encoding said image data according to a first encoding format to generate first encoded image data;

storing with the first encoded image data encoding complexity level information indicating at least one determined level of encoding complexity associated with the first encoded image data; and

selecting the first encoding format from a plurality of supported encoding formats, as a function of the determined level of encoding complexity.

17. The method of claim 16, further comprising the step of:

receiving data storage limitation information; and

wherein the step of selecting the first encoding format is also performed as a function of the received data storage limitation information.

18. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

performing an automated scene analysis operation on said image data to be encoded to generate image content information said image content information indicating the type of scene depicted by said image data to be encoded;

encoding said image data according to a first encoding format to generate first encoded image data; and

storing the generated image content information in a file with the first encoded image data.

19. A method of operating a system to process image data for storage and retrieval, the method comprising the steps of:

performing an automated scene analysis operation on said image data to be encoded to generate image content information;

encoding said image data according to a first encoding format to generate first encoded image data;

storing the generated image content information in a file with the first encoded image data;

receiving additional image content information from a user of the system;

storing the additional image content information in said file with the first encoded image data;

retrieving from storage the stored first encoded image data, said generated content information, and said additional content information;

selecting a second encoding format to be used for outputting images represented by said first encoded image data as a function of at least one of said generated content information and said additional content information; and

re-encoding said first encoded image data to said second encoding format to generate second encoded image data.

20. The method of claim 19, wherein the generated image content information indicates the pictorial content of an image.

21. The method of claim 20, wherein the additional image content information includes a description of the pictorial content of at least one image.

22. The method of claim 20, wherein the step of re-encoding said first encoded image data includes:

decoding said first encoded image data to generate decoded image data; and

re-encoding said first encoded image data using at least one encoding parameter generated as a function of said generated image content information.

23. A system for processing image data for storage and retrieval purposes, the system comprising:

a scene analysis module for performing scene analysis on said image data to generate image content information;

means for receiving additional image content information from a user of the system;

an encoder for encoding said image data according to a first encoding format to generate first encoded image data; and

a storage device for storing the first encoded image data, said generated image content information and said additional image content information in a file.

24. A system for processing image data for storage and retrieval purposes, the system comprising:

a scene analysis module for performing scene analysis on said image data to generate image content information;

means for receiving additional image content information from a user of the system;

an encoder for encoding said image data according to a first encoding format to generate first encoded image data;

a storage device for storing the first encoded image data, said generated image content information and said additional image content information in a file;

means for retrieving from storage the stored first encoded image data, said generated content information, and said additional content information;

means for selecting, as a function of at least one of said generated content information and said additional content information, a second encoding format to be used for outputting images represented by said first encoded image data; and

a second encoder for re-encoding said first encoded image data to said second encoding format to generate second encoded image data.

25. A system for processing data including at least one of image data and audio data, the system comprising:

an analysis module for analyzing data to be encoded and to assign one of a plurality of encoding levels of complexity to the data to be encoded;

an encoder for generating encoded data from said data to be encoded;

a file wrapper module for incorporating an encoding complexity level identifier indicating the encoding complexity level assigned to said data to be encoded and said encoded data into a single file; and

a data storage device for storing said single file.

26. A system for processing encoded image data, the system comprising:

an analysis module for analyzing data to be encoded and to assign one of a plurality of encoding levels of complexity to the data to be encoded;

an encoder for generating encoded data from said data to be encoded;

a file wrapper module for incorporating an encoding complexity level identifier indicating the encoding complexity level assigned to said data to be encoded and said encoded data into a single file; and

a data storage device for storing said single file; and

a preview module coupled to said encoder and said analysis module for displaying a subset of the images represented by encoded data generated by said encoder, the displayed images being selected for display as a function of encoding complexity level information associated with said encoded data.

27. The system of claim 26, wherein the preview module selects images represented by image data assigned a higher than average encoding complexity level for display.