Methods and apparatus for indexing and archiving encoded audio/video data

• US 20030206717A1
• Filed: 04/20/2001
• Published: 11/06/2003
• Est. Priority Date: 04/20/2001
• Status: Active Grant

• Current Assignee: Rovi Technologies Corporation (Adeia Inc.)
• Original Assignee: Front Porch Digital Inc. (Genstar Capital LLC)
• Inventors: Gupta, Bobby R., Martucci, Stephen A., Fendt, Melanie R., Yogeshwar, Jay, Venzie, Frederick M. III
• Granted Patent: US 7,110,664 B2
• Time in Patent Office: Days
• US Class Current: 386/69
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 9/7921 for more than one processing mode
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8047 using transform coding
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal
• CPC Class Codes: H04N 9/8233 the additional signal being a character code signal

Abstract

Archival storage and retrieval of audio/video information is described. Audio and/or video information is digitized, compressed and stored in an intermediate archive format (IAF), which preserves the content at a high-enough quality for subsequent retrieval and conversion into various formats required at the time of use or distribution. A single capture operation is performed with ancillary metadata being added to facilitate subsequent searching, indexing and format conversion. Captured data content is catalogued and indexed at or subsequent to the creation of an IAF file that includes the archived information. The IAF includes a family of audio-video digital encoding formats based on public standards. The encoding format used in any particular application is determined at encoding time from information provided by the archive system user. At encoding, the particular encoding scheme is selected to optimize a tradeoff between storage constraints and end use quality requirements.

Claims

1. A method of operating a system to process image data, the method comprising:

encoding said image data to generate first encoded image data;

decoding said first encoded image data to generate first decoded image data;

analyzing the content of the first decoded image data to generate indexing information; and

storing the first encoded image data with the generated indexing information on a digital storage medium.

2. The method of claim 1, wherein the previously generated indexing information is a first set of indexing information, the method further comprising:

retrieving the stored first encoded image data from the digital data storage device;

decoding said first encoded image data for a second time to generate decoded image data;

analyzing the content of the first decoded image data to generate a second set of indexing information; and

storing the second set of indexing information with the first encoded image data and the previously generated indexing information on the digital storage media.

3. The method of claim 2, further comprising:

retrieving the stored first encoded image data for a second time;

decoding said first encoded image data for a third time to generate decoded image data;

analyzing the content of the first decoded image data to generate a third set of indexing information; and

storing the third set of indexing information with the first encoded image data, and the first and second sets of indexing information on the digital storage media.

4. The method of claim 1, further comprising:

receiving image data information relating to said image data prior to performing said encoding step; and

storing, on the digital storage media, the received image data information with the first encoded image data and the generated indexing information.

5. The method of claim 4, wherein the received image data information, first encoded image data, and the generated indexing information are stored together in a first file on the digital storage media.

6. The method of claim 5, wherein the digital storage media further includes:

a second file which includes second encoded image data, second generated indexing information and second received image data; and

a file directory, the file directory including;

a copy of said generated indexing information included in the first file; and

a copy of the second generated indexing information included in the second file.

7. A data processing method, the method comprising:

receiving a first set of encoded data;

decoding the first set of encoded data to generate first decoded data;

analyzing the content of the first decoded data to generate first indexing information; and

storing the first set of encoded data with the first indexing information in a first file on a digital storage medium.

8. The method of claim 7, further comprising the step of:

receiving a second set of encoded data;

decoding the second set of encoded data to generate second decoded data;

analyzing the content of the second decoded data to generate second indexing information; and

storing the second set of encoded data with the second indexing information in a second file on the digital storage medium.

9. The method of claim 8, further comprising:

storing in a directory file on the digital storage medium a copy of the first indexing information and a copy of the second indexing information.

10. The method of claim 9, further comprising the step of:

retrieving the first set of encoded data from the digital storage medium;

decoding the first set of encoded data for a second time to generate third decoded data;

analyzing the content of the third decoded data to generate a third set of indexing information; and

storing the third set of indexing information in a file on the digital storage medium with the first set of encoded data and the first indexing information.

11. The method of claim 10, further comprising the step of:

adding a copy of the third set of indexing information to the directory file on the digital storage medium.

12. The method of claim 11, further comprising:

retrieving the first set of encoded data from the digital storage medium for a second time;

decoding the first set of encoded data for a third time to generate fourth decoded data;

analyzing the content of the fourth decoded data to generate a fourth set of indexing information; and

storing the fourth set of indexing information in a file on the digital storage medium with the first set of encoded data, the first indexing information, and the third set of indexing information.

13. The method of claim 12, further comprising:

adding a copy of the fourth set of indexing information to the directory file on the digital storage medium.

14. The method of claim 10, wherein the step of analyzing the content of the third decoded data includes:

analyzing the third decoded data for at least some information which is different than the information for which the first decoded data was analyzed.

15. The method of claim 12,

wherein the step of analyzing the content of the third decoded data includes:

analyzing the third decoded data for at least some information which is different than the information for which the first decoded data was analyzed; and

wherein the step of analyzing the content of the fourth decoded data includes;

analyzing the fourth decoded data for at least some information which is different than the information for which the first and third decoded data was analyzed.

16. The method of claim 15, wherein the first set of encoded data includes at least one of encoded audio data and encoded video data.

17. The method of claim 9, further comprising the step of:

receiving search information from a system user;

accessing the directory file to identify stored encoded data corresponding to the received search information; and

retrieving from the digital storage medium encoded data identified as corresponding to the received search information.

18. The method of claim 17, further comprising the step of:

converting the retrieved encoded data from a first encoding format to a second encoding format thereby generating data encoded according to the second encoding format, the second encoding format being different from the first encoding format; and

supplying the data encoded according to the second encoding format to a data distribution system.

19. The method of claim 18, further comprising the step of:

converting the retrieved encoded data from the first encoding format to a third encoding format thereby generating data encoded according to the third encoding format, the third encoding format being different from the first and second encoding formats; and

supplying the data encoded according to the third encoding format to the data distribution system.

20. An apparatus for indexing encoded data including at least one of encoded audio data and encoded image data, the apparatus comprising:

a decoder module for decoding the encoded data to generate first decoded data;

an indexing module for performing content analysis on the decoded data to generate information content indexing information; and

a storage device for storing the generated information content indexing information in a file with the encoded data.

21. The apparatus of claim 20, further comprising:

a retrieval module for retrieving from the storage device the encoded data included in the first file and for supplying the retrieved encoded data to the decoder module;

wherein the indexing module indexes decoded data generated by decoding the retrieved encoded data to produce second information content information, the second information content indexing information including information on different features than the previously generated information content indexing information; and

means for appending the second information content indexing information to the file including the retrieved encoded data.

22. The apparatus of claim 21, further comprising:

means for storing a copy of the indexing information included in said file with the encoded data in a file directory stored in the same storage device as the file including the encoded data.

23. The apparatus of claim 22, wherein the information content indexing information identifies physical objects included in images represented by said encoded data.

24. The apparatus of claim 23, wherein the information content indexing information identifies words included in songs stored using said encoded data.

25. A digital data storage device, comprising:

a plurality of data files, each data file including index information and at least one of encoded image data and encoded audio data; and

a file directory, the file directory including a copy of the index information included in each one of the plurality of data files.

26. The digital data storage device of claim 25,

wherein each of said plurality of data files includes encoded image data and wherein said index information included in each data field includes image content information.

27. The digital data storage device of claim 26, wherein the image content information in at least some of the plurality of data files includes image content information added to the data file after the initial creation of the data file.

28. The digital data storage device of claim 27, the encoded image data included in said plurality of data files is encoded using a plurality of different image encoding formats, the encoded image data in each particular data file being encoded according to a single image encoding format.

29. The digital data storage device of claim 28, wherein the plurality of different image encoding formats are encoding formats for which public standards exist.

30. The digital data storage device of claim 29, wherein the plurality of different image encoding formats include MPEG-2, JPEG and DV.

31. The digital data storage device of claim 29, wherein the index information included in each of the plurality of data files is encoded according to the same encoding format.

32. The digital data storage device of claim 31, wherein the format used to encode said index information is not a public standards based encoding format.

33. The digital data storage device of claim 31, wherein the index information is included in each data file in a manner that results in the index information being discarded when processed by a decoder which does not support decoding of the format used to encode said index information.

34. A method of processing an encoded data file, the method comprising the step of:

searching the encoded data file for content information which can be obtained by examining encoded data included in said file without fully decoding said encoded data;

retrieving from the data file encoded data satisfying a set of search criteria; and

fully decoding the retrieved encoded data.

35. The method of claim 34, wherein the step of searching the encoded data file includes the step of:

performing a variable length decoding operation to produce data including DCT coefficients;

examining the DCT coefficients to determine if the search criteria are satisfied.

36. The method of claim 35, wherein the step of retrieving from the data file encoded data includes:

retrieving one but not all of a plurality of encoded data streams included in said encoded data file.

37. The method of claim 36, wherein the step of retrieving from the data file encoded data includes:

retrieving some but not all of the encoded data included in an encoded data stream.

38. The method of claim 34, further comprising the step of:

performing an indexing operation on the decoded data produced by decoding the retrieved encoded data to thereby generate indexing information; and

storing the indexing information in the data file from which the encoded data was retrieved.

39. A method of indexing encoded information, the method comprising the steps of:

retrieving encoded information from a data file;

performing a partial decoding operation on the encoded information to generate partially decoded data; and

performing an indexing operation on said partially decoded data to generate indexing information.

40. The method of claim 39, further comprising the step of:

storing the generated indexing information in the data file from which the encoded information was retrieved.

41. The method of claim 39, wherein the step of performing a partial decoding operation includes the step of:

performing a decoding operation to generate DCT coefficients from the encoded data; and

wherein the step of performing an indexing operation includes examining the DCT coefficients to assess the information content of the retrieved encoded data.

42. The method of claim 40, wherein the step of storing the generated indexing information includes appending the indexing information to said data file.

43. The method of claim 39, wherein the step of retrieving encoded information includes:

retrieving one but not all of a plurality of encoded data streams included in said encoded data file.

44. The method of claim 39, wherein the step of retrieving from the data file encoded data includes:

retrieving some but not all of the encoded data included in an encoded data stream.