Image and video indexing scheme for content analysis

• US RE43,896 E1
• Filed: 05/16/2008
• Issued: 01/01/2013
• Est. Priority Date: 11/08/2000
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Goodman, Rodney M., Keaton, Patricia A.
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US12/152,827
• Time in Patent Office: 1,691 Days
• Field of Search: 382/157, 382/218, 382/224, 382/232, 382/233, 382/240, 382/305, 382/125, 707/6, 707/9, 707/10, 707/101, 707/999.006, 707/999.101, 715/716, 715/722
• US Class Current: 382/218
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06K 9/6217 Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation regression analysis G06F17/18
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

The present invention provides an image and video indexing scheme for content analysis. According to the invention, a database of images or videos is compressed. By examining patterns in the compression scheme of each image or video, the present invention identifies the content of the data. In one embodiment, an unsupervised learning method is employed where each image or video is sub-divided into smaller blocks (8 pixels×8 pixels, for instance) and each of the smaller blocks is examined for its compression pattern. Then, the patterns associated with each of the smaller blocks is recorded for each of the images in the database and content is retrieved from the database by associating certain patterns or groups of patterns with certain content.

Claims

1. A method for retrieving one or more output images, the method comprising:

building a learned dictionary of transform codes by examining images in a database for one or more compression patterns and recording said compression patterns in the learned dictionary, wherein the one or more compression patterns comprise one or more transform codes that are learned from the images in the database;

receiving a request for one or more output images;

transforming the requested output images into requested transform codes; and

retrieving said output images from the database by comparing the requested transform codes to the learned transform codes.

2. The method of claim 1, wherein said receiving a request comprises:

receiving a text input;

locating one or more compression patterns associated with said text input; and

retrieving said output images associated with said compression patterns.

3. The method of claim 1, wherein said receiving a request comprises:

receiving an input image;

transforming the input image into input transform codes;

comparing said input transform codes to the learned transform codes; and

retrieving said output images from the database by comparing the input transform codes to the learned transform codes.

4. The method of claim 1, wherein said examining images in the database comprises:

dividing said images into one or more blocks; and

obtaining compression patterns by examining said blocks.

5. The method of claim 1 further comprising:

applying a latent variable modeling technique to obtain said transform codes.

6. The method of claim 5, wherein said latent variable modeling is a Gaussian latent variable modeling.

7. The method of claim 1, wherein retrieving said output images further comprises:

applying a Bayes decision rule.

8. A system for retrieving one or more output images, the system comprising:

one or more images in a database, wherein the images are configured to be examined for one or more compression patterns, wherein the one or more compression patterns comprise one or more transform codes that are learned from the images in the database;

a learned dictionary built from the transform codes learned from the images in the database by recording said compression patterns;

means for receiving a request for one or more output images;

means for transforming the requested output images into requested transform codes; and

means for retrieving said output images from the database by comparing the requested transform codes to the learned transform codes.

9. The system of claim 8, wherein said means for receiving a request comprises:

means for receiving a text input,

wherein one or more compression patterns are associated with said text input and are locatable using said text input, and

wherein said output images are associated with said compression patterns and are retrievable using said compression patterns.

10. The system of claim 9, wherein said means for receiving a request comprises:

means for receiving an input image,

wherein said input image is transformed into input transform codes,

wherein said input transform codes are compared to the learned transform codes, and

wherein said output images are retrieved by comparing the input transform codes to the learned transform codes.

11. The system of claim 8, wherein said images are divided into one or more blocks;

and said compression patterns are obtained by examining said blocks.

12. The system of claim 8, wherein a latent variable modeling technique is used to obtain said transform codes.

13. The system of claim 12, wherein said latent variable modeling is a Gaussian latent variable modeling.

14. The system of claim 8, wherein a Bayes decision rule is applied to retrieve said output images.

15. A non-transitory computer program product comprising a computer usable readable medium having computer readable program code embodied therein configured to obtain one or more output images, said computer program product comprising:

computer readable code for building a learned dictionary of transform codes by examining images in a database for one or more compression patterns and recording said compression patterns in the learned dictionary, wherein the one or more compression patterns comprise one or more transform codes that are learned from the images in the database;

computer readable code for receiving a request for one or more output images;

computer readable code for transforming the requested output images into requested transform codes; and

computer readable code for retrieving said output images from the database by comparing the requested transform codes to the learned transform codes.

16. The computer program product of claim 15, wherein said computer readable code for receiving a request comprises:

computer readable code for receiving a text input;

computer readable code for locating one or more compression patterns associated with said text input; and

computer readable code for retrieving said output images associated with said compression patterns.

17. The computer program product of claim 16, wherein said computer readable code for receiving a request comprises:

computer readable code for receiving an input image;

computer readable code for transforming the input image into input transform codes;

computer readable code for comparing said input transform codes to the learned transform codes; and

computer readable code for retrieving said output images from the database by comparing the input transform codes to the learned transform codes.

18. The computer program product of claim 15, wherein said computer readable code for examining images in a database comprises:

computer readable code for dividing said images into one or more blocks; and

computer readable code for obtaining said first compression patterns by examining said blocks.

19. The computer program product of claim 15, further comprising:

computer readable code for applying a latent variable modeling technique to obtain said transforms codes.

20. The computer program product of claim 19 wherein said latent variable modeling is a Gaussian latent variable modeling.

21. The computer program product of claim 15, wherein said computer readable code for retrieving said output images further comprises:

computer readable code for applying a Bayes decision rule.

22. A method comprising:

obtaining, by a computer based system for content analysis, an image or video from a database;

subdividing, by the computer based system, the image or video into blocks;

examining, by the computer based system, the blocks for transform codes used to compress the blocks; and

adding, by the computer based system, one or more of the transform codes to a dictionary that is configured to be used for image retrieval.

23. The computer-implemented method of claim 22, wherein said examining comprises using latent variable modeling to find the transform codes.

24. The computer-implemented method of claim 22, wherein said examining comprises obtaining compression patterns by examining said blocks.

25. The computer-implemented method of claim 22, wherein said examining comprises learning a collection or mixture of local linear subspaces over a set of the blocks.

26. The computer-implemented method of claim 22, wherein said examining comprises defining a probability density model over an input space and performing data partitioning and reduction within a maximum likelihood framework.

27. A method comprising:

obtaining, by a computer based system for content analysis, an image or video from a database;

examining, by the computer based system, said image or video by applying Gaussian latent variable modeling to the image or video to obtain a transform code used to compress at least a portion of the image or video; and

adding, by the computer based system, the transform code to a dictionary that is configured to be used for image retrieval.

28. The computer-implemented method of claim 27, wherein said examining comprises subdividing said image or video into one or more blocks and applying the Gaussian latent variable modeling to the one or more blocks.

29. The computer-implemented method of claim 27, wherein the Gaussian latent variable modeling defines a probability density model over an input space and performs data partitioning and reduction within a maximum likelihood framework.

30. A method comprising:

receiving, by a computer based system for content analysis, a request for one or more output images, the request comprising an input image;

transforming, by the computer based system, the input image into requested transform codes;

retrieving the one or more output images from a database by comparing the requested transform codes to learned transform codes in a learned dictionary, wherein the learned dictionary includes compression patterns that comprise the learned transform codes associated with the one or more output images in the database.

31. The computer-implemented method of claim 30, wherein said retrieving comprises applying a Bayes decision rule.

32. A system comprising:

a database comprising one or more images, wherein the one or more images embody one or more compression patterns, wherein the one or more compression patterns comprise one or more transform codes that are associated with the one or more images in the database;

a learned dictionary comprising the one or more transform codes associated with the one or more images in the database; and

wherein the learned dictionary is configured to be used to retrieve output images from the database based upon a comparison of requested transform codes to the one or more transform codes contained in the learned dictionary.

33. The system of claim 32, wherein said one or more images are divided into one or more blocks;

and

said compression patterns are based on said one or more blocks.

34. The system of claim 32, wherein said one or more transform codes in the learned dictionary are a function of a latent variable modeling technique applied to aspects of the one or more images.

35. The system of claim 32, wherein said one or more transform codes in the learned dictionary are a function of a Gaussian latent variable modeling technique applied to aspects of the one or more images.

36. A computer program product comprising a non-transitory computer readable medium having computer readable program code embodied therein that, in response to execution by a computing device, perform operations comprising:

obtaining an image or video from a database;

subdividing the image or video into blocks;

examining the blocks for transform codes used to compress the blocks; and

adding one or more of the transform codes to a dictionary that is configured to be used for image retrieval.

37. The computer program product of claim 36 further comprising computer readable program code that, in response to execution by a computing device, perform operations further comprising applying a latent variable modeling technique to obtain said transform codes.

38. The computer program product of claim 36 further comprising computer readable program code that, in response to execution by a computing device, perform operations further comprising applying Gaussian latent variable modeling to obtain said transform codes.

39. A non-transitory computer program product comprising a computer readable medium having computer readable program code embodied therein that, in response to execution by a computing device, perform operations comprising:

receiving a request for one or more output images, the request comprising an input image;

transforming the input image into requested transform codes; and

retrieving the one or more output images from a database by comparing the requested transform codes to learned transform codes in a learned dictionary, wherein the learned dictionary includes compression patterns that comprise the learned transform codes associated with the one or more output images in the database.

40. A system comprising:

means for obtaining an image or video from a database;

means for subdividing the image or video into blocks;

means for examining the blocks for transform codes used to compress the blocks; and

means for adding one or more of the transform codes to a dictionary that is configured to be used for image retrieval.

41. The system of claim 40, wherein said means for examining comprises means for using latent variable modeling to find the transform codes.

42. The system of claim 40, wherein said means for examining comprises means for obtaining compression patterns by examining said blocks.

43. The system of claim 40, wherein said means for examining comprises means for learning a collection or mixture of local linear subspaces over a set of the blocks.

44. The system of claim 40, wherein said means for examining comprises means for defining a probability density model over an input space and means for performing data partitioning and reduction within a maximum likelihood framework.

45. A system comprising:

means for obtaining an image or video from a database;

means for examining said image or video by applying Gaussian latent variable modeling to the image or video to obtain a transform code used to compress at least a portion of the image or video; and

means for adding the transform code to a dictionary that is configured to be used for image retrieval.

46. The system of claim 45, wherein said means for examining comprises means for subdividing said image or video into one or more blocks and means for applying the Gaussian latent variable modeling to the one or more blocks.

47. The system of claim 45, wherein means for examining comprises means for defining a probability density model over an input space and means for performing data partitioning and reduction within a maximum likelihood framework.

48. A system comprising:

means for receiving a request for one or more output images, the request comprising an input image;

means for transforming the input image into requested transform codes;

means for retrieving the one or more output images from a database by comparing the requested transform codes to learned transform codes in a learned dictionary, wherein the learned dictionary includes compression patterns that comprise the learned transform codes associated with the one or more output images in the database.

49. The system of claim 48, wherein said means for retrieving comprises means for applying a Bayes decision rule.