Representing an image with a posterized joint histogram

• US 6,522,779 B2
• Filed: 12/28/2000
• Issued: 02/18/2003
• Est. Priority Date: 12/15/2000
• Status: Expired due to Term

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: America Online Inc. (Warner Bros. Discovery, Inc.)
• Inventors: Pass, Gregory S., Wood, Frank
• Primary Examiner(s): Tran, Phucc
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US09/749,628
• Publication Number:
  US 20020102021A1
• Time in Patent Office: 782 Days
• Field of Search: 382/218, 382/209, 382/219-220, 382/168, 382/170, 382/162, 382/165, 382/305, 382/108, 382/224, 358/403, 358/522, 358/518, 345/700, 345/738, 345/835, 345/788, 345/419, 345/589, 348/405.1, 707/1-6, 707/404.1, 707/520-522, 707/515-516
• US Class Current: 382/170
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

Representing an image includes extracting information reflecting one or more characteristics of an image that then is used to compute a first histogram vector. The first histogram vector includes one or more vector elements, each representing information for a different characteristic extracted about the image. Multiple subsets are identified within at least one of the vector elements included in the first histogram vector and a second histogram is created. The second histogram vector includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector. Data within the vector elements of the second histogram vector represents the extracted image characteristics.

Claims

1. A method for representing an image, the method comprising:

extracting information reflecting one or more characteristics of an image;

computing a first histogram vector based on the information extracted, wherein the first histogram vector includes one or more vector elements, each representing information for a different characteristic extracted for the image;

identifying multiple subsets within at least one of the vector elements included in the first histogram vector, wherein the multiple subsets represent a range for a count of pixels for the vector element; and

creating a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector, wherein data within the vector elements of the second histogram vector represent the image characteristics extracted.

2. The method as in claim 1 wherein the first histogram vector is computed based on the information extracted, with the first histogram vector including one or more vector elements that each represent information for a different combination of characteristics extracted for the image.

3. The method as in claim 1 further comprising applying a weighting factor to the first histogram vector.

47. The method as in claim 1 wherein the extracted characteristics include at least one of color, edge density, texturedness, gradient magnitude, and rank features about pixels in the image.

48. The method as in claim 1 wherein the extracted characteristics include more than one of color, edge density, texturedness, gradient magnitude, and rank features about pixels in the image.

4. A method for representing an image, the method comprising:

extracting information about an image;

computing a histogram having one or more elements based on the information extracted about the image;

identifying multiple subsets within at least one of the elements of the histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

calculating based on the histogram and the subsets a posterized histogram that represents the image.

5. The method as in claim 4 wherein:

extracting information includes extracting more than one type of information about the image;

computing the histogram includes computing a joint histogram having one or more elements based on the more than one type of information;

identifying the multiple subsets includes identifying multiple subsets within at least one of the elements of the joint histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

calculating the posterized histogram includes calculating a posterized joint histogram based on the joint histogram and the subsets.

6. The method as in claim 5 further comprising applying a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and calculating the posterized joint histogram includes calculating based on the weighted joint histogram and the subsets.

7. The method as in claim 6 wherein the weighting factor is applied to the joint histogram based on more than one type of information extracted about the image.

8. The method as in claim 6 wherein applying the weighting factor includes:

extracting one or more additional types of information about the image that differ from the types of information used to compute the histogram; and

determining whether to modify the element in the joint histogram based on the additional types of information extracted about the image.

9. The method as in claim 8 wherein extracting one or more additional types of information includes extracting a centeredness feature about each pixel in the image and the determining includes determining whether to modify the one or more elements in the joint histogram based on the centeredness feature extracted about each pixel in the image.

10. The method as in claim 5 further comprising applying a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image that differs from the types of information used to compute the joint histogram before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and calculating the posterized joint histogram includes calculating based on the weighted joint histogram and the subsets.

11. The method as in claim 10 wherein the weighting factor is applied to the joint histogram based on more than one type of information extracted about the image that differs from the types of information used to compute the joint histogram.

12. The method as in claim 4 wherein the extracted information includes at least one feature of the image.

13. The method as in claim 4 wherein the extracted information includes more than one feature of the image.

14. The method as in claim 13 wherein the extracted information includes at least one of color, edge density, texturedness, gradient magnitude, and rank features about pixels in the image.

15. The method as in claim 13 wherein the extracted information includes more than one of color, edge density, texturedness, gradient magnitude, and rank features about pixels in the image.

16. The method as in claim 4 wherein:

computing the histogram includes computing a first histogram vector based on information extracted about the image, and

the first histogram vector includes one or more vector elements that each represent information for a different feature extracted about the image.

17. The method as in claim 16 wherein calculating the posterized histogram includes:

creating a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector; and

data within the vector elements of the second histogram vector represent the image features extracted.

18. The method as in claim 16 wherein computing the first histogram vector includes computing a first histogram vector based on the types of information extracted about the image, the first histogram vector including one or more vector elements each representing information for a different combination of features extracted about the image.

19. A computer program stored on a computer readable medium or a propagated signal for representing an image, the computer program comprising:

an extracting code segment that causes the computer to extract information about an image;

a computing code segment that causes the computer to compute a histogram having one or more elements based on the information extracted about the image;

an identifying code segment that causes the computer to identify multiple subsets within at least one of the elements of the histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

a calculating code segment that causes the computer to calculate a posterized histogram that represents the image based on the histogram and the subsets.

20. The computer program of claim 19 wherein:

the extracting code segment includes a code segment that causes the computer to extract more than one type of information about the image;

the computing code segment includes a code segment that causes the computer to compute a joint histogram having one or more elements based on the more than one type of information;

the identifying code segment includes a code segment that causes the computer to identify multiple subsets within at least one of the elements of the joint histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

the calculating code segment includes a code segment that causes the computer to calculate a posterized joint histogram based on the joint histogram.

21. The computer program of claim 20 further comprising a weighting code segment that causes the computer to apply a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and the calculating code segment includes a code segment that causes the computer to calculate based on the weighted joint histogram and the subsets.

22. The computer program of claim 21 wherein the weighting code segment includes a code segment that causes the computer to apply the weighting factor to the joint histogram based on more than one type of information extracted about the image.

23. The computer program of claim 21 wherein the weighting code segment includes:

a code segment that causes the computer to extract one or more additional types of information about the image that differ from the types of information used to compute the histogram; and

a code segment that causes the computer to determine whether to modify the element in the joint histogram based on the additional types of information extracted about the image.

24. The computer program of claim 20 further comprising a weighting code segment that causes the computer to apply a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image that differs from the types of information used to compute the joint histogram before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and the calculating code segment includes a code segment that causes the computer to calculate based on the weighted joint histogram and the subsets.

25. The computer program of claim 24 wherein the weighting code segment includes a code segment that causes the computer to apply the weighting factor to the joint histogram based on more than one type of information extracted about the image that differs from the types of information used to compute the joint histogram.

26. The computer program of claim 19 wherein the extracted information includes at least one feature of the image.

27. The computer program of claim 19 wherein the extracted information includes more than one feature of the image.

28. The computer program of claim 19 wherein the computing code segment includes a code segment that causes the computer to compute a first histogram vector based on information extracted about the image, wherein the first histogram vector includes one or more vector elements, each representing information for a different feature extracted about the image.

29. The computer program of claim 28 wherein the calculating code segment includes:

a code segment that causes the computer to create a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector, with data within the vector elements of the second histogram vector representing the image features extracted.

30. The computer program of claim 28 wherein:

the computing includes a code segment that causes the computer to compute a first histogram vector based on the types of information extracted about the image, and

the first histogram vector includes one or more vector elements each representing information for a different combination of features extracted about the image.

31. A computer program stored on a computer readable medium or a propagated signal for representing an image, the computer program comprising:

a code segment that causes the computer to extract information reflecting one or more characteristics of an image;

a code segment that causes the computer to compute a first histogram vector based on the information extracted, wherein the first histogram vector includes one or more vector elements, each representing information for a different characteristic extracted about the image;

a code segment that causes the computer to identify multiple subsets within at least one of the vector elements included in the first histogram vector, wherein the multiple subsets represent a range for a count of pixels for the vector element; and

a code segment that causes the computer to create a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector, wherein data within the vector elements of the second histogram vector represent the image characteristics extracted.

32. The computer program of claim 31 wherein the code segment that causes the computer to compute includes a code segment that causes the computer to compute the first histogram vector based on the information extracted, the first histogram vector including one or more vector elements, each representing information for a different combination of characteristics extracted about the image.

33. The computer program of claim 31 further comprising a weighting code segment that causes the computer to apply a weighting factor to the first histogram vector.

34. A system for representing an image, the system comprising:

an extracting component that is structured and arranged to extract information about an image;

a computing component that is structured and arranged to compute a histogram having one or more elements based on the information extracted about the image;

an identifying component that is structured and arranged to identify multiple subsets within at least one of the elements of the histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

a calculating component that is structured and arranged to calculate a posterized histogram that represents the image based on the histogram and the subsets.

35. The system of claim 34 wherein:

the extracting component extracts more than one type of information about the image;

the computing component computes a joint histogram based on the more than one type of information;

the identifying component identifies multiple subsets within at least one of the elements of the joint histogram, wherein the multiple subsets represent a range for a count of pixels for the element; and

the calculating component calculates a posterized joint histogram based on the joint histogram and the subsets.

36. The system of claim 35 further comprising a weighting component that is structured and arranged to apply a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and the calculating component calculates the posterized joint histogram based on the weighted joint histogram and the subsets.

37. The system of claim 36 wherein the weighting component is structured and arranged to apply the weighting factor to the joint histogram based on more than one type of information extracted about the image.

38. The system of claim 36 wherein:

the extracting component extracts one or more additional types of information about the image that differ from the types of information used to compute the histogram; and

the weighting component determines whether to modify the element in the joint histogram based on the additional types of information extracted about the image.

39. The system of claim 35 further comprising a weighting component that is structured and arranged to apply a weighting factor to a selected element in the joint histogram based on at least one type of information extracted about the image that differs from the types of information used to compute the joint histogram before calculating the posterized joint histogram such that the joint histogram represents a weighted joint histogram that reflects application of the weighting factor, wherein the multiple subsets represent a range for a weighted count of pixels for the selected element and the calculating component calculates the posterized joint histogram based on the weighted joint histogram and the subsets.

40. The system of claim 39 wherein the weighting component is structured and arranged to apply the weighting factor to the joint histogram based on more than one type of information extracted about the image that differs from the types of information used to compute the joint histogram.

41. The system of claim 34 wherein:

the computing component computes a first histogram vector based on information extracted about the image, and

the first histogram vector includes one or more vector elements, each representing information for a different feature extracted about the image.

42. The system of claim 41 wherein:

the calculating component is structured and arranged to create a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector, and

data within the vector elements of the second histogram vector represent the image features extracted.

43. The system of claim 41 wherein:

the computing component computes a first histogram vector based on the types of information extracted about the image, and

the first histogram vector includes one or more vector elements, each representing information for a different combination of features extracted about the image.

44. A system for representing an image, the system comprising:

an extracting component that is structured and arranged to extract information reflecting one or more characteristics of an image;

a computing component that is structured and arranged to compute a first histogram vector based on the information extracted, wherein the first histogram vector includes one or more vector elements, each representing information for a different characteristic extracted about the image;

an identifying component that is structured and arranged to identify multiple subsets within at least one of the vector elements included in the first histogram vector, wherein the multiple subsets represent a range for a count of pixels for the vector element; and

a creating component that is structured and arranged to create a second histogram vector that includes a vector element for each of the subsets identified within the vector elements included in the first histogram vector, wherein data within the vector elements of the second histogram vector represent the image characteristics extracted.

45. The system of claim 44 wherein the computing component is structured and arranged to compute the first histogram vector based on the information extracted, the first histogram including one or more vector elements, each representing information for a different combination of characteristics extracted about the image.

46. The system of claim 44 further comprising a weighting component that is structured and arranged to apply a weighting factor to the first histogram vector.