Method and apparatus for image processing by generating probability distribution of images

• US 6,704,454 B1
• Filed: 06/08/2000
• Issued: 03/09/2004
• Est. Priority Date: 07/23/1999
• Status: Expired due to Fees

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Spence, Clay Douglas, Parra, Lucas
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US09/589,715
• Time in Patent Office: 1,370 Days
• Field of Search: 382/232, 382/236, 382/238-240, 382/242, 382/248, 382/250, 358/432, 358/433, 358/261.2, 358/261.3, 348/384.1, 348/394--39, 348/400--40, 348/407--41, 348/420--42, 348/425.2, 348/430--43, 375/240-2-2, 375/240-1-2, 375/240-8-2, 375/240-2-2, 341/51, 341/63, 341/65, 341/67, 341/79, 341/107, 708/300, 708/308, 708/313, 708/317, 708/400-405, 708/203
• US Class Current: 382/240
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

An apparatus and a concomitant method for modeling local and non-local information in an image to compute an image probability distribution for the image is disclosed. In one embodiment, such an image probability distribution is determined in an object recognition system.

Claims

1. A method for computing an image probability distribution for an image, the image containing local and non-local information, said method comprising the steps of:

(a) decomposing the image into a feature pyramid and a subsampled feature pyramid to model local information in the image;

(b) modeling non-local information in the image with a plurality of labels;

(c) computing the image probability distribution using said plurality of labels and at least one of said feature pyramid and said subsampled feature pyramid; and

(d) performing image processing on the image using the computed image probability distribution.

2. The method of claim 1, wherein said decomposing step (a) comprises the steps of:

(a1) decomposing the image into a low-pass pyramid having a plurality of levels; and

(a2) extracting features at each level of said low-pass pyramid to create said feature pyramid having a plurality of levels corresponding to said plurality of levels of said low-pass pyramid.

3. The method of claim 2 wherein said low-pass pyramid is a gaussian pyramid.

4. The method of claim 1 wherein said computing step (c) comprises the step of (c1) factoring the image probability distribution over said plurality of labels and at least one position at each level in at least one of said feature pyramid and said subsampled feature pyramid.

5. The method of claim 4, wherein said computing step (c) is performed in accordance with:

$\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I\right)\propto <br><br>\underset{A}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left\{\underset{l=0}{\overset{L-1}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{x\in <br><br>I_{l+1}}{\overset{\text{\hspace{1em}<br><br>}}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(g_l<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)|f_{l+1}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right),A\right)\right\}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I_L|A\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(A\right)$

where Pr(I) represents the image probability distribution, I represents the image, A represents said plurality of labels, f_l+1(x) represents a feature vector at position x of level l+1 of said feature pyramid, g_l(x) represents a feature vector at position x of level l of said subsampled feature pyramid, and L represents the number of levels of said feature and subsampled feature pyramids.

6. The method of claim 1, wherein said plurality of labels is structured as a label pyramid having a plurality of levels.

7. The method of claim 6, wherein each label is conditionally dependent upon a label at the next higher level of said plurality of labels.

8. The method of claim 7, wherein computing step (c) is performed in accordance with:

$\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I\right)\propto <br><br>\underset{A_0<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},A_{L-1}}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{l=0}{\overset{L-1}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{x\in <br><br>I_{l+1}}{\overset{\text{\hspace{1em}<br><br>}}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left[\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(g_l<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)|f_{l+1}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right),a_l<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(a_l<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)|a_{l+1}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)\right)\right]<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I_L\right)$

where Pr(I) represents the image probability distribution, I represents the image, f_l+1(x) represents a feature vector at position x of level l+1 of said feature pyramid, g_l(x) represents a feature vector at position x of level l of said subsampled feature pyramid, a_l(x) represents said label at position x of level l of said label pyramid, a_l+1(x) represents said label at position x of level l+1 of said label pyramid, L represents the number of levels of said feature pyramid and said subsampled feature pyramid, and A_l represents an label image or said plurality of labels at level l of said label pyramid.

9. The method of claim 8, wherein Pr(G_l(x)|f_l+1(x), a_l(x)) and Pr(a_l(x)|a_l+1(x)) for each level l and position x are determined using at least one parameter, where said at least one parameter is matched to the image with an EM (expectation-maximization) method.

10. The method of claim 1 wherein said performing step (d) comprises the steps of:

(d1) associating image probability distributions for at least two classes; and

(d2) identifying an object in the image if the image probability distribution of one of at least two classes exceeds a threshold level.

11. The method of claim 1 wherein said performing step (d) comprises the step of (d1) allocating fewer bits at images having a higher image probability distribution.

12. The method of claim 1 wherein said performing step (d) comprises the steps of:

(d1) detecting the presence of noise in the image; and

(d2) estimating a refined image with said noise removed.

13. A method for detecting an object in an image having local and non-local information, said method comprising the steps of:

(a) decomposing the image into a feature pyramid and subsampled feature pyramid to model local information in the image;

(b) implementing a plurality of labels to model non-local information in the image;

(c) computing an image probability distribution from said feature pyramid, said subsampled feature pyramid and said plurality of labels; and

(d) detecting the object in the image using the image distribution.

14. The method of claim 13 wherein said computing step (c) comprises the step of (c1) factoring the image probability distribution over said plurality of labels and at least one position at each level in at least one of said feature pyramid and said subsampled feature pyramid.

15. The method of claim 13 wherein said computing step (c) is performed in accordance with:

$\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I\right)\propto <br><br>\underset{A}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left\{\underset{l=0}{\overset{L-1}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{x\in <br><br>I_{l+1}}{\overset{\text{\hspace{1em}<br><br>}}{\prod <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(g_l<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right)|f_{l+1}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x\right),A\right)\right\}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(I_L|A\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{Pr}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(A\right)$

where Pr(I) represents the image probability distribution, I represents the image, A represents said plurality of labels, f_l+1(x) represents a feature vector at position x of level l+1 of said feature pyramid, g_l(x) represents a feature vector at position x of level l of said subsampled feature pyramid, and L represents the number of levels of said feature and subsampled feature pyramids.

16. The method of claim 13 wherein said detecting step (d) comprises the steps of:

(d1) associating image probability distributions for at least two classes; and

(d2) identifying an object in the image if the image probability distribution of one of at least two classes exceeds a threshold level.

17. An apparatus for detecting objects in an image having local and non-local information, said apparatus comprising:

a pyramid generator for generating a feature pyramid and a subsampled feature pyramid from the input image;

a hierarchical image probability (HIP) module, coupled to said pyramid generator, for implementing a plurality of labels to model non-local information in the image, and computing a image probability distribution from said feature pyramid, said subsampled feature pyramid and said plurality of labels; and

an object processor, coupled to said HIP module, for detecting objects in the image from said image distribution.

18. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

decomposing an image containing local and non-local information into a feature pyramid and a subsampled feature pyramid, where at least one of said feature pyramid and said subsampled feature pyramid models local information in said image;

modeling non-local information in said image with a plurality of labels;

computing the image probability distribution using said plurality of labels and at least one of said feature pyramid and said subsampled pyramid; and

performing image processing on said image using the computed image probability distribution.