Methods and systems for the detection of the insertion, removal, and change of objects within a scene through the use of imagery

• US 7,702,183 B1
• Filed: 05/17/2006
• Issued: 04/20/2010
• Est. Priority Date: 05/17/2006
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Neff, Michael G., Johnson, Ted L.
• Primary Examiner(s): Alavi; Amir
• Application Number:
  US11/383,914
• Publication Number:
  US 20100104185A1
• Time in Patent Office: 1,434 Days
• Field of Search: 382/103, 382/107, 382/164, 382/173, 382/181, 382/190, 382/209, 382218-220, 382276-277, 382/284, 382/287, 382/294, 348/169, 348/188, 348/263, 348/584, 348/591, 348/699, 348/E9.007
• US Class Current: 382/294
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters

Abstract

Methods and systems for detecting the insertion, removal, and change of objects of interest through the use of imagery are disclosed. In one embodiment, a method includes performing a scene registration including aligning image patterns in the first image to those in the second image; performing a feature content analysis to determine a likelihood of change for each pixel in the first and second images; performing a region identification to group pixels within the first and second images into one or more image regions based upon their likelihood of change; and performing an image region partitioning to prioritize the one or more image regions according to an image region score for each of the one or more image regions, the image region score being indicative of at least one of insertion, removal, and change of an object of interest within the common area of interest. Embodiments of the invention may advantageously provide an autonomous capability to detect the insertion, removal, and change of objects of interest.

Claims

1. A method for detecting insertion, removal, and change of an object of interest through the comparison of a first image and a second image containing a common area of interest, the method comprising using a computing device to identify and prioritize image regions within the images based on changes in feature content over time in a manner that is consistent with the insertion, removal and change of the object of interest, the identification and prioritization including:

performing a scene registration including aligning image patterns in the first image to those in the second image;

performing a feature content analysis to determine a General Pattern Change (GPC) likelihood of change over time for each pixel in the first and second images;

performing a region identification to group pixels within the first and second images into one or more image regions based upon their likelihood of change; and

performing an image region partitioning to prioritize the one or more image regions according to an image region score for each of the one or more image regions, the image region score being indicative of insertion, removal, and change over time of an object of interest within the common area of interest.

2. The method of claim 1, wherein performing a scene registration includes transforming at least one of the first and second images into a common reference frame, and aligning image patterns between the transformed first and second images.

3. The method of claim 2, wherein performing a feature content analysis to determine a likelihood of change for each pixel in the first and second images includes determining the GPC likelihood for each pixel in the transformed first and second images.

4. The method of claim 3, wherein determining the GPC likelihood includes determining the GPC likelihood using a GPC algorithm that includes

$\mathrm{Min}$

{ [ ∑

Si ⁢

⁢

∑

Rj ⁢

⁢

N Si , Rj ⁢

⁢

ln ⁡

( N Si , Rj N Si ) ] , ⁢

[ ∑

Si ⁢

⁢

∑

Rj ⁢

⁢

N Si , Rj ⁢

⁢

ln ⁡

( N Si , Rj N Rj ) ] } .

5. The method of claim 1, wherein performing a feature content analysis to determine a likelihood of change for each pixel in the first and second images includes:

creating a relative offset between a polygonal shape and the first and second images;

selecting one or more one or more pixels within the polygonal shape;

determining a GPC likelihood for the pixel at the center of the polygonal shape; and

repeating the creating, selecting, and determining for a plurality of different relative offsets.

6. The method of claim 5, wherein performing a feature content analysis includes, for a plurality of different polygonal shapes:

creating a relative offset between a polygonal shape and the one or more GPC likelihoods;

selecting one or more one or more pixels within the polygonal shape;

determining a GPC likelihood for the pixel at the center of the polygonal shape; and

repeating the creating, selecting, and determining for a plurality of different relative offsets and polygonal shapes.

7. The method of claim 1, wherein performing an image region partitioning includes performing an image region partitioning using an algorithm that includes:

$\underset{R_i=0}{\overset{N_R}{\sum <br><br>}}<br><br><br><br>\underset{C_j=0}{\overset{N_{\mathrm{Ri}}}{\sum <br><br>}}<br><br><br><br>\left(\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)<br><br>\ln <br><br>\left[\frac{\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)}{\underset{S_j=0}{\overset{N_{R_i}}{\sum <br><br>}}<br><br><br><br>\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(S_j,N_{R_i<br><br>G_k}\right)}\right]\right)$

Where f(a,b)=1, when a=b

f(a,b)=0, when a≠

b

N_R=number of regions

N_Ri=number of values in region i

R_imin=minimum value in region i

R_imax=maximum value in region i.

8. The method of claim 1, wherein performing a region identification includes:

creating a relative offset between a polygonal shape and the one or more GPC likelihoods;

selecting one or more one or more GPC likelihoods within the polygonal shape;

determining a region score for the polygonal shape; and

repeating the creating, selecting, and determining for a plurality of different relative offsets and one or more different polygonal shapes.

9. A method comprising using a computing device to detect at least one of insertion, removal, and change of objects of interest through the comparison of a first image and a second image containing a common area of interest, including:

determining a likelihood of change for each of a plurality of portions of the first and second images;

grouping the plurality of portions into one or more image regions based upon their likelihood of change; and

prioritizing the one or more image regions according to an image region score for each of the one or more image regions, the image region score being indicative of at least one of insertion, removal, and change of an object of interest within the common area of interest.

10. The method of claim 9, wherein prioritizing the one or more image regions includes performing an image region partitioning using an algorithm that includes:

$\underset{R_i=0}{\overset{N_R}{\sum <br><br>}}<br><br><br><br>\underset{C_j=0}{\overset{N_{\mathrm{Ri}}}{\sum <br><br>}}<br><br><br><br>\left(\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)<br><br>\ln <br><br>\left[\frac{\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)}{\underset{S_j=0}{\overset{N_{R_i}}{\sum <br><br>}}<br><br><br><br>\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(S_j,N_{R_i<br><br>G_k}\right)}\right]\right)$

Where f(a,b)=1, when a=b

f(a,b)=0, when a≠

b

N_R=number of regions

N_Ri=number of values in region i

R_imin=minimum value in region i

R_imax=maximum value in region i.

11. The method of claim 9, wherein determining a likelihood of change includes using a GPC algorithm that includes

$\mathrm{Min}$

{ [ ∑

Si ⁢

⁢

∑

Rj ⁢

⁢

N Si , Rj ⁢

⁢

ln ⁡

( N Si , Rj N Si ) ] , ⁢

[ ∑

Si ⁢

⁢

∑

Rj ⁢

⁢

N Si , Rj ⁢

⁢

ln ⁡

( N Si , Rj N Rj ) ] } .

12. The method of claim 9, wherein determining a likelihood of change includes, for a plurality of different polygonal shapes:

performing a sample region partitioning using each of the different polygonal shapes; and

determining a GPC likelihood for each of a plurality of sample regions of the sample region partitioning determined using each of the different polygonal shapes.

13. The method of claim 9, further comprising performing a scene registration including aligning image patterns in the first image to those in the second image.

14. The method of claim 13, wherein performing a scene registration includes transforming at least one of the first and second images into a common reference frame, and aligning image patterns between the transformed first and second images.

15. An image analysis system for detecting at least one of insertion, removal, and change of objects of interest through the comparison of a first image and a second image containing a common area of interest, the system comprising a computer programmed with:

a first component configured to determine a likelihood of change for a plurality of portions of the first and second images;

a second component configured to group the plurality of portions into one or more image regions based upon their likelihood of change; and

a third component configured to prioritize the one or more image regions according to an image region score for each of the one or more image regions, the image region score being indicative of at least one of insertion, removal, and change of an object of interest within the common area of interest.

16. The system of claim 15, wherein the third component is configured to prioritize the one or more image regions using an algorithm that includes:

$\underset{R_i=0}{\overset{N_R}{\sum <br><br>}}<br><br><br><br>\underset{C_j=0}{\overset{N_{\mathrm{Ri}}}{\sum <br><br>}}<br><br><br><br>\left(\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)<br><br>\ln <br><br>\left[\frac{\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(C_j,N_{R_i<br><br>G_k}\right)}{\underset{S_j=0}{\overset{N_{R_i}}{\sum <br><br>}}<br><br><br><br>\underset{G_k=R_{i_{\min }}}{\overset{R_{i_{\max }}}{\sum <br><br>}}<br><br><br><br>f<br><br>\left(S_j,N_{R_i<br><br>G_k}\right)}\right]\right)$

Where f(a,b)=1, when a=b

f(a,b)=0, when a≠

b

N_R=number of regions

N_Ri=number of values in region i

R_imin=minimum value in region i

R_imax=maximum value in region i.

17. The system of claim 15, wherein the first component is configured to determine a likelihood of change for a plurality of portions of the first and second images using a GPC algorithm that includes, for a plurality of different polygonal shapes:

performing a sample region partitioning using each of the different polygonal shapes; and

determining the GPC likelihood for each of a plurality of sample regions of the sample region partitioning determined using each of the different polygonal shapes.

18. The system of claim 15, wherein the computer is further programmed with a fourth component configured to perform a scene registration including aligning image patterns in the first image to those in the second image.

19. The system of claim 15, wherein the computer includes a memory device coupled to a processor, the processor being configured to execute computer-readable instructions, and wherein one of more of the first, second, and third components is configured as a module of computer-readable instructions stored within the memory device.

20. The system of claim 19, further comprising a data acquisition system configured to acquire the first and second images of the common area of interest, and to communicate the first and second images to the computer.