Systems and methods for analyzing image data using adaptive neighborhooding

• US 8,718,363 B2
• Filed: 01/16/2009
• Issued: 05/06/2014
• Est. Priority Date: 01/16/2008
• Status: Active Grant

• Current Assignee: Charles Stark Draper Laboratory Incorporated
• Original Assignee: Charles Stark Draper Laboratory Incorporated
• Inventors: Desai, Mukund, Mangoubi, Rami
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): YEH, EUENG NAN
• Application Number:
  US12/355,567
• Publication Number:
  US 20090180693A1
• Time in Patent Office: 1,936 Days
• Field of Search: 382/173, 382/268, 348/143
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/174 involving the use of two or more images

Abstract

The invention relates to methods and systems for extracting information about a scene from a set of image data by interdependently smoothing the data, segmenting the imaged scene into coherent elements by determining edges, and estimating attributes present within the scene. These methods and systems include attribute estimation, adaptive neighborhood adjustment and preferential use of different images or imaging modalities for information extraction.

Claims

1. A method for segmenting an image comprising:

storing, in a computer database, a set of image data representing a plurality of pixel locations within an image;

associating a respective neighborhood with each of the plurality of pixel locations in the stored image data;

identifying, by a computer, a plurality of segments within the image by iteratively performing the processes of;

individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods;

for each pixel location and associated neighborhood, obtaining a smoothed data value for the pixel location based on data values of the other pixel locations within the associated neighborhood; and

segmenting the image using the image data, the obtained smoothed data values, and the plurality of adjusted neighborhoods; and

outputting, by the computer, the plurality of segments within the image identified by the iterative process.

2. The method of claim 1, wherein a plurality of segments within the image comprises an edge value for each of the plurality of pixel locations.

3. The method of claim 1, wherein the processes of smoothing and segmenting are interdependent.

4. The method of claim 3, wherein the processes of smoothing and segmenting are substantially concurrent.

5. The method of claim 4, further comprising, for each of at least one attribute, identifying, by the computer, a plurality of attribute segments within the image by iteratively performing the processes of individually adjusting an orientation and at least one of a shape and a size defining an attribute neighborhood associated with each of the plurality of pixel locations and segmenting the image using the image data and the plurality of attribute neighborhoods.

6. The method of claim 5, wherein the at least one attribute includes at least one of texture, a model parameter, boundary shape and edge strength.

7. The method of claim 5, further comprising, for each of the at least one attribute of the image, adjusting an attribute value for said pixel location based on a data value of at least one other pixel location within the attribute neighborhood.

8. The method of claim 7, wherein, for each of the at least one attribute of the image, the process of individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods associated with each of the plurality of pixel locations comprises reducing the value of an energy function associated with an error metric.

9. The method of claim 8, wherein the error metric includes a penalty for at least one of a mismatch between the smoothed data value and a first transformation of the image data, discontinuity of a first transformation of the smoothed data value within each of the plurality of segments, a first transformation of the edge values, discontinuity of a second transformation of the edge values, discontinuity of a first transformation of the attribute value within each of the plurality of associated segments, a first transformation of the edge values associated with each of the at least one attribute, and discontinuity of a second transformation of the edge values associated with each of the at least one attribute.

10. The method of claim 7, further comprising utilizing, with the computer, a weighted combination of the image data for each of the at least one attribute in which the at least one attribute is better observed, wherein the process of utilizing is performed for each of the plurality of pixel locations.

11. The method of claim 10, wherein the processes of smoothing and segmenting are interdependent.

12. The method of claim 2, wherein the edge value is a matrix.

13. The method of claim 1, wherein the set of image data comprises data obtained via a plurality of imaging modalities.

14. The method of claim 1, further comprising specifying a set of prior information about the content of the image and segmenting the image using the image data, the plurality of adjusted neighborhoods and the set of prior information.

15. The method of claim 1, wherein the process of individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods associated with each of the plurality of pixel locations comprises reducing the value of an energy function associated with an error metric.

16. The method of claim 15, wherein the error metric includes a penalty for at least one of a mismatch between the smoothed data values and a first transformation of the image data, discontinuity of a first transformation of the smoothed data values within each segment, a first transformation of edge values, and discontinuity of a second transformation of the edge values.

17. The method of claim 1, comprising selecting, by the computer, at a plurality of the pixel locations, a parametric model from a plurality of parametric models for evaluating the image data at the respective pixel locations by identifying a parametric model for the pixel location that reduces the value of an energy function associated with an error metric.

18. The method of claim 1, wherein each neighborhood corresponds to a plurality of pixel locations surrounding its respective pixel location.

19. A system for segmenting an image, the system comprising:

a memory;

computerized image data representing a plurality of pixel locations within an image stored on the memory;

a processor configured to;

retrieve from memory a subset of the computerized image data;

associate a respective neighborhood with each of the plurality of pixel locations in the stored image data;

identify a plurality of segments within the image by iteratively performing the processes of;

individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods;

for each pixel location and associated neighborhood, obtaining a smoothed data value for the pixel location based on data values of the other pixel locations within the associated neighborhood; and

segmenting the image using the image data, the obtained smoothed data values, and the plurality of adjusted neighborhoods; and

output the plurality of segments within the imaged scene identified by the iterative process.

20. The system of claim 19, wherein the plurality of segments within the image comprises an edge value for each of the plurality of pixel locations.

21. The system of claim 20, wherein the processes of smoothing and segmenting are interdependent and substantially concurrent.

22. The system of claim 21, wherein the processor is further configured to, for each of at least one attribute, identify a plurality of attribute segments within the image by iteratively performing the processes of individually adjusting an orientation and at least one of a shape and a size defining an attribute neighborhood associated with each of the plurality of pixel locations, adjust an attribute value for said pixel location based on a data value of at least one other pixel location within the attribute neighborhood, and segment the image using the image data and the plurality of attribute neighborhoods.

23. The system of claim 22, wherein the at least one attribute includes at least one of texture, a model parameter, boundary shape and edge strength.

24. The system of claim 22, wherein the processor is further configured to utilize a weighted combination of the image data for each of the at least one attribute in which the at least one attribute is better observed, wherein the process of utilizing is performed for each of the plurality of pixel locations.

25. The system of claim 19, wherein the processor is further configured to specify a set of prior information about the content of the image and segment the image using the image data, the plurality of adjusted neighborhoods and the set of prior information.

26. The system of claim 19, wherein individually adjusting an orientation and at least one of a shape and a size of a neighborhood associated with each of the plurality of pixel locations comprises reducing the value of an energy function associated with an error metric.

27. The system of claim 19, wherein the processor is further configured to select at a plurality of the pixel locations, a parametric model from a plurality of parametric models for evaluating the image data at the respective pixel locations by identifying a parametric model for the pixel location that reduces the value of an energy function associated with an error metric.

28. A non-transitory computer readable medium storing computer executable instructions, which, when executed by a processor, cause the processor to carry out a method for segmenting an image comprising:

storing by the processor, in a computer database, a set of image data representing a plurality of pixel locations within an image;

associating a respective neighborhood with each of the plurality of pixel locations in the stored image data;

identifying by the processor a plurality of segments within the image by iteratively performing the processes of;

individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods;

for each pixel location and associated neighborhood, obtaining a smoothed data value for the pixel location based on data values of the other pixel locations within the associated neighborhood; and

segmenting the image using the image data, the obtained smoothed data values, and the plurality of adjusted neighborhoods; and

outputting by the processor the plurality of segments within the image identified by the iterative process.

29. The non-transitory computer readable medium of claim 28, wherein the plurality of segments within the image comprises an edge value for each of the plurality of pixel locations.

30. The non-transitory computer readable medium of claim 28, wherein the processes of smoothing and segmenting are interdependent and substantially concurrent.

31. The non-transitory computer readable medium of claim 30, storing instructions for causing the processor to, for each of at least one attribute, identify a plurality of attribute segments within the image by iteratively performing the processes of individually adjusting an orientation and at least one of a shape and a size defining an attribute neighborhood associated with each of the plurality of pixel locations, adjust an attribute value for said pixel location based on a data value of at least one other pixel location within the attribute neighborhood, and segment the image using the image data and the plurality of attribute neighborhoods.

32. The non-transitory computer readable medium of claim 31, wherein the at least one attribute includes at least one of texture, a model parameter, boundary shape and edge strength.

33. The non-transitory computer readable medium of claim 31, storing instructions for causing the processor to utilize a weighted combination of the image data for each of the at least one attribute in which the at least one attribute is better observed, wherein the process of utilizing is performed for each of the plurality of pixel locations.

34. The non-transitory computer readable medium of claim 28, storing instructions for causing the processor to specify a set of prior information about the content of the imaged scene and segment the imaged scene using the image data, the plurality of adjusted neighborhoods and the set of prior information.

35. The non-transitory computer readable medium of claim 28, wherein individually adjusting an orientation and at least one of a shape and a size of a plurality of the neighborhoods associated with each of the plurality of pixel locations comprises reducing the value of an energy function associated with an error metric.

36. The non-transitory computer readable medium of claim 28, storing instructions for causing the processor to select at a plurality of pixel locations, a parametric model from a plurality of parametric models for evaluating the image data at the respective pixel locations by identifying a parametric model for the pixel location that reduces the value of an energy function associated with an error metric.