ACTIVE VISUAL ATTENTION MODELS FOR COMPUTER VISION TASKS

• US 20180060691A1
• Filed: 12/15/2016
• Published: 03/01/2018
• Est. Priority Date: 08/26/2016
• Status: Active Grant

• Current Assignee: Goodrich Corporation (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Goodrich Corporation (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Bernal, Edgar A., Reddy, Kishore K., Giering, Michael J.
• Granted Patent: US 10,452,951 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

An imaging method includes obtaining an image with a first field of view and first effective resolution and the analyzing the image with a visual attention algorithm to one identify one or more areas of interest in the first field of view. A subsequent image is then obtained for each area of interest with a second field of view and a second effective resolution, the second field of view being smaller than the first field of view and the second effective resolution being greater than the first effective resolution.

Claims

1. An image analysis system, comprising:

an imaging device;

a controller operatively connected to the imaging device; and

a non-transitory machine-readable memory disposed in communication with the controller and having instructions recorded thereon that, when read by the controller, cause the controller to;

obtain an image with a first field of view and first effective resolution;

analyze the image with a visual attention module to identify one or more areas of interest in the first field of view;

obtain one or more subsequent images for each area of interest with a second or more fields of view with each of the associated effective resolutions being greater than the first effective resolution; and

make a decision about the scene by aggregating information extracted from the images.

2. The system as recited in claim 1, wherein the instructions cause the controller to make the decision about the scene based on (a) identification of a type of object in the scene, and (b) identification of a type of activity taking place in the scene.

3. The system as recited in claim 1, wherein analyzing the image comprises analyzing raw image data.

4. The system as recited in claim 1, wherein analyzing the image comprises analyzing compressed domain image data.

5. The system as recited in claim 4, wherein the instructions further cause the controller to compress the image prior to analyzing the image with the visual attention algorithm.

6. The system as recited in claim 5, wherein the visual attention algorithm includes a plurality of compression parameters, wherein the instructions cause the controller to select a compression parameter from the plurality of compression parameters according to a decision made about the scene.

7. The system as recited in claim 1, wherein instructions further cause the controller to one or more of pan, zoom, or tilt the imaging device.

8. The system as recited in claim 1, wherein the instructions further cause the controller to adjust a positional parameter of an aerial vehicle carrying the imaging device.

9. The system recited in claim 1, wherein the instructions cause the controller to obtain the second image in real-time or near-real-time with analysis of the prior image.

10. The system as recited in claim 1, wherein the instructions cause the controller to generate one or more of a pan, zoom, and tilt positioning instructions for the imaging device based on the analysis of the acquired image.

11. The system as recited in claim 10, wherein the instructions further cause the controller to issue the positioning instructions to the imaging systems prior to obtaining the subsequent image.

12. The system as recited in claim 1, wherein the imaging device includes a visible wavelength band sensor array for obtaining images based on visible light incident upon the imaging device.

13. The system as recited in claim 1, wherein the imaging device includes a sensor array configured to obtain image data from one or more of a visible wavelength band, a SWIR wavelength band, a NIR wavelength band, a MIR wavelength band, or a thermal wavelength band.

14. The system as recited in claim 1, wherein the instruction cause the controller to cease imaging upon making a determination that sufficient information has been obtained from the processing of the acquired imagery.

15. An image analysis method for making a decision about a scene, comprising:

obtaining an image of the scene with a first field of view and first effective resolution;

analyzing the image with a visual attention algorithm to identify one or more areas of interest in the first field of view;

obtaining a subsequent image for one or more areas of interest, each of the subsequent images having an associated field of view that is smaller than the first field of view and an effective resolution that is greater than the first effective resolution; and

making a decision about the scene by aggregating information extracted from the obtained images.

16. The method as recited in claim 15, further comprising obtaining a plurality of subsequent images, each of the plurality of subsequent images having different fields of view and/or effective resolution from one another.

17. The method as recited in claim 15, wherein making a decision about the scene comprises one or more of (a) identifying a type of object in the scene, and (b) identifying a type of activity taking place in the scene.

18. The method as recited in claim 15, wherein analyzing the image comprises analyzing raw image data.

19. The method as recited in claim 15, wherein analyzing the image comprises analyzing image data in the compressed domain.

20. The method as recited in claim 19, further comprising compressing the image prior to analyzing the image with the visual attention algorithm, wherein the compression algorithms used to compress the image have parameters that are adjusted based on the type of decision being made about the scene.

21. The method as recited in claim 15, wherein the visual attention algorithm includes a plurality of compression parameters, wherein the method includes selecting a compression parameter according to a decision made about the scene.

22. The method as recited in claim 15, wherein obtaining a subsequent image comprises one or more of panning, zooming, or tilting an imaging apparatus carried by an aerial vehicle.

23. The method as recited in claim 15, wherein obtaining the subsequent image comprises obtaining the subsequent image in real-time or near-real-time.

24. The method as recited in claim 15, further comprising generating one or more of a pan, zoom, and tilt positioning instructions for an imaging system carried by an unmanned aerial vehicle based on the analysis of acquired imagery.

25. The method as recited in claim 24, further comprising adjusting a positional parameter of an aerial vehicle carrying an imaging device.

26. The method as recited in claim 15, wherein obtaining the image includes receiving electromagnetic radiation within one or more of a visible, SWIR, NIR, MIR, and thermal wavelength bands.

27. The method as recited in claim 15, further comprising ceasing imaging when the decision making process determines that sufficient information has been obtained.