Method and system for performing adaptive image acquisition

• US 20060077255A1
• Filed: 08/10/2005
• Published: 04/13/2006
• Est. Priority Date: 08/10/2004
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Cheng, Hui
• Granted Patent: US 8,102,423 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/1965 the vehicle being an aircraft
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

An adaptive image acquisition system and method that generates virtual view of a surveillance scene to a user (operator), in which, the user operates the system. Through viewing the virtual view, the user controls sensors that create the virtual view. The sensors comprise at least one first sensor having a higher resolution than at least one second sensor. Images from the second sensor are processed to create an image mosaic that is overlaid with images from the higher resolution first sensor. In one embodiment of the invention, the first sensor is moved using Saccade motion. In another embodiment of the invention, a user'"'"'s intent is used to control the Saccade motion.

Claims

1. An adaptive image acquisition system comprising:

at least one imaging sensor for generating at least one first image having a higher resolution than an at least one second image;

a registration module for registering the at least one first image and the at least one second image;

a virtual view generation module for creating a virtual view of a scene imaged by the at least one imaging sensor using the registered images; and

a cognitive interface, coupled to the virtual view generation module, for enabling a user to control the plurality of imaging sensors while viewing the virtual view.

2. The system of claim 1 wherein the at least one first image is produced by a first sensor and the at least one second image is produced by a second sensor.

3. The system of claim 1 wherein the second image is formed by sub-sampling the at least one first image.

4. The system of claim 2 wherein the at least one sensor comprises at least one first sensor is a narrow field of view camera and the at least one second sensor is a wide field of view camera.

5. The system of claim 2 wherein the at least one first sensor is moved using Saccade motion.

6. The system of claim 5 wherein the virtual view comprises an image mosaic of images captured by the second sensor that is overlaid with images from the higher resolution first sensor.

7. The system of claim 1 further comprising an intent estimation module for deriving a user'"'"'s intent from a user'"'"'s control of the cognitive interface and using the user'"'"'s intent to control motion

11. The system of claim 1 wherein a user indirectly controls the at least one sensor through the cognitive interface.

8. The system of claim 8 wherein the intent estimation module comprises an exploration task estimation module and a user intent estimation module.

9. The system of claim 8 wherein the exploration task estimation module uses a hidden Markov model.

10. The system of claim 8 wherein the user intent estimation module creates an interest map of a user'"'"'s interest in specific regions and/or objects that have been imaged.

12. A method of performing adaptive image acquisition comprising:

capturing images from at least one sensor, where the at least one sensor produces a higher resolution image and a lower resolution image;

registering the images from the at least one sensor;

creating a virtual view of a scene imaged by the at least one imaging sensor using the registered images; and

enabling a user to indirectly control the at least one imaging sensor while viewing the virtual view.

13. The method of claim 12 wherein the at least one sensor comprises:

at least one first sensor is a narrow field of view camera for producing a higher resolution image and the at least one second sensor is a wide field of view camera for producing lower resolution images.

14. The method of claim 13 further comprising moving the at least one first sensor using Saccade motion.

15. The method of claim 14 wherein said creating step further comprises overlaying images from the higher resolution first sensor over an image mosaic of images captured by the second sensor.

16. The method of claim 12 further comprising deriving a user'"'"'s intent from a user'"'"'s control of the cognitive interface and using the user'"'"'s intent to control motion of the at least one sensor.

17. The method of claim 16 wherein the deriving step further comprises estimating an exploration task being performed.

18. The method of claim 17 wherein the deriving step further comprises using a hidden Markov model to estimate the exploration task.

19. The method of claim 15 wherein the deriving step further comprises creating an interest map of a user'"'"'s interest in specific regions and/or objects that have been imaged.

20. A method of performing adaptive image acquisition comprising:

capturing images from at least one imaging sensor,

creating a virtual view of a scene imaged by the at least one imaging sensor;

enabling a user to indirectly control the at least one imaging sensors while viewing the virtual view;

processing a user'"'"'s historical control of the at least one imaging sensor to estimate the user'"'"'s intent with respect to controlling the at least one sensor; and

controlling the at least one sensor in response to the user'"'"'s estimated intent.