Dynamically reconfigurable vision system

• US 7,106,374 B1
• Filed: 04/04/2000
• Issued: 09/12/2006
• Est. Priority Date: 04/05/1999
• Status: Expired due to Term

• Current Assignee: Comptek-Amherst Systems, Inc.
• Original Assignee: Amherst Systems, Inc. (Northrop Grumman Corporation)
• Inventors: Stack, David J., Bandera, Cesar, Kramer, Christopher, McLoughlin, Terence H.
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): Henn, Timothy J.
• Application Number:
  US09/542,562
• Time in Patent Office: 2,352 Days
• Field of Search: 348/207.1, 348/207.11, 348/211.9, 348/211.99, 348169-172, 348/208.1, 348/208.14, 348/308, 382/103
• US Class Current: 348/308
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A closed-loop vision system is disclosed that utilizes a concept known as Dynamically Reconfigurable Vision (DRV), which is adaptive image sensing driven by a computer or human operator'"'"'s response to changing scenery. The system reduces the amount of irrelevant video information sensed and thus achieves more effective bandwidth and computational resource utilization, as compared to traditional vision systems. One or more reconfigurable photodetector arrays sensitive to either visible, infrared or ultraviolet radiation are present in the DRV system. These photodetector arrays feature on-chip means for spatial and temporal data reduction implemented through multiple independently controllable, time-correlated, frequently overlapping windows on the photodetector array that may be programmed according to their size, location, resolution, integration time, and frame rate. All photodetector array windows are dynamically reconfigurable in real time on a frame-by-frame basis. Furthermore, a DRV system is constructed in a client-server architecture in which a vision processor client passes window request command messages to the reconfigurable photodetector array server, which in turn delivers the requested video back to the client processor. The ability to simultaneously reconfigure, integrate, process, and readout multiple photodetector array video windows is an important characteristic of the DRV system.

Claims

1. A vision system for imaging a scene using a dynamically reconfigurable photodetector array comprising:

a video camera having at least one reconfigurable photodetector array capable of imaging a scene;

means for controlling characteristics of imaging by said photodetector array; and

a computer system for receiving signals from said photodetector array representing the scene imaged by said photodetector array and sending signals to said controlling means to configure the characteristics of imaging by said photodetector, wherein said photodetector array simultaneously provides signals representing a frame with one or more windows having pixels in which said signals are simultaneously readout from the photodetector array, and two or more of said windows are capable of overlapping in said frame.

2. The system according to claim 1 wherein said photodetector array further comprises means for controlling the resolution of the pixels in each of said windows.

3. The system according to claim 1 further comprising a client-server interface between said computer system and said controlling means.

4. The system according to claim 1 wherein said controlling means represents a controller located on said photodetector array.

5. The system according to claim 1 in which said controlling means represents a logic device separate from said photodetector array.

6. The system according to claim 1 wherein said computer system is integrated into said video camera.

7. The system according to claim 1 wherein said computer system is separate from said video camera.

8. The system according to claim 1 wherein said computer system sends signals to said controlling means representing window request commands that contain information used to dynamically reconfigure one or more of said photodetector array'"'"'s imaging characteristics.

9. The system according to claim 1 wherein said characteristics represent spatial and temporal parameters.

10. The system according to claim 1 wherein said photodetector array contains a plurality of individual pixels that have fixed dimensions.

11. The system according to claim 1 wherein said photodetector array represents a complementary metal oxide semiconductor photodetector array responsive to radiation in the visible spectral range.

12. The system according to claim 1 wherein said photodetector array represents a device responsive to infrared wavelengths.

13. The system according to claim 1 wherein said photodetector array is bonded to a complementary metal oxide semiconductor read-out integrated circuit.

14. The system according to claim 1 wherein said computer system and said video camera are part of a closed-loop interactive system.

15. The system according to claim 1 wherein said computer system is capable of analyzing video imagery for target objects and determining the necessary position, size, shape, resolution, and frame rate of multiple, overlapping windows within said vision system'"'"'s field of view.

16. The system according to claim 1 wherein said computer system is capable of determining the spectral signatures of said target objects.

17. The system according to claim 1 wherein said computer system further comprises a user interface enabling a human operator to analyze video imagery for target objects and determine the necessary position, size, shape, resolution, frame rate, and spectral content of multiple windows within said vision system'"'"'s field of view.

18. The system according to claim 1 wherein said computer system further comprises means for generating a window request command to said video camera identifying said characteristics.

19. The system according to claim 1 wherein said computer system further comprises means for generating a window request command to said video camera identifying said characteristics, in which said characteristics represents any of position, size, shape, resolution, frame rate, and spectral content of multiple windows.

20. The system according to claim 1 wherein when said windows overlap said windows overlap non-destructively in said frame.

21. The system according to claim 1 wherein said windows which overlap in said frame are of same or different resolutions.

22. The system according to claim 1 wherein said controlling means controls resolution of each of said windows by averaging pixels.

23. A method of tracking objects in a scene comprising a frame of windows comprising the steps of:

locating at least one object to be tracked in one of said windows in a low resolution;

forming one or more windows having said object in a high resolution which are capable of overlapping each other and overlapping said one or more of said windows at said low resolution in the frame in which said overlapping windows are simultaneously provided in the frame from signals simultaneously outputted from a photodetector array; and

identifying said object in said windows of said high resolution.

24. The method according to claim 23 wherein a plurality of said windows of high resolution are formed corresponding to the number of targets in said scene.

25. The method according to claim 23 further comprises the step of:

changing at least one window between different resolutions by averaging pixels of the window.

26. A system for imaging comprising:

at least one photodetector array capable of imaging successive frames in which each frame has windows of pixels at a low resolution; and

means for controlling the resolution of one or more windows of the frames to enable tracking of one or more objects at higher resolution windows in successive ones of said frames in which said windows are dynamically reconfigurable in real time between successive frames to facilitate said tracking, and said windows at said higher resolution non-destructively overlap one or more of said lower-resolution windows in the frame which are in the local of said higher resolution windows and at least one of said higher resolution windows are imaged simultaneously with at least one of said lower resolution windows being overlapped, and the pixels of the overlapping windows that are simultaneously imaged in the frame are simultaneously readout from the photodetector array.

27. The system according to claim 26 wherein said means is capable of controlling the resolution of windows by averaging pixels of said windows.

28. The system according to claim 26 wherein said overlapping windows are time-correlated frame-by-frame with location of said one or more objects in each successive frame.

29. The system according to claim 26 wherein said higher resolution windows are capable of overlapping each other.

30. A video imaging apparatus comprising:

at least one detector array for imaging a scene capable of simultaneous readout of multiple windows; and

a controller for controlling characteristics of imaging by said detector array to simultaneously output one or more windows in which two or more of the windows when simultaneously readout by said detector array are capable of overlapping.

31. The video imaging apparatus according to claim 30 wherein said controller is operative in response to a host computer system.

32. The video imaging apparatus according to claim 30 wherein said windows simultaneously outputted are of the same or different resolution.

33. The video imaging apparatus according to claim 30 wherein said overlapping windows overlap non-destructively.