INTELLIGENT HIGH RESOLUTION VIDEO SYSTEM

• US 20090219387A1
• Filed: 02/28/2008
• Published: 09/03/2009
• Est. Priority Date: 02/28/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Videolq Incorporated
• Inventors: Marman, Douglas H., Saptharishi, Mahesh
• Granted Patent: US 9,584,710 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/575 with a response composed of multiple slopes
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An automated electronic video surveillance system enables a high-resolution mega-pixel camera to capture high quality, detailed, magnified images at multiple locations, simultaneously with an overview of the whole scene. A preferred embodiment requiring no moving parts provides full 360-degree continuous viewing with up to 5× all-digital zoom capability. The system performs continuous surveillance and active resolution allocation in the form of a feedback control subsystem that dynamically allocates resources so that important details within a scene receive appropriate scrutiny, while uninteresting areas are imaged at a lower resolution.

Claims

1. A method of dynamically controlling an imager of a video camera to provide with selectable resolution image information relating to one or more regions of interest in a scene acquired by the video camera, the imager having a specified maximum number of image pixels that establish a characteristic maximum image resolution of the imager, comprising:

controlling the imager to acquire a first series of sub-sampled video data corresponding to a first field of view of the scene at a first image resolution;

analyzing the sub-sampled video data in the first series to detect a region of interest in the scene;

controlling the imager to acquire a second series of sub-sampled video data corresponding to a second field of view of the scene at a second image resolution, the sub-sampled video data in the second series representing, at the second image resolution, a windowed view of the detected region of interest in the first field of view of the scene; and

each of the first and second series of sub-sampled video data corresponds to fewer image pixels than the specified maximum number of image pixels of the imager.

2. The method of claim 1, in which the imager using the specified maximum number of image pixels captures a full field of view of the scene at the characteristic maximum image resolution, in which the first field of view to which the first series of sub-sampled video data corresponds represents substantially the full field of view of the scene at the first image resolution, and in which the first image resolution is less than the characteristic maximum image resolution.

3. The method of claim 1, in which the imager using the specified maximum number of image pixels captures a full field of view of the scene at the characteristic maximum image resolution, and in which the first series of sub-sampled video data represents a portion of the full field of view of the scene.

4. The method of claim 3, in which the first image resolution of the first field of view of the scene is substantially equal to the characteristic maximum image resolution.

5. The method of claim 1, in which the sub-sampled video data in the second series representing a windowed view of the detected region of interest includes a magnified view portion as compared to the first field of view to which the sub-sampled video data of the first series corresponds, and in which the second image resolution is higher than the first image resolution.

6. The method of claim 1, further comprising analyzing the sub-sampled video data in the second series of sub-sampled video data to recognize a characteristic of the region of interest.

7. The method of claim 6, in which:

the characteristic of the region of interest is a presence of an object in the region of interest;

the first and second image resolutions represent respective lower and higher image resolutions;

the analysis of the sub-sampled video data in the first series is performed at the lower image resolution to detect the region of interest; and

the analysis of the sub-sampled video data in the second series is performed at the higher image resolution to more accurately recognize the object present in the region of interest.

8. The method of claim 6, further comprising controlling the imager to acquire a third series of sub-sampled video data corresponding to the scene, the sub-sampled video data in the third series representing the recognized characteristic in a magnified localized portion of the windowed view of the detected region of interest.

9. The method of claim 6, in which:

the recognized characteristic includes an object located in the region of interest, the object having a size;

the video camera produces video image data in response to light propagating from a scene; and

the second image resolution of the second series of sub-sampled video data is determined by recognition of the size of the object over time for the video image data produced.

10. The method of claim 1, in which the video camera produces video image data in response to light propagating from a scene, and further comprising:

analyzing the sub-sampled video data in the first series of sub-sampled video data to recognize an object in the region of interest, the object having a size; and

determining from the size of the object the second image resolution of the second series of sub-sampled video data for the video image data produced.

11. The method of claim 1, in which the region of interest is manually selected.

12. The method of claim 1, further comprising:

analyzing the sub-sampled video data in the second series of sub-sampled video data to recognize a characteristic of the region of interest; and

upon verification of the characteristic of interest after the analysis of the sub-sampled video data, controlling the imager to acquire a continual stream of the second series of sub-sampled video data representing successive observations of the windowed view of the detected region of interest to closely observe for a time the recognized characteristic.

13. The method of claim 1, in which the windowed view has position coordinates representing its location in the first field of view of the scene, and further comprising:

analyzing the sub-sampled video data in the first series of sub-sampled video data to identify an object in the region of interest; and

controlling the imager to change the position coordinates of the windowed view to track motion of the object through the first field of view of the scene.

14. The method of claim 13, further comprising changing to different values the second image resolution of the windowed view as the motion of the object is tracked through the first field of view of the scene.

15. The method of claim 1, in which the windowed view has position coordinates representing its location in the first field of view of the scene, and further comprising:

analyzing the sub-sampled video data in the second series of sub-sampled video data to recognize a number of objects in the region of interest; and

controlling the imager to change the position coordinates of the windowed view to track motion of each of the number of objects through the first field of view of the scene.

16. The method of claim 15, in which the number of objects is greater than one and the windowed view is of a size that covers all of number of objects recognized.

17. The method of claim 1, in which the video camera produces video image data in response to light propagating from a scene such that the first and second series of sub-sampled video data correspond to images of, respectively, the first and second fields of view of the scene, and further comprising delivering from the imager the sub-sampled video data in the first and second series as interleaved images of, respectively, the first and second fields of view of the scene.

18. The method of claim 1, in which the video camera produces video image data in response to light propagating from a scene such that the first and second series of sub-sampled video data correspond to images of, respectively, the first and second fields of view of the scene in which the first and second image resolutions are different, and further comprising transmitting through a communication medium for delivery to a destination the first and second series of sub-sampled video data in combination as a video stream representing the first and second fields of view at the different image resolutions.

19. The method of claim 18, in which the imager using the specified maximum number of image pixels captures a full field of view of the scene at the characteristic maximum image resolution, in which the first field of view to which the first series of sub-sampled video data corresponds represents substantially the full field of view of the scene at the first image resolution, and in which the first image resolution is less than the characteristic maximum image resolution.

20. The method of claim 1, in which the video camera produces video image data in response to light propagating from a scene such that the first and second series of sub-sampled video data correspond to images of, respectively, the first and second fields of view of the scene in which the first and second image resolutions are different, and further comprising transmitting through a communication medium for delivery to a destination the first and second series of sub-sampled video data as, respectively, a first video stream representing a first windowed view as the first field of view of the scene and a second video stream representing a second windowed view as the second field of view of the scene.

21. The method of claim 1, further comprising:

analyzing the sub-sampled video data in the first series of sub-sampled video data to recognize a characteristic of the region of interest; and

controlling the imager to acquire at a frame rate one of the first and second series of sub-sampled video data corresponding to the scene, the frame rate being set in response to the characteristic recognized by the analysis of the sub-sampled video data in the first series.

22. The method of claim 21, in which the recognized characteristic includes an object located in an area of the region of interest, in which the object and the area are characterized by brightness levels, and in which the frame rate is set to provide a frame integration time that is adequate to provide a brightness level contrast between the object and the area to render them visually distinguishable from each other.

23. The method of claim 21, in which the frame rate constitutes a first frame rate and in which the controlling the imager acquires the first series of sub-sampled video data at the first frame rate, and further comprising controlling the imager to acquire at a second frame rate the second series of sub-sampled video data.

24. An automated active video surveillance system, comprising:

a high image data capacity video camera operable to produce images of a scene, the camera including a high resolution imager that acquires image data corresponding to the scene;

a video analytics engine implemented to perform analysis of data acquired by the imager;

an imager controller operatively associated with the imager and the video analytics engine to provide for analysis by the video analytics engine multiple portions of the acquired image data to detect regions of interest at different locations in the scene, the video analytics engine producing analysis output data representing results of the analysis performed; and

a communications medium receiving and transmitting to a destination separate from the camera data corresponding to the analysis output data.

25. The system of claim 24, in which the imager controller in response to the video analytics engine allocates resolution so that at least some of the multiple portions are analyzed with different image resolutions, thereby enabling smaller objects within the scene to be imaged at higher resolution to undergo appropriate scrutiny and larger objects within the scene to be imaged at lower resolution.

26. The system of claim 24, in which the multiple portions of the acquired image data are provided substantially simultaneously for analysis.

27. The system of claim 24, further comprising an image display operatively connected to the communications medium to display as visual images data corresponding to the analysis output data.

28. The system of claim 24, further comprising an image display operatively connected to the communications medium, and in which data corresponding to the analysis output data represent a closer range view of a feature of the scene and a farther range view of a region of the scene, and in which the image display is operable to display simultaneously images of the closer range view and the farther range view.

29. The system of claim 24, further comprising a storage medium operatively connected to the communications medium to store the data corresponding to the analysis output data.

30. The system of claim 24, in which the video camera is attached to a motorized pan mechanism, and in which the motorized pan mechanism is driven by stepper motors controlled by the video analytics engine to move the camera in response to analysis by the video analytics engine detecting activity in at least one of the regions of interest.

31. The system of claim 24, in which the video camera includes a high image resolution lens.

32. The system of claim 31, in which the analysis of data performed by the video analytics engine includes adjustment of light exposure levels associated with the high image resolution lens to optimize recognition of an object of interest within the scene, the adjustment of light exposure levels excluding from the analysis output data portions corresponding to high contrast areas of the scene.

33. The system of claim 31, in which the high image resolution lens is of fixed focal length and an f-stop of 1.0 or smaller.