Video analysis using segmentation gain by area

• US 20050213815A1
• Filed: 03/24/2005
• Published: 09/29/2005
• Est. Priority Date: 03/24/2004
• Status: Active Grant

• Current Assignee: CheckVideo, LLC (Kastle Systems International LLC)
• Original Assignee: Cernium Corporation (Kastle Systems International LLC)
• Inventors: Garoutte, Maurice V.
• Granted Patent: US 7,218,756 B2
• Time in Patent Office: Days
• US Class Current: 382/171
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Video segmentation wherein scene images of subjects and background are captured by video and are automatically segmented according to subject content, as by system determination of the nature of subjects, but segmentation of the subjects can be compromised by visual noise. Segmentation transformations of an original background and foreground images in the scenes are provided as a terrain map, and the segmentation may also incorporate noise blobs. Automatic segmentation threshold adaptation is based on measuring the segmentation noise while holding the number count of noise blobs constant, whereby to achieve noise-adaptive segmentation gain. Automatic threshold adaptation within multiple areas of the scene results in different possible thresholds in different areas based on the amount of segmentation noise in each area. Segmentation is improved accordingly.

Claims

1. In a system for capturing video of scenes, including a processor-controlled segmentation system for providing software-implemented segmentation of subjects of interest in said scenes based on processor-implemented interpretation of the content of the captured video, the improvement comprising software implementation for determining and controlling segmentation gain by areas of an image frame to obtain multiple values of segmentation gain in different areas of the scene according to visual noise level in said areas.

2. In a system as set forth in claim 1, wherein said software implementation controls video gain in respective ones of said areas, and wherein areas found to have greater noise over a time interval are given lower gain, and areas of lesser noise are given higher gain.

3. In a system as set forth in claim 1, wherein said scenes are divided into a grid having a predetermined number of areas, and wherein said software implementation calculates proportional integral derivative over each of the of predetermined number of areas according to the noise of each area, to permit independent gain setting for each of the areas, where the gain setting for each section is determined relative to a base segmentation gain.

4. In a system as set forth in claim 1, wherein said software implementation controls video gain in respective ones of said areas wherein subjects in said scenes are segmented according to a segmentation threshold based upon visual noise in segmentations of said subjects.

5. In a system as set forth in claim 4 wherein said threshold is automatically based on measurement of the segmentation noise.

6. In a system as set forth in claim 5, wherein segmentation noise is determined by measuring the number of noise blobs for segmented subjects of interest, and the system maintains the number of noise blobs constant for segmented subjects of interest.

7. In a system as set forth in claim 5, wherein the noise levels by area numbers are periodically checked at frequent intervals for subject segmentation and backgrounds noise levels are checked substantially less frequently.

8. In a system as set forth in claim 7, wherein the determination of noise levels is updated at an update interval which is a function of segmentation noise build-up between updates.

9. In a system as set forth in claim 7, wherein the determination of noise levels is updated at an update interval determined by maintaining segmentation noise build-up in the range from about 10% to about 20%, and preferably not more than 20%, between updates.

10. In a video segmentation system wherein scene images captured by video are automatically segmented according to subject content as by system determination of the nature of subjects, wherein scenes have both background and subjects may appear in the foreground, wherein segmentation of the subjects could be compromised by visual noise, wherein segmentation transformations of an original background and foreground images in scenes are provided as a terrain map, and wherein the segmentation may also incorporate noise blobs caused by the visual noise, which produces segmentation noise, the improvement comprising:

employing automatic segmentation threshold adaptation based on measuring the segmentation noise while

holding the number count of noise blobs substantially constant, whereby to achieve adaptive segmentation gain; and

using automatic threshold adaptation within multiple areas of the scene resulting in different possible thresholds in different areas based on the amount of segmentation noise in each area.

11. In a system as set forth in claim 10, the improvement further characterized by:

said segmentation is carried out by two-pass sequence wherein the second pass takes place only inside the outlines of targets segmented on the first pass, and

wherein the second pass uses a lower threshold than the first pass.

12. In a system as set forth in claim 10, the improvement further characterized by:

said segmentation employing a multi-variable segmentation technique based on image primitives of a terrain map that are transformations of the original background and foreground images in scenes.

13. A video processing method of automatically segmenting of subjects within a video scene, comprising:

determining and controlling segmentation gain by areas of an image frame to obtain different possible multiple values of segmentation gain in different areas of the scene according to the level of visual noise in the different areas.

14. A method according to claim 13 wherein scene areas of lesser noise are given higher segmentation gain.

15. A method according to claim 14 wherein a video scene for segmentation is divided into a grid having a predetermined number of areas, and wherein software-driven processing determines segmentation gain over each of the of predetermined number of areas.

16. A method according to claim 15 wherein segmentation gain is determined by proportional integral derivative control algorithm to provide automatic independent gain setting for each of the many areas, where the gain setting for each segment is determined relative to a base segmentation gain, and wherein a a single rate-of-change is not determined over an entire video scene.

17. A method according to claim 15 wherein:

image segmentation is carried out by two-pass sequence wherein the second pass takes place only inside the outlines of targets segmented on the first pass, and

wherein the second pass uses a lower threshold than the first pass.