BEHAVIOR AND PATTERN ANALYSIS USING MULTIPLE CATEGORY LEARNING

• US 20080175482A1
• Filed: 01/22/2007
• Published: 07/24/2008
• Est. Priority Date: 01/22/2007
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Cohen, Isaac, Ma, Yunqian
• Granted Patent: US 8,103,090 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A video processing system is configured to receive training video samples from a plurality of video sensing devices. The training video samples are sets of pair video samples. These pair video samples can include both substantially similar subject matter and different subject matter. In the first step, there is a patch pool sampled from videos, and the system select patches with more saliency. The saliency is represented by the conditional probability density function of the similar subject and the conditional probability of the different subject. During the testing phase, the system applies the selected patches from the training phase, and returns the matched subjects.

Claims

1. A video processing system comprising a configuration to:

receive first training video samples from a plurality of video sensing devices, the first training video samples comprising substantially similar subject matter;

generate a first training probability density function using features extracted from the first training video samples;

receive second training video samples from the plurality of video sensing devices, the second training video samples comprising insubstantially similar subject matter; and

generate a second training probability density function using features extracted from the second training video samples.

2. The video processing system of claim 1, wherein the features extracted from the first training video samples are extracted from one or more sub-images in the first training video samples, and further wherein the features extracted from the second training video samples are extracted from one or more sub-images in the second training video samples, and further wherein the sub-images in the second video training samples correspond to the sub-images in the first training video samples.

3. The video processing system of claim 1, further comprising a configuration to:

receive testing video samples from the plurality of video sensing devices;

extract features from the testing video samples;

calculate a distance between features from the testing video samples and stored video samples; and

apply the distance to the first training probability density function and the second probability density functions to determine whether the testing video samples correlate to one or more of the first training video samples and the second video training samples.

4. The video processing system of claim 3, further comprising a configuration to extract the features from the testing video samples from one or more sub-images in the testing video samples.

5. The video processing system of claim 3, further comprising a configuration to apply a threshold to the first training probability density function, wherein the threshold provides an indication of whether the testing video samples correlate with the first training video samples.

6. The video processing system of claim 2, wherein the video processing system is further configured to identify the one or more sub-images by identifying points of interest in the first and second training video samples.

7. The video processing system of claim 1, wherein the features comprise one or more of a temporal feature, a spatial feature, and a spatial-temporal feature.

8. The video processing system of claim 7, wherein the distance feature comprises a distance between a point within the sub-image and a point within an object in the sub-image.

9. The video processing system of claim 7, wherein the spatial features include directional and derived features, and the temporal features include optical flow components.

10. The video processing system of claim 9, wherein the directional features comprise one or more of a vertical feature, a horizontal feature, a forward diagonal feature, a backward diagonal feature, a diagonal energy feature, and a mean energy feature.

11. The video processing system of claim 1, wherein the first training probability density function is generated by:

estimating the distance between features on the similar subject; and

further wherein

the second training probability density function is generated by;

estimating the distance between features on the insubstantially similar subject.

12. A video processing system comprising a configuration to:

receive first training video samples,

the first training video samples captured by a plurality of video sensing devices, each video sensing device representing a different view of a field of view,

each first training video sample comprising a first video sequence and a second video sequence, the first video sequence and the second video sequence comprising substantially similar subject matter captured by a single video sensing device of the plurality of video sensing devices;

identify a plurality of sub-images in each frame of the first video sequence and the second video sequence, each sub-image in the first video sequence having a corresponding sub-image in the second video sequence;

extract features from each of the sub-images; and

generate a first training probability density function for each sub-image and corresponding sub-image as a function of the extracted features;

and wherein the video processing system further comprises a configuration to;

receive second training video samples,

the second training video samples captured by the plurality of video sensing devices,

each second training video sample comprising a first video sequence and a second video sequence, the first video sequence and the second video sequence of the second training video sample comprising insubstantially similar subject matter captured by a single video sensing device of the plurality of video sensing devices;

identify a plurality of sub-images in each frame of the first and second video sequences of the second training video sample, each sub-image in the first video sequence of the second training video sample having a corresponding sub-image in the second video sequence of the second training video sample;

extract features from each of the sub-images of the second training video sample; and

generate a second training probability density function for each sub-image and corresponding sub-image as a function of the extracted features of the second training video samples.

13. The video processing system of claim 12, further comprising a configuration to:

receive testing video samples, the testing video samples captured by the plurality of video sensing devices;

identify a plurality of sub-images in each frame of the testing video samples, each sub-image in the testing video samples corresponding to a sub-image in one or more of the first training video samples and the second training video samples;

extract features from each of the sub-images of the testing video samples;

calculate a feature number for each of the sub-images of the testing video samples; and

apply the feature number to the first training probability density function and the second probability density function to determine whether the testing video samples correlate to the first training probability density function or the second training probability density function.

14. The video processing system of claim 12, wherein the video processing system is further configured to identify the plurality of sub-images in each of the first training video samples, the second training video samples, and the testing video samples by identifying points of interest in the frame of a video sequence.

15. The video processing system of claim 12, wherein the features comprise one or more of a spatial feature, a temporal feature, and a spatial-temporal feature.

16. The video processing system of claim 12, wherein the first training probability density function is generated by:

estimating the distance between features for each sub-image of the similar subject in the first training video sample;

and further wherein

the second training probability density function is generated by;

estimating the distance between features for each sub-image of the insubstantially similar subject in the second training video sample.

17. The video processing system of claim 12, wherein

each first training video sample comprises a single image frame;

each second training video sample comprises a single image frame; and

each testing video sample comprises a single image frame.

18. An image processing system comprising a configuration to:

receive first training images from a plurality of video sensing devices, the first training images comprising substantially similar subject matter;

identify a plurality of sub-images in each first training image;

generate a first training probability density function using features extracted from the first training images;

receive second training images from the plurality of video sensing devices, the second training images comprising insubstantially similar subject matter;

identify a plurality of sub-images in each second training image, each sub-image in the second training image having a corresponding sub-image in the first training images; and

generate a second training probability density function using features extracted from the second training images;

wherein the first probability density function is generated by;

estimating the distance between features for each sub-image of the similar subject;

wherein the second probability density function is generated by;

estimating the distance between features for each sub-image of the insubstantially similar subject.

19. The image processing system of claim 18, further comprising a configuration to:

receive testing images from the plurality of video sensing devices;

identify a plurality of sub-images in each testing image, the plurality of sub-images in each testing image corresponding to a sub-image in one or more of the first training image and the second training image;

extract features from each of the sub-images in the testing images;

combine the features from each of the sub-images of the testing images to form a feature number; and

apply the distance between features to the first training probability density function and the second training probability density function to determine whether the testing images correlate to the first training probability density function or the second training probability density function.

20. The image processing system of claim 19, wherein the image processing system is further configured to identify the plurality of sub-images in each of the first training images, the second training images, and the testing images by identifying points of interest in the frame of a video sequence.