Method and apparatus for modeling behavior using a probability distrubution function

• US 7,202,791 B2
• Filed: 06/28/2002
• Issued: 04/10/2007
• Est. Priority Date: 09/27/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Trajkovic, Miroslav
• Primary Examiner(s): GOINS, DAVETTA WOODS
• Application Number:
  US10/184,512
• Publication Number:
  US 20030059081A1
• Time in Patent Office: 1,747 Days
• Field of Search: 340/573.1, 340/573.4, 340/540, 340/3.1, 340/573.7, 382/165, 382/181, 382/190, 382/195, 382/218, 382/225, 382/254, 382/291, 382/156, 382/224
• US Class Current: 340/573.1
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method and apparatus are disclosed for modeling patterns of behavior of humans or other animate objects and detecting a violation of a repetitive pattern of behavior. The behavior of one or more persons is observed over time and features of the behavior are recorded in a multi-dimensional space. Over time, the multi-dimensional data provides an indication of patterns of human behavior. Activities that are repetitive in terms of time, location and activity, such as sleeping and eating, would appear as a Gaussian distribution or cluster in the multi-dimensional data. Probability distribution functions can be analyzed using known Gaussian or clustering techniques to identify repetitive patterns of behavior and characteristics thereof, such as a mean and variance. Deviations from repetitive patterns of behavior can be detected and an alarm can be triggered, if appropriate.

Claims

1. A method for modeling behavior of an animate object, the method comprising:

obtaining a plurality of images of the object;

extracting a plurality of features related to the object from the plurality of images; and

analyzing the plurality of extracted features in a multi-dimensional space to provide a set of probability distribution functions corresponding to a set of behavior patterns associated with the object that are each characterized by a mean and a variance.

2. The method of claim 1, wherein

analyzing the plurality of extracted features includes:

clustering the extracted feature data to identify at least one repetitive pattern of behavior corresponding to at least one of the set of probability distribution functions.

3. The method of claim 1, wherein

analyzing the plurality of extracted features includes:

fitting the extracted feature data to at least one Gaussian distribution corresponding to at least one of the set of probability distribution functions.

4. The method of claim 1, wherein

the extracted feature data includes an indication of time.

5. The method of claim 1, wherein

the extracted feature data includes an indication of location.

6. The method of claim 1, wherein

the extracted feature data includes an indication of activity.

7. The method of claim 1, wherein

the extracted feature data includes an indication of body posture.

13. The method of claim 1, further including

obtaining a subsequent image,

characterizing the image in the multi-dimensional space to provide a characterization of the image, and

comparing the characterization of the image to the set of probability distribution functions to determine a distance between the characterization and each of the set of probability distribution functions.

14. The method of claim 13, further including

providing a report if a minimum of the distance between the characterization and each of the set of probability distribution functions is above a given threshold.

15. The method of claim 13, further including

determining a minimum of the distance between the characterization and each of the set of probability distribution functions, and

associating the subsequent image to the probability distribution function corresponding to the minimum from the characterization.

8. A system for modeling behavior of an animate object, the system comprising:

a memory that stores computer-readable code, and

a processor operatively coupled to the memory,

wherein

the processor is configured to implement the computer-readable code to;

obtain a plurality of images of the object;

extract a plurality of features related to the object from the plurality of images; and

analyze the plurality of extracted features in a multi-dimensional space to provide a set of probability distribution functions corresponding to a set of behavior patterns associated with the object that are each characterized by a mean and a variance.

9. The system of claim 8, wherein

the processor is configured to analyze the plurality of features by clustering the extracted feature data to identify at least one repetitive pattern of behavior corresponding to at least one of the probability distribution functions.

10. The system of claim 8, wherein

the processor is configured to analyze the plurality of features by fitting the extracted feature data to at least one Gaussian distribution corresponding to at least one of the probability distribution functions.

16. The system of claim 8, wherein

the processor is further configured to:

obtain a subsequent image,

characterize the image in the multi-dimensional space to provide a characterization of the image, and

compare the characterization of the image to the set of probability distribution functions to determine a distance between the characterization and each of the set of probability distribution functions.

17. The system of claim 16, wherein

the processor is further configured to

provide a report if a minimum of the distance between the characterization and each of the set of probability distribution functions is above a given threshold.

18. The system of claim 16, wherein

the processor is further configured to

determine a minimum of the distance between the characterization and each of the set of probability distribution functions, and

associate the subsequent image to the probability distribution function corresponding to the minimum from the characterization.

11. An article of manufacture for modeling behavior of an animate object, the article of manufacture comprising:

a computer-readable medium having computer-readable code embodied thereon,

the computer-readable program code comprising;

a step to obtain a plurality of images;

a step to extract a plurality of features from the plurality of images; and

a step to analyze the plurality of extracted features in a multi-dimensional space to provide a set of probability distribution functions corresponding to a set of behavior patterns associated with the object that are each characterized by a mean and a variance.

12. The article of manufacture of claim 11, wherein

the computer-readable program code further includes:

a step to obtain a subsequent image,

a step to characterize the image in the multi-dimensional space to provide a characterization of the image, and

a step to compare the characterization of the image to the set of probability distribution functions to determine a distance between the characterization and each of the set of probability distribution functions.