Method and apparatus for detecting abnormal movement

• US 9,286,693 B2
• Filed: 01/23/2014
• Issued: 03/15/2016
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation), Seoul National University Industry Foundation (Seoul National University)
• Original Assignee: Hanwha Techwin Co., Ltd. (Hanwha Corporation), Seoul National University Industry Foundation (Seoul National University)
• Inventors: Yoo, Young Joon, Jeong, Ha Wook, Yi, Kwang Moo, Choi, Jin Young, Park, Dong Jun, Lim, Jeong Eun
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US14/161,890
• Publication Number:
  US 20140241619A1
• Time in Patent Office: 782 Days
• Field of Search: 382/103, 382/118, 382/154, 382/159, 382/190, 382/195, 348/155, 348/169, 348/180, 348/208
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Provided are a method and apparatus for detecting an abnormal movement. The apparatus includes a feature tracing unit configured to extract features of a moving object in an input image, trace a variation in position of the extracted features according to time, and ascertain trajectories of the extracted features; a topic online learning unit configured to classify the input image in units of documents which are bundles of the trajectories, and ascertain probability distribution states of topics, which constitute the classified document, by using an online learning method which is a probabilistic topic model; and a movement pattern online learning unit configured to learn a velocity and a direction for each of the ascertained topics, and learn a movement pattern by inferring a spatiotemporal correlation between the ascertained topics.

Claims

1. An apparatus for detecting an abnormal movement, the apparatus comprising:

at least one processor;

a feature tracing unit executed or controlled by the processor to extract features of a moving object in an input image, trace a variation in a position of the extracted features according to time, and ascertain trajectories of the extracted features;

a topic online learning unit executed or controlled by the processor to classify the input image in units of documents that are bundles of the trajectories expressed by a set of words representing positions of grid points through which the trajectories pass and a set of vector differences representing a set of differences in vectors between a position of an actual feature in a current frame and a position of an actual feature in a previous frame, and ascertain, by using an online learning method which is a probabilistic topic model, probability distribution states of topics included in a classified document; and

a movement pattern online learning unit executed or controlled by the processor to learn a velocity and a direction for each of the ascertained topics, and learn a movement pattern by inferring a spatiotemporal correlation between the ascertained topics by using a K-means clustering method.

2. The apparatus of claim 1, further comprising an abnormality detecting unit configured to classify the movement pattern, in response to the trajectories comprised in each frame of the input image having a low probability of being included in the learned movement pattern, as an abnormal movement pattern by using a Gaussian learning result of the learned movement pattern.

3. The apparatus of claim 1, wherein the feature tracing unit is configured to extract features of a moving object in an input image by using a Kanade-Lucas-Tomasi (KLT) tracing technique.

4. The apparatus of claim 1, wherein in the topic online learning unit, the document includes a plurality of topics which are generated by multinomial distribution, and a multinomial distribution parameter probability vector value indicating a probability distribution state of the plurality of topics constituting the document is inferred to cluster positions of the trajectories for each topic.

5. The apparatus of claim 1, wherein the multinomial distribution parameter probability vector value indicating a probability distribution state of the plurality of topics constituting the document comprises probability distribution (θ

_d) and topic-word probability distribution (φ

_k).

6. The apparatus of claim 1, wherein the trajectory is expressed by a set of words w_ji and a set of vector differences v_jiT, and

wherein w_ji indicates that a j-th trajectory passes through an i-th grid, and the set of vector differences represents a set of differences v_jiT in vectors between a position of an actual feature in the word and a position of the actual feature before T frames.

7. The apparatus of claim 6, wherein the word w_ji is based on multinomial distribution, and the difference v_jiτ

in vectors comprised in the words has Gaussian distribution.

8. The apparatus of claim 6, wherein a velocity is ascertained for each work position by using a position value of a current frame of a specific grid and an actual position value of the specific grid before τ

frames.

9. The apparatus of claim 1, wherein the online learning method is an online latent dirichlet distribution (OLDA) learning method.

10. The apparatus of claim 9, wherein in the OLDA learning method, LDA is inferred using variational bayes (VB).

11. An apparatus for detecting an abnormal movement, the apparatus comprising:

at least one processor;

a feature tracing unit executed or controlled by the processor to extract features of a moving object in an input image, trace a variation in a position of the extracted features according to time, and ascertain trajectories of the extracted features;

a trajectory classifying unit executed or controlled by the processor to classify the input image in units of documents indicating a bundle of the trajectories expressed by a set of words representing positions of grid points through which the trajectories pass and a set of vector differences representing a set of differences in vectors between a position of an actual feature in a current frame and a position of an actual feature in a previous frame, and infer, by using an online learning method which is a probabilistic topic model, a multinomial distribution parameter probability vector value indicating histogram distribution of topics constituting each document in order to cluster positions of the trajectories for each topic in the document;

a spatiotemporal correlation inferring unit executed or controlled by the processor to infer a spatiotemporal correlation on the basis of the inferred multinomial distribution parameter probability vector value; and

a movement pattern online learning unit executed or controlled by the processor to learn a velocity and a direction for each of the clustered topics, and learn a movement pattern by inferring a spatiotemporal correlation between the ascertained topics by using a K-means clustering method.

12. The apparatus of claim 11, further comprising an abnormality detecting unit configured to classify the movement pattern, in response to the trajectories comprised in each frame of the input image having a low probability of being comprised in the learned movement pattern, as an abnormal movement pattern by using a Gaussian learning result of the learned movement pattern.

13. The apparatus of claim 11, wherein the multinomial distribution parameter probability vector value comprises probability distribution (θ

_d) and topic-word probability distribution (φ

_k).

14. The apparatus of claim 11, wherein the trajectory is expressed by a set of words w_ji and a set of vector differences v_jiT, and

wherein w_ji indicates that a j-th trajectory passes through an i-th grid, and the set of vector differences represents a set of differences v_jiT in vector between a position of an actual feature in the word and a position of the actual feature before T frames.

15. The apparatus of claim 14, wherein the word w_ji is based on multinomial distribution, and the difference v_jiτ

in vectors comprised in the words has Gaussian distribution.

16. The apparatus of claim 14, wherein a velocity is ascertained for each work position by using a position value of a current frame of a specific grid and an actual position value of the specific grid before τ

frames.

17. The apparatus of claim 11, wherein the feature tracing unit is configured to extract features of the moving object in the input image by using a Kanade-Lucas-Tomasi (KLT) tracing technique.

18. The apparatus of claim 11, wherein the online learning method is an online latent dirichlet distribution (OLDA) learning method.

19. A method of detecting an abnormal movement, the method comprising:

extracting, using a processor, features of a moving object in an input image;

tracing a variation in position of the extracted features according to time;

ascertaining trajectories of the extracted features;

classifying the input image in units of documents that are bundles of the trajectories expressed by a set of words representing positions of grid points through which the trajectories pass and a set of vector differences representing a set of differences in vectors between a position of an actual feature in a current frame and a position of an actual feature in a previous frame;

ascertaining probability distribution states of topics, which constitute a classified document, by using an online learning method which is a probabilistic topic model;

learning a velocity and a direction for each of the ascertained topics; and

learning a movement pattern by inferring a spatiotemporal correlation between the ascertained topics by using a K-means clustering method.

20. The method of claim 19, further comprising classifying the moment pattern, in response to the trajectories comprised in each frame of the input image having a low probability of being comprised in the learned movement pattern, as an abnormal movement pattern by using a Gaussian learning result of the learned movement pattern.

21. A method of detecting an abnormal movement, the method comprising:

extracting, using a processor, features of a moving object in an input image;

tracing a variation in position of the extracted features according to time;

ascertaining trajectories of the extracted features;

classifying the input image in units of documents indicating a bundle of the trajectories expressed by a set of words representing positions of grid points through which the trajectories pass and a set of vector differences representing a set of differences in vectors between a position of an actual feature in a current frame and a position of an actual feature in a previous frame;

inferring, by using an online learning method which is a probabilistic topic model, a multinomial distribution parameter probability vector value indicating histogram distribution of topics constituting each document in order to cluster positions of the trajectories for each topic in the document;

inferring a spatiotemporal correlation on the basis of the inferred multinomial distribution parameter probability vector value;

learning a velocity and a direction for each of the clustered topics; and

learning a movement pattern by inferring a spatiotemporal correlation between the ascertained topics by using a K-means clustering method.