ACTIVITY DETERMINATION AS FUNCTION OF TRANSACTION LOG

• US 20120075450A1
• Filed: 09/24/2010
• Published: 03/29/2012
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated, ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Ding, Lei, Fan, Quanfu, Hampapur, Arun, Pankanti, Sharathchandra U.
• Granted Patent: US 8,610,766 B2
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Human behavior alerts are determined from a video stream through application of video analytics that parse a video stream into a plurality of segments, wherein each of the segments are either temporally related to at least one of a plurality of temporally distinct transactions in an event data log; or they are each associated with a pseudo transaction marker if not temporally related to at least one of the temporally distinct transactions and an image analysis indicates a temporal correlation with at least one of the distinct transactions is expected. Visual image features are extracted from the segments and one-SVM classification is performed on the extracted features to categorize segments into inliers or outliers relative to a threshold boundary. Event of concern alerts are issued with respect to the inlier segments associated with the associated pseudo transaction marker.

Claims

1. A method for determination of human behavior alerts from a video stream, the method comprising:

parsing a video stream via a programmable device into a plurality of segments, each of the segments either temporally related to at least one of a plurality of temporally distinct transactions in an event data log, or associated with a pseudo transaction marker if not temporally related to at least one of the temporally distinct transactions and an image analysis of said not temporally related segment indicates a temporal correlation with at least one of the distinct transactions is expected;

extracting visual image features from the segments;

performing a one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to a threshold boundary; and

issuing event of concern alerts with respect to each of the inlier segments associated with the associated pseudo transaction marker.

2. The method of claim 1, wherein the human behavior is sweet-hearting within a scanning lane video stream;

wherein the programmable device segments a scanning lane video stream into the segments; and

wherein the issuing the event of concern alerts comprises identifying the each of the inlier segments associated with the associated pseudo transaction marker as fake-scans.

3. The method of claim 1, wherein the parsing the video stream into the plurality of segments comprises:

defining each segment as spanning three consecutive temporally distinct transactions each within a specified time threshold of a next one of the temporally distinct transactions.

4. The method of claim 3, wherein the specified time threshold is selected from a range of from two to five seconds.

5. The method of claim 3, wherein the parsing the video stream via the programmable device into the plurality of segments, each associated with the pseudo transaction marker if not temporally related to the at least one of the temporally distinct transactions and the image analysis of said not temporally related segment indicates the temporal correlation with at least one of the distinct transactions is expected, comprises:

identifying a one segment occurring between a first and a second of the consecutive temporally distinct transactions wherein a time gap between the first and the second transactions is longer than the specified threshold;

performing an automated analysis of the one segment occurring between the first and the second transactions to determine a presence of an expected specified visual activity; and

associating the one segment occurring between the first and the second transactions with the pseudo transaction if an expected specified visual activity is determined by the performed automated analysis.

6. The method of claim 5, wherein the performing the automated analysis of the one segment occurring between the first and the second transactions to determine the presence of the expected specified visual activity comprises:

taking locally maximum points of a squared weighted gradient norm function as interest points in the one segment;

extracting averaged color values and image gradient vectors relevant to the interest points; and

composing color feature vectors from the extracted averaged color values and the extracted image gradient vectors.

7. The method of claim 6, wherein the performing the one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to the threshold boundary comprises:

learning dominant clusters of inliers within the threshold boundary in feature space to represent normal human activity; and

wherein the threshold boundary is a threshold beyond which the segments are determined to be visually dissimilar to the outliers.

8. The method of claim 7, wherein performing the one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to the threshold boundary comprises:

defining a data set of points within an input space radius; and

transforming the data set to a high-dimensional space to find a smallest radius of an enclosing dominant cluster or threshold boundary as a function of a slack variable soft constraint.

9. The method of claim 7, wherein the issuing the event of concern alerts with respect to each of the inlier segments associated with the associated pseudo transaction marker further comprises:

distinguishing outliers associated with the pseudo transaction markers from outliers temporally related to at least one of the plurality of temporally distinct transactions in the event data log.

10. A computer system for harmonizing business process tasks, the computer system comprising:

a processing unit, computer readable memory and a computer readable storage system;

first program instructions to parse a video stream into a plurality of segments, each of the segments either temporally related to at least one of a plurality of temporally distinct transactions in an event data log, or associated with a pseudo transaction marker if not temporally related to at least one of the temporally distinct transactions and an image analysis of said not temporally related segment indicates a temporal correlation with at least one of the distinct transactions is expected;

second program instructions to extract visual image features from the segments and perform a one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to a threshold boundary; and

third program instructions to issue event of concern alerts with respect to each of the inlier segments associated with the associated pseudo transaction marker; and

wherein the first, second and third program instructions are stored on the computer readable storage system for execution by the processing unit via the computer readable memory.

11. The computer system of claim 10, wherein the human behavior is sweet-hearting within a scanning lane video stream;

wherein the first program instructions are to segment a scanning lane video stream into the segments; and

wherein the third program instructions are to issue event of concern alerts identifying the inliers associated with the pseudo transaction marker as fake-scans.

12. The computer system of claim 10, wherein the first program instructions are further to parse the video stream into the plurality of segments by defining each segment as spanning three consecutive temporally distinct transactions, each within a specified time threshold of a next one of the temporally distinct transactions.

13. The computer system of claim 12, wherein the first program instructions are further to parse the video stream into the plurality of segments, each associated with the pseudo transaction marker if not temporally related to the at least one of the temporally distinct transactions and the image analysis of said not temporally related segment indicates the temporal correlation with at least one of the distinct transactions is expected, by:

identifying a one segment occurring between a first and a second of the consecutive temporally distinct transactions wherein a time gap between the first and the second transactions is longer than the specified threshold;

performing an automated analysis of the one segment occurring between the first and the second transactions to determine a presence of an expected specified visual activity; and

associating the one segment occurring between the first and the second transactions with the pseudo transaction marker if an expected specified visual activity is determined by the performed automated analysis.

14. The computer system of claim 13, wherein the first program instructions are further to perform the automated analysis of the one segment occurring between the first and the second transactions to determine the presence of the expected specified visual activity by:

taking locally maximum points of a squared weighted gradient norm function as interest points in the one segment;

extracting averaged color values and image gradient vectors relevant to the interest points; and

composing color feature vectors from the extracted averaged color values and the extracted image gradient vectors.

15. The computer system of claim 14, wherein the second program instructions are further to perform the one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to the threshold boundary by:

learning dominant clusters of inliers within the threshold boundary in feature space to represent normal human activity; and

wherein the threshold boundary is a threshold beyond which the segments are determined to be visually dissimilar to the outliers.

16. A computer program product for harmonizing business process tasks, the computer program product comprising:

a computer readable storage medium;

first program instructions to parse a video stream into a plurality of segments, each of the segments either temporally related to at least one of a plurality of temporally distinct transactions in an event data log, or associated with a pseudo transaction marker if not temporally related to at least one of the temporally distinct transactions, and an image analysis of said not temporally related segment indicates a temporal correlation with at least one of the distinct transactions is expected;

second program instructions to extract visual image features from the segments and perform a one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to a threshold boundary; and

third program instructions to issue event of concern alerts with respect to each of the inlier segments associated with the associated pseudo transaction marker; and

wherein the first, second and third program instructions are stored on the computer readable storage medium.

17. The computer program product of claim 16, wherein the human behavior is sweet—

hearting within a scanning lane video stream;

wherein the first program instructions are to segment a scanning lane video stream into the segments; and

wherein the third program instructions are to issue event of concern alerts identifying the inliers associated with the pseudo transaction marker as fake-scans.

18. The computer program product of claim 16, wherein the first program instructions are further to parse the video stream into the plurality of segments by defining each segment as spanning three consecutive temporally distinct transactions each within a specified time threshold of a next one of the temporally distinct transactions.

19. The computer system of claim 18, wherein the first program instructions are further to parse the video stream into the plurality of segments, each associated with the pseudo transaction marker if not temporally related to the at least one of the temporally distinct transactions and the image analysis of said not temporally related segment indicates the temporal correlation with at least one of the distinct transactions is expected, by:

identifying a one segment occurring between a first and a second of the consecutive temporally distinct transactions wherein a time gap between the first and the second transactions is longer than the specified threshold;

performing an automated analysis of the one segment occurring between the first and the second transactions to determine a presence of an expected specified visual activity; and

associating the one segment occurring between the first and the second transactions with the pseudo transaction marker if an expected specified visual activity is determined by the performed automated analysis.

20. The computer program product of claim 19, wherein the first program instructions are further to perform the automated analysis of the one segment occurring between the first and the second transactions to determine the presence of the expected specified visual activity by:

taking locally maximum points of a squared weighted gradient norm function as interest points in the one segment;

extracting averaged color values and image gradient vectors relevant to the interest points; and

composing color feature vectors from the extracted averaged color values and the extracted image gradient vectors.

21. The computer program product of claim 20, wherein the second program instructions are further to perform the one-SVM classification on the extracted segment features to categorize the segments into inliers or outliers relative to the threshold boundary by:

learning dominant clusters of inliers within the threshold boundary in feature space to represent normal human activity; and

wherein the threshold boundary is a threshold beyond which the segments are determined to be visually dissimilar to the outliers.