WARNING A USER ABOUT ADVERSE BEHAVIORS OF OTHERS WITHIN AN ENVIRONMENT BASED ON A 3D CAPTURED IMAGE STREAM

• US 20080169929A1
• Filed: 01/12/2007
• Published: 07/17/2008
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Albertson, Jacob C, Goldman, Steven D., Sessa, Anthony J., Arnold, Kenneth C., Paolini, Michael A.
• Granted Patent: US 8,269,834 B2
• Time in Patent Office: Days
• US Class Current: 340/573.1
• CPC Class Codes: G01V 5/20 Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253
• CPC Class Codes: Y10S 706/933 Law, law enforcement, or government

Abstract

A computer-implemented method, system, and program includes a behavior processing system for capturing a three-dimensional movement of a monitored user within a particular environment monitored by a supervising user, wherein the three-dimensional movement is determined by using at least one image capture device aimed at the monitored user. The behavior processing system identifies a three-dimensional object properties stream using the captured movement. The behavior processing system identifies a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with multiple adverse behavior definitions. In response to identifying the particular defined adverse behavior from among the multiple adverse behavior definitions, the behavior processing system triggers a warning system to notify the supervising user about the particular defined adverse behavior of the monitored user through an output interface only detectable by the supervising user.

Claims

1. A computer-implemented method for warning a supervising user of behaviors of a monitored user representative of adverse behavior, comprising:

capturing a three-dimensional movement of a monitored user within a particular environment monitored by the supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions;

in response to identifying the particular defined adverse behavior of the monitored user from among the plurality of adverse behavior definitions, notifying the supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface only detectable by the supervising user.

2. The method according to claim 1, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.

3. The method according to claim 1, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using at least one stereoscopic image device and at least one sensor enabled device to detect at least one additional characteristic of a detected object in the three-dimensional movement.

4. The method according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises calculating a percentage of probability that the captured three-dimensional movement represents a particular behavior defined in the particular adverse behavior definitions and calculating a percentage of probability that the captured three-dimensional movement is adverse.

5. The method according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises identifying at least one of body movement of the monitored user comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the monitored user within the three-dimensional object properties stream matching at least one of the plurality of adverse behavior definitions.

6. The method according to claim 1, further comprising:

responsive to detecting the supervising user conducting law enforcement activity within the particular environment comprising a law enforcement environment in which the supervising user is conducting at least one of a traffic stop, a security checkpoint, and an interrogation, capturing the three-dimensional movement of the monitored user within the particular environment.

7. The method according to claim 1, further comprising:

responsive to detecting the supervising user conducting psychological evaluation within the particular environment comprising a psychology professional monitoring a psychological condition of the monitored user, capturing the three-dimensional movement of the monitored user within the particular environment.

8. The method according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions further comprises identifying from the three-dimensional object properties one of a suspicious activity by the monitored user, an impaired condition of the monitored user, a concealed weapon carried by the monitored person, an aggressive stance by the monitored person, an angered facial expression of the monitored person, and a nervous condition of the monitored person.

9. The method according to claim 1, wherein notifying a supervising user for the particular environment about the particular defined adverse behavior of the monitored user further comprises:

sending the particular defined adverse behavior to a warning system for controlling warning signals output to the supervising user; and

translating, at the warning system, the particular defined adverse behavior into an output signal for output to an output controller of a output device for controlling output of the warning signals to a supervising user that are undetectable by the monitored user.

13. The system according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises identifying at least one of body movement of the monitored user comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the monitored user within the three-dimensional object properties stream matching at least one of the plurality of adverse behavior definitions.

10. A system for warning a supervising user of behaviors of a monitored user representative of adverse behavior, comprising:

a behavior processing system communicatively connected to a network, further comprising;

means for capturing a three-dimensional movement of a monitored user within a particular environment monitored by the supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

means for identifying a three-dimensional object properties stream using the captured movement;

means for identifying a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions;

a warning system communicatively connected to the behavior processing system via the network for transmitting at least one warning signal for notifying a supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface only detectable by the supervising user.

11. The system according to claim 10, wherein the means for capturing a three-dimensional movement of the monitored user further comprises means for capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.

12. The system according to claim 10, wherein the means for identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises means for calculating a percentage of probability that the captured three-dimensional movement represents a particular behavior defined in the particular adverse behavior definitions and calculating a percentage of probability that the captured three-dimensional movement is adverse.

14. The system according to claim 10, the behavior processing system further comprising means, responsive to detecting the supervising user conducting law enforcement activity within the particular environment comprising a law enforcement environment in which the supervising user is conducting at least one of a traffic stop, a security checkpoint, and an interrogation, for capturing the three-dimensional movement of the monitored user within the particular environment.

15. The system according to claim 10, the behavior processing system further comprising means, responsive to detecting the supervising user conducting psychological evaluation within the particular environment comprising a psychology professional monitoring a psychological condition of the monitored user, for capturing the three-dimensional movement of the monitored user within the particular environment.

16. The system according to claim 10, wherein the means for identifying a particular defined adverse behavior representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions further comprises means for identifying from the three-dimensional object properties one of a suspicious activity by the monitored user, an impaired condition of the monitored user, a concealed weapon carried by the monitored person, an aggressive stance by the monitored person, an angered facial expression of the monitored person, and a nervous condition of the monitored person.

17. The system according to claim 10, wherein the warning system further comprises:

means for translating the particular defined adverse behavior into an output signal for output to an output controller of a output device for controlling output of the warning signals to a supervising user that are undetectable by the monitored user.

18. A program product comprising a storage-type computer-usable medium including a computer-readable program for warning a supervising user of behaviors of a monitored user representative of adverse behavior, wherein the computer-readable program when executed on a computer causes the computer to:

capture a three-dimensional movement of a monitored user within a particular environment monitored by the supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

identify a three-dimensional object properties stream using the captured movement;

identify a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions;

notify the supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface only detectable by the supervising user.

19. The program product according to claim 18, wherein said computer-readable program is transmitted over a network.

20. The program product according to claim 18, wherein the computer-readable program when executed on a computer further causes the computer to capture the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.