WARNING A USER ABOUT ADVERSE BEHAVIORS OF OTHERS WITHIN AN ENVIRONMENT BASED ON A 3D CAPTURED IMAGE STREAM

• US 20120140042A1
• Filed: 02/09/2012
• Published: 06/07/2012
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: SINOEAST CONCEPT LIMITED (Tencent Holdings Limited)
• Original Assignee: International Business Machines Corporation
• Inventors: Albertson, Jacob C., Arnold, Kenneth C., Goldman, Steven D., Paolini, Michael A., Sessa, Anthony J.
• Granted Patent: US 9,208,678 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G01V 5/20 Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253
• CPC Class Codes: Y10S 706/933 Law, law enforcement, or government

Abstract

A computer-implemented method, system, and program includes a behavior processing system for capturing a three-dimensional movement of a monitored user within a particular environment monitored by a supervising user, wherein the three-dimensional movement is determined by using at least one image capture device aimed at the monitored user. The behavior processing system identifies a three-dimensional object properties stream using the captured movement. The behavior processing system identifies a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with multiple adverse behavior definitions. In response to identifying the particular defined adverse behavior from among the multiple adverse behavior definitions, the behavior processing system triggers a warning system to notify the supervising user about the particular defined adverse behavior of the monitored user through an output interface only detectable by the supervising user.

Claims

1. A computer-implemented method for warning a supervising user of behaviors of a monitored user representative of adverse behavior, comprising:

monitoring a current location of a supervising user;

identifying a particular environment from among the plurality of environments comprising the current location;

responsive to detecting the supervising user conducting law enforcement activity within the particular environment comprising a law enforcement environment, triggering at least one image capture device within the particular environment to capture the three-dimensional movement of a monitored user within the particular environment to capture a focus area surrounding the supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular type of the interface accessible to the supervising user from among a plurality of types of interfaces;

accessing a selection of adverse behavior definitions specified for the particular environment from among a plurality of adverse behavior definitions specified for a plurality of environments from a behavior database and for at least one adverse behavior definition specified for the particular type of interface in the behavior database from among a plurality of adverse behavior definitions each specified for a separate one of the plurality of types of interfaces;

identifying a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with the selection of adverse behavior definitions; and

in response to identifying the particular defined adverse behavior of the monitored user from among the plurality of adverse behavior definitions, notifying the supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface of the particular type of interface only detectable by the supervising user.

2. The method according to claim 1, further comprising:

capturing a response by the supervising user to the supervising user receiving the notification of the particular defined adverse behavior through the interface, wherein the response indicates whether the particular defined adverse behavior is correctly identified; and

adjusting at least one definition in the selection of adverse behavior definitions specified for the particular environment and the particular type of interface in the behavior database to reflect the response.

3. The method according to claim 1, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.

4. The method according to claim 1, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using at least one stereoscopic image device and at least one sensor enabled device to detect at least one additional characteristic of a detected object in the three-dimensional movement.

5. The method according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises calculating a percentage of probability that the captured three-dimensional movement represents a particular behavior defined in the selection of adverse behavior definitions and calculating a percentage of probability that the captured three-dimensional movement is adverse.

6. The method according to claim 1, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions further comprises identifying from the three-dimensional object properties stream one of a suspicious activity by the monitored user, an impaired condition of the monitored user, a concealed weapon carried by the monitored user, an aggressive stance by the monitored user, an angered facial expression of the monitored user, and a nervous condition of the monitored user.

7. A computer system for warning a supervising user of behaviors of a monitored user representative of adverse behavior, comprising:

one or more processors;

a behavior system for execution by at least one of the one or more processors and operative to monitor a current location of a supervising user;

the behavior system operative to identify a particular environment from among the plurality of environments comprising the current location;

the behavior system, responsive to detecting the supervising user conducting law enforcement activity within the particular environment comprising a law enforcement environment, operative to trigger at least one image capture device within the particular environment to capture the three-dimensional movement of a monitored user within the particular environment to capture a focus area surrounding the supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

the behavior system operative to identify a three-dimensional object properties stream using the captured movement;

the behavior system operative to identify a particular type of the interface accessible to the supervising user from among a plurality of types of interfaces;

the behavior system operative to access a selection of adverse behavior definitions specified for the particular environment from among a plurality of adverse behavior definitions specified for a plurality of environments from a behavior database and for at least one adverse behavior definition specified for the particular type of interface in the behavior database from among a plurality of adverse behavior definitions each specified for a separate one of the plurality of types of interfaces;

the behavior system operative to identify a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with the selection of adverse behavior definitions; and

the behavior system, in response to identifying the particular defined adverse behavior of the monitored user from among the plurality of adverse behavior definitions, operative to notify the supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface of the particular type of interface only detectable by the supervising user.

8. The system according to claim 7, further comprising:

the behavior system operative to capture a response by the supervising user to the supervising user receiving the notification of the particular defined adverse behavior through the interface, wherein the response indicates whether the particular defined adverse behavior is correctly identified; and

the behavior system operative to adjust at least one definition in the selection of adverse behavior definitions specified for the particular environment and the particular type of interface in the behavior database to reflect the response.

9. The system according to claim 7, further comprising:

the behavior system operative to capture the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.

10. The system according to claim 7, further comprising:

the behavior system operative to capture the three-dimensional movement using at least one stereoscopic image device and at least one sensor enabled device to detect at least one additional characteristic of a detected object in the three-dimensional movement.

11. The system according to claim 7, further comprising:

the behavior system operative to calculate a percentage of probability that the captured three-dimensional movement represents a particular behavior defined in the selection of adverse behavior definitions and calculating a percentage of probability that the captured three-dimensional movement is adverse.

12. The system according to claim 7, further comprising:

the behavior system operative to identify from the three-dimensional object properties stream one of a suspicious activity by the monitored user, an impaired condition of the monitored user, a concealed weapon carried by the monitored user, an aggressive stance by the monitored user, an angered facial expression of the monitored user, and a nervous condition of the monitored user.

13. A computer-implemented method for warning a supervising user of behaviors of a monitored user representative of adverse behavior, comprising:

capturing a three-dimensional movement of a monitored user within a particular environment monitored by a supervising user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the monitored user;

identifying a three-dimensional object properties stream using the captured movement;

accessing a selection of adverse behavior definitions specified for the particular environment from among a plurality of adverse behavior definitions specified for a plurality of environments from a behavior database;

identifying a particular defined adverse behavior of the monitored user represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with the selection of adverse behavior definitions; and

in response to identifying the particular defined adverse behavior of the monitored user from among the plurality of adverse behavior definitions, notifying the supervising user for the particular environment about the particular defined adverse behavior of the monitored user through an interface of the particular type of interface only detectable by the supervising user.

sending the particular defined adverse behavior to a warning system for controlling warning signals output to the supervising user; and

translating the particular defined adverse behavior into an image layer specifying a transparent overlay image and a position indicator mapping a position of the transparent overlay image to the particular defined adverse behavior within the captured three-dimensional movement for output to an image overlay controller of a display device for controlling overlay of the image layer over output of the captured three-dimensional movement to graphically indicate the warning signals for the particular defined adverse behavior to a supervising user at a display device that is undetectable by the monitored user.

14. The method according to claim 13, further comprising:

capturing a response by the supervising user to the supervising user receiving the notification of the particular defined adverse behavior through the display device, wherein the response indicates whether the particular defined adverse behavior is correctly identified; and

adjusting at least one definition in the selection of adverse behavior definitions specified for the particular environment in the behavior database to reflect the response.

15. The method according to claim 13, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the monitored user.

16. The method according to claim 13, wherein capturing a three-dimensional movement of the monitored user further comprises capturing the three-dimensional movement using at least one stereoscopic image device and at least one sensor enabled device to detect at least one additional characteristic of a detected object in the three-dimensional movement.

17. The method according to claim 13, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream further comprises calculating a percentage of probability that the captured three-dimensional movement represents a particular behavior defined in the selection of adverse behavior definitions and calculating a percentage of probability that the captured three-dimensional movement is adverse.

18. The method according to claim 13, wherein identifying a particular defined adverse behavior representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of adverse behavior definitions further comprises identifying from the three-dimensional object properties stream one of a suspicious activity by the monitored user, an impaired condition of the monitored user, a concealed weapon carried by the monitored user, an aggressive stance by the monitored user, an angered facial expression of the monitored user, and a nervous condition of the monitored user.