System and method for pattern based thresholding applied to video surveillance monitoring

• US 8,659,657 B2
• Filed: 01/25/2008
• Issued: 02/25/2014
• Est. Priority Date: 01/25/2008
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brumfield, Sara Carlstead, Chen, Xiaoping, Marshburn, Tara Leigh, Tipton, Sandra Lee
• Primary Examiner(s): Serrao, Ranodhi
• Assistant Examiner(s): Hussain, Farrukh
• Application Number:
  US12/019,768
• Publication Number:
  US 20090189983A1
• Time in Patent Office: 2,223 Days
• Field of Search: None
• US Class Current: 348/143
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream

Abstract

A system, method, and program product is provided that configures video handlers pertaining to a dependent individual. Configuring includes setting alert thresholds. Visual locations are configured. Visual images that pertain to caregivers of the dependent individual are configured. Video streams are received from video sources. Video streams are compared to configured locations to classify the dependent individual'"'"'s location. Video stream is analyzed to determine whether the dependent individual is alone or with others. If with others, a list of known persons is determined by comparing the video streams with the configured visual images. The configured video handlers are initiated based on the inputs of the location and the people present with the dependent individual. Video handlers trigger alerts when thresholds are reached. Alerts include performing actions to protect the dependent individual from harm.

Claims

1. A computer-implemented method comprising:

configuring a plurality of video handlers that pertain to a dependent individual, wherein the configuring comprises setting an alert threshold for each configured video handler;

configuring a plurality of visual locations, wherein one or more of the visual locations correspond to a habitat of the dependent individual;

configuring a plurality of visual object images that pertain to one or more physical objects;

receiving one or more video streams from one or more video sources directed to the dependent individual;

determining by a surveillance monitoring program, based on comparing the received video streams with the configured plurality of visual locations, a location of the dependent individual;

determining by the surveillance monitoring program, based on comparing the received video streams with the configured plurality of visual object images, one or more physical objects that are in proximity to the dependent individual,

initiating one or more of the video handlers based on the location of the dependent individual, and the physical objects that are in proximity to the dependent individual, wherein the initiating comprises adding the video handlers to a state machine, and wherein the video handlers are selected from the group consisting of a physical aggression video handler, a water proximity video handler, an inappropriate touching video handler, an unattended dependent individual video handler, a dangerous object video handler, and an unknown person video handler;

executing the initiated video handlers, wherein the executing video handlers receive updated data from the surveillance monitoring program corresponding to the location of the dependent individual and the physical objects that are in proximity to the dependent individual; and

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing video handler, triggering an alert that includes performing at least one action intended to protect the dependent individual.

2. The method of claim 1 wherein the configuring of at least one of the video handlers includes selecting an action corresponding to the alert threshold for the video handler, and wherein the triggering further comprises performing the action corresponding to the selected executing video handler.

3. The method of claim 2 wherein the action is selected from the group consisting of contacting law enforcement, sounding an audible alarm, contacting a managing caregiver, and contacting a fire department.

4. The method of claim 1 further comprising:

configuring a plurality of audio handlers that pertain to the dependent individual, wherein the configuring comprises setting an alert threshold for each configured audio handler;

configuring a plurality of voice identities that pertain to one or more caregivers of the dependent individual;

configuring a plurality of audible samples that pertain to one or more inanimate objects;

receiving one or more audio streams from one or more audio sources directed to the dependent individual;

determining, by the surveillance monitoring program, based on comparing the received audio streams with the configured plurality of audible samples, one or more additional physical objects in proximity to the dependent individual;

determining, by the surveillance monitoring program, one or more people present with the dependent individual by comparing the plurality of voice identities with the received audio streams;

initiating one or more of the audio handlers based on the additional physical objects in proximity to the dependent individual and the one or more people present with the dependent individual, wherein the initiating comprises adding the audio handlers to the state machine;

executing the initiated audio handlers, wherein the executing audio handlers receive updated data corresponding to the additional physical objects in proximity to the dependent individual and the people present with the dependent individual; and

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing audio handler, triggering a second alert that includes performing at least one second action intended to protect the dependent individual.

5. The method of claim 4 further comprising:

maintaining a log of the initiated video handlers and the initiated audio handlers, wherein the log includes a timestamp when the initiated video and audio handlers were initiated, a list of one or more locations where the dependent individual was located, a list of one or more physical objects in proximity to the dependent individual, and a list of one or more people in proximity to the dependent individual.

6. A information handling system comprising:

one or more processors;

a memory accessible by at least one of the processors;

a nonvolatile storage device accessible by at least one of the processors;

one or more video input devices that provide one or more digital video streams accessible by the one or more processors;

one or more audio input devices that provide one or more digital audio streams accessible by the one or more processors;

a set of instructions stored in the memory and executed by at least one of the processors in order to perform actions of;

configuring a plurality of video handlers that pertain to a dependent individual, wherein the configuring comprises setting an alert threshold for each configured video handler;

configuring a plurality of visual locations, wherein one or more of the visual locations correspond to a habitat of the dependent individual;

configuring a plurality of visual object images that pertain to one or more physical objects;

receiving the digital video streams from the one or more video input devices, wherein the digital video streams are directed to the dependent individual;

determining by a surveillance monitoring program, based on comparing the received digital video streams with the configured plurality of visual locations, a location of the dependent individual;

determining by the surveillance monitoring program, based on comparing the received video streams with the configured plurality of visual object images, one or more physical objects that are in proximity to the dependent individual,

initiating one or more of the video handlers based on the location of the dependent individual, and the physical objects that are in proximity to the dependent individual, wherein the initiating comprises adding the video handlers to a state machine executed by at least one of the one or more processors, and wherein the video handlers are selected from the group consisting of a physical aggression video handler, a water proximity video handler, an inappropriate touching video handler, an unattended dependent individual video handler, a dangerous object video handler, and an unknown person video handler;

executing the initiated video handlers, wherein the executing video handlers receive updated data from the surveillance monitoring program corresponding to the location of the dependent individual and the physical objects that are in proximity to the dependent individual; and

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing video handler, triggering an alert that includes performing at least one action intended to protect the dependent individual.

7. The information handling system of claim 6 wherein the configuring of at least one of the video handlers includes selecting an action corresponding to the alert threshold for the video handler, and wherein the triggering further comprises performing the action corresponding to the selected executing video handler.

8. The information handling system of claim 7 wherein the action is selected from the group consisting of contacting law enforcement, sounding an audible alarm, contacting a managing caregiver, and contacting a fire department.

9. The information handling system of claim 6 wherein the set of instructions, when executed, cause at least one of the processors to perform further actions comprising:

configuring a plurality of audio handlers that pertain to the dependent individual, wherein the configuring comprises setting an alert threshold for each configured audio handler;

configuring a plurality of voice identities that pertain to one or more caregivers of the dependent individual;

configuring a plurality of audible samples that pertain to one or more inanimate objects;

receiving one or more digital audio streams from one or more audio input devices directed to the dependent individual;

determining, by the surveillance monitoring program, based on comparing the received digital audio streams with the configured plurality of audible samples, one or more additional physical objects in proximity to the dependent individual;

determining, by the surveillance monitoring program, one or more people present with the dependent individual by comparing the plurality of voice identities with the received digital audio streams;

initiating one or more of the audio handlers based on the additional physical objects in proximity to the dependent individual and the one or more people present with the dependent individual, wherein the initiating comprises adding the audio handlers to the state machine;

executing the initiated audio handlers, wherein the executing audio handlers receive updated data corresponding to the additional physical objects in proximity to the dependent individual and the people present with the dependent individual; and

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing audio handler, triggering a second alert that includes performing at least one second action intended to protect the dependent individual.

10. The information handling system of claim 9 wherein the set of instructions, when executed, cause at least one of the processors to perform further actions comprising:

maintaining a log of the initiated video handlers and the initiated audio handlers, wherein the log includes a timestamp when the initiated video and audio handlers were initiated, a list of one or more locations where the dependent individual was located, a list of one or more physical objects in proximity to the dependent individual, and a list of one or more people in proximity to the dependent individual.

11. A computer program product stored in a non-transitory computer readable medium, comprising functional descriptive material that, when executed by an information handling system, causes the information handling system to perform actions that include:

configuring a plurality of video handlers that pertain to a dependent individual, wherein the configuring comprises setting an alert threshold for each configured video handler;

configuring a plurality of visual locations, wherein one or more of the visual locations correspond to a habitat of the dependent individual;

configuring a plurality of visual object images that pertain to one or more physical objects;

receiving one or more video streams from one or more video sources directed to the dependent individual;

determining by a surveillance monitoring program, based on comparing the received video streams with the configured plurality of visual locations, a location of the dependent individual;

determining by the surveillance monitoring program, based on comparing the received video streams with the configured plurality of visual object images, one or more physical objects that are in proximity to the dependent individual,

initiating one or more of the video handlers based on the location of the dependent individual, and the physical objects that are in proximity to the dependent individual, wherein the initiating comprises adding the video handlers to a state machine, and wherein the video handlers are selected from the group consisting of a physical aggression video handler, a water proximity video handler, an inappropriate touching video handler, an unattended dependent individual video handler, a dangerous object video handler, and an unknown person video handler;

executing the initiated video handlers, wherein the executing video handlers receive updated data from the surveillance monitoring program corresponding to the location of the dependent individual and the physical objects that are in proximity to the dependent individual; and

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing video handler, triggering an alert that includes performing at least one action intended to protect the dependent individual.

12. The computer program product of claim 11 wherein the configuring of at least one of the video handlers includes selecting an action corresponding to the alert threshold for the video handler, and wherein the triggering further comprises performing the action corresponding to the selected executing video handler.

13. The computer program product of claim 12 wherein the action is selected from the group consisting of contacting law enforcement, sounding an audible alarm, contacting a managing caregiver, and contacting a fire department.

14. The computer program product of claim 11 further comprising functional descriptive material that causes the data processing system to perform additional actions that include:

configuring a plurality of audio handlers that pertain to the dependent individual, wherein the configuring comprises setting an alert threshold for each configured audio handler;

configuring a plurality of voice identities that pertain to one or more caregivers of the dependent individual;

configuring a plurality o audible samples that pertain to one or more inanimate objects;

receiving one or more audio streams from one or more audio sources directed to the dependent individual;

determining, by the surveillance monitoring program, based on comparing the received audio streams with the configured plurality of audible samples, one or more additional physical objects in proximity to the dependent individual;

determining, by the surveillance monitoring program one or more people present with the dependent individual by comparing the plurality of voice identities with the received audio streams;

initiating one or more of the audio handlers based on the additional physical objects in proximity to the dependent individual and the one or more people present with the dependent individual, wherein the initiating comprises adding the audio handlers to the state machine;

executing the initiated audio handlers, wherein the executing audio handlers receive updated data corresponding to the additional physical objects in proximity to the dependent individual and the people present with the dependent individual;

in response to determining that at least some of the updated data exceeds the alert threshold for a selected executing audio handler, triggering a second alert that includes performing at least one second action intended to protect the dependent individual; and

maintaining a log of the initiated video handlers and the initiated audio handlers, wherein the log includes a timestamp when the initiated video and audio handlers were initiated, a list of one or more locations where the dependent individual was located, a list of one or more physical objects in proximity to the dependent individual, and a list of one or more people in proximity to the dependent individual.