Context aware surveillance system using a hybrid sensor network

• US 7,619,647 B2
• Filed: 04/20/2005
• Issued: 11/17/2009
• Est. Priority Date: 04/20/2005
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Erdem, Ugur M., Wren, Christopher R., Azarbayejani, Ali J.
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US11/110,528
• Publication Number:
  US 20060238618A1
• Time in Patent Office: 1,672 Days
• Field of Search: 348 36- 39, 348142-170
• US Class Current: 348/143
• CPC Class Codes: G08B 13/196 using television cameras

Abstract

A surveillance system detects events in an environment. The system includes a camera arranged in the environment, and multiple context sensors arranged in the environment. The sensors are configured to detect events in the environment. A processor is coupled to the camera and the context sensors via a network. The processor provides the camera with actions based only on the events detected by the context sensors. The actions cause the camera to view the detected events.

Claims

1. A surveillance system for detecting events in an environment, comprising:

a camera arranged in an environment;

a plurality of context sensors arranged in the environment and configured to detect events in the environment; and

a processor coupled to the camera and the plurality of context sensors via a network, the processor further comprising;

means for providing the camera with actions based only on the events detected by the context sensors, the actions causing the camera to view the detected events;

a memory storing the events and actions, in which the events and actions are stored in a table of the memory, and an entry a_j in the table maps an event to an action;

means for selecting the entry a_j according to;

$a_j=\arg <br><br>\underset{i\in <br><br>I}{\max }<br><br>\frac{R_{\mathrm{pc}}<br><br>\left(p_i<br><br>\left[t\right],c_j<br><br>\left[t\right]\right)}{R_{\mathrm{pp}}<br><br>\left(p_i<br><br>\left[t\right]\right)},$

where p_i[t] is a sequence of events generated by the camera in a particular pose corresponding to i, c_j[t] is a sequence of events generated by a particular context sensor j, R_pc is a correlation between the two event sequences p_i[t] and c_j[t], R_pp is an auto-correlation of the event sequence p_i[t], and t is an instant in time at which a particular event is detected.

2. The system of claim 1, in which the context sensors are motion detectors.

3. The system of claim 1, in which the context sensors produce a sequence of binary values, the binary values being true when there is motion in the environment, and the binary values being false when there is no motion.

4. The system of claim 1, further comprising:

means for acquiring a video stream with the camera; and

means for encoding the video stream with poses of the camera.

5. The system of claim 4, in which a current pose encodes output pan, tilt, and zoom parameters from the camera when the motion is detected.

6. The system of claim 1, in which the actions include input pan, tilt, and zoom parameters for the camera to view the detected events.

7. The system of claim 1, in which the events and actions are stored in a table of the memory, and a selected entry a_jk in the table maps a sequence of events to an action.

8. The system of claim 1, further comprising:

means for selecting the entry a_j according to;

$a_j=\arg <br><br>\underset{i\in <br><br>I}{\max }<br><br>\frac{<br><br>p_i<br><br>\left[t\right]\bigwedge <br><br>c_j<br><br>\left[t\right]<br><br>}{<br><br>p_i<br><br>\left[t\right]<br><br>},$

where p_i[t] is a sequence of events generated by the camera in a particular pose corresponding to i, c_j[t] is a sequence of events generated by a particular context sensor j, the ∥

.∥

operator represents events in binary process, and ^ is a Boolean intersection operator, to select the action based on how events coincide at a given instant in time.

9. The system of claim 7, further comprising:

means for selecting the entry a_jk according to;

$a_{\mathrm{jk}}=\arg <br><br>\underset{i\in <br><br>I}{\max }<br><br>\frac{<br><br>p_i<br><br>\left[t\right]\bigwedge <br><br>c_j<br><br>\left[t\right]\bigwedge <br><br>c_k<br><br>\left[t-\mathrm{\Delta <br><br>}<br><br><br><br>t\right]<br><br>}{<br><br>p_i<br><br>\left[t\right]<br><br>},$

where p_i[t] is a sequence of events generated by the camera in a particular pose corresponding to i, c_j[t] is a sequence of events generated by a first context sensor j, c_k[t] is a sequence of following events generated by a second context sensor k, the ∥

.∥

operator represents events in binary process, ^ is a Boolean intersection operator, t is an instant in time, and Δ

t is a particular time delay between detecting events with the first and second sensors, to model a dynamic relationships between the event sequences, delayed in time.

10. The system of claim 7, further comprising:

means for selecting the entry a_jk according to;

$a_{\mathrm{jk}}=\arg <br><br>\underset{i\in <br><br>I}{\max }<br><br>\frac{<br><br>p_i<br><br>\left[t\right]\bigwedge <br><br>c_j<br><br>\left[t\right]\bigwedge <br><br>\underset{\mathrm{\delta <br><br>}=0}{\overset{\mathrm{\Delta <br><br>}<br><br><br><br>t}{\bigcup <br><br>}}<br><br>c_k<br><br>\left[t-\mathrm{\delta <br><br>}\right]<br><br>}{<br><br>p_i<br><br>\left[t\right]<br><br>},$

where p_i[t] is a sequence of events generated by the camera in a particular pose corresponding to i, c_j[t] is a sequence of events generated by a first context sensor j, c_k[t] is a sequence of following events generated by a second context sensor k, the ∥

.∥

operator represents events in binary process, ^ is a Boolean intersection operator, t is an instant in time, Δ

t is a particular time delay, the operator ∪

is the union over the detected events, and δ

is a predetermined time period between a first event and a second event.