Automatically calibrating regions of interest for video surveillance

• US 8,345,101 B2
• Filed: 10/31/2008
• Issued: 01/01/2013
• Est. Priority Date: 10/31/2008
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bobbitt, Russell Patrick, Fan, Quanfu, Hampapur, Arun, Kjeldsen, Frederik, Pankanti, Sharathchandra Umapathirao, Yanagawa, Akira, Zhai, Yun
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): DRABIK, SARAH E
• Application Number:
  US12/262,446
• Publication Number:
  US 20100110183A1
• Time in Patent Office: 1,523 Days
• Field of Search: 348/143, 348/150, 348/157, 348/169
• US Class Current: 348/150
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

Techniques for automatically calibrating one or more regions of interest for video surveillance are provided. The techniques include at a user-defined frequency, determining if one or more regions of interest (ROIs) are present within a field of view of a camera, if one or more ROIs are present within the field of view of the camera, automatically calibrating the one or more ROIs within the field of view of the camera, and if one or more ROIs are not present within the field of view of the camera, sending an alert to a user.

Claims

1. A method for automatically calibrating one or more regions of interest for video surveillance, comprising the steps of:

at a user-defined frequency, determining if one or more regions of interest are present within a field of view of a camera via;

performing motion detection on a video image;

tracking one or more foreground moving objects in the video image;

extracting one or more features from the one or more foreground moving objects;

generating a spatiotemporal heat map for the camera field-of-view based on the one or more extracted features, with values in the heat map representing motion probabilities at corresponding image locations to produce a temporal sequence order between multiple high motion interest regions; and

temporally matching one or more visual activities from the video image based on the heat map and one or more recorded activities received from a source independent of the video image;

if one or more regions of interest are present within the field of view of the camera, automatically calibrating the one or more regions of interest within the field of view of the camera by positioning the one or more regions of interest within the field of view of the camera; and

if one or more regions of interest are not present within the field of view of the camera, sending an alert to a user to assess status of the camera.

2. The method of claim 1, wherein sending an alert to a user comprises sending an alert to a user to manually adjust the camera.

3. The method of claim 1, further comprising outputting a set of one or more target regions of interest that correspond to one or more high-interest regions in the camera field-of-view.

4. The method of claim 1, further comprising, if an activity reflected in the heat map is inconsistent with an expected level of activity for the one or more regions of interest, sending an alert to a user.

5. The method of claim 1, wherein determining if one or more regions of interest are present within a field of view of a camera comprises using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

6. The method of claim 5, wherein using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view comprises:

collecting a set of one or more pre-selected region of interest models;

obtaining a video image;

extracting a set of one or more features from the video image;

using one or more spatial features of the video image to construct one or more region of interest templates; and

comparing the one or more templates with one or more feature maps of the video image.

7. The method of claim 6, further comprising:

locating one region in the camera field-of-view; and

using domain knowledge and the located region to locate one or more spatially related regions of interest.

8. The method of claim 6, further comprising outputting a set of one or more target regions of interest that correspond to one or more high-interest regions in the camera field-of-view.

9. The method of claim 1, further comprising validating that activity in a scanner region corresponds to transaction item entry events for a user-defined period of time.

10. The method of claim 9, further comprising, if the activity in a scanner region does not correspond to transaction item entry events for a user-defined period of time, sending an alert to a user.

11. The method of claim 1, wherein one or more regions of interest are drawn according to a location of one or more user-defined items.

12. The method of claim 11, wherein the one or more user-defined items comprise at least one of a checkout area, an entry area, an entry belt, a scan area, a scanner, a barcode scanner, an exit area, an exit belt, a cashier work area, a register, a customer area, a credit card terminal, a receipt printer, a lane open sign, a lane closed sign and a keyboard entry area.

13. The method of claim 1, further comprising processing the one or more regions of interest in an activity analysis module.

14. The method of claim 1, wherein determining if one or more regions of interest are present within a field of view of a camera comprises determining if one or more regions of interest at a retail checkout environment are present within the field of view of the camera.

15. A computer program product comprising a non-transitory tangible computer readable recordable storage medium having computer readable program code for automatically calibrating one or more regions of interest for video surveillance, said computer program product including:

computer readable program code for, at a user-defined frequency, determining if one or more regions of interest are present within a field of view of a camera via;

performing motion detection on a video image;

tracking one or more foreground moving objects in the video image;

extracting one or more features from the one or more foreground moving objects;

generating a spatiotemporal heat map for the camera field-of-view based on the one or more extracted features, with values in the heat map representing motion probabilities at corresponding image locations to produce a temporal sequence order between multiple high motion interest regions; and

temporally matching one or more visual activities from the video image based on the heat map and one or more recorded activities received from a source independent of the video image;

computer readable program code for, if one or more regions of interest are present within the field of view of the camera, automatically calibrating the one or more regions of interest within the field of view of the camera by positioning the one or more regions of interest within the field of view of the camera; and

computer readable program code for, if one or more regions of interest are not present within the field of view of the camera, sending an alert to a user to assess status of the camera.

16. The computer program product of claim 15, wherein the computer readable program code for determining if one or more regions of interest are present within a field of view of a camera comprises computer readable program code for using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

17. The computer program product of claim 16, wherein the computer readable program code for using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view comprises:

computer readable program code for collecting a set of one or more pre- selected region of interest models;

computer readable program code for obtaining a video image;

computer readable program code for extracting a set of one or more features from the video image;

computer readable program code for using one or more spatial features of the video image to construct one or more region of interest templates; and

computer readable program code for comparing the one or more templates with one or more feature maps of the video image.

18. The computer program product of claim 17, further comprising:

computer readable program code for locating one region in the camera field-of-view; and

computer readable program code for using domain knowledge and the located region to locate one or more spatially related regions of interest.

19. A system for automatically calibrating one or more regions of interest for video surveillance, comprising:

a memory; and

at least one processor coupled to said memory and operative to;

at a user-defined frequency, determine if one or more regions of interest are present within a field of view of a camera via;

performing motion detection on a video image;

tracking one or more foreground moving objects in the video image;

extracting one or more features from the one or more foreground moving objects;

generating a spatiotemporal heat map for the camera field-of-view based on the one or more extracted features, with values in the heat map representing motion probabilities at corresponding image locations to produce a temporal sequence order between multiple high motion interest regions; and

temporally matching one or more visual activities from the video image based on the heat map and one or more recorded activities received from a source independent of the video image;

if one or more regions of interest are present within the field of view of the camera, automatically calibrate the one or more regions of interest within the field of view of the camera by positioning the one or more regions of interest within the field of view of the camera; and

if one or more regions of interest are not present within the field of view of the camera, send an alert to a user to assess status of the camera.

20. The system of claim 19, wherein in determining if one or more regions of interest are present within a field of view of a camera, the at least one processor coupled to said memory is further operative to use one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

21. The system of claim 20, wherein in using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view, the at least one processor coupled to said memory is further operative to:

collect a set of one or more pre-selected region of interest models;

obtain a video image;

extract a set of one or more features from the video image;

use one or more spatial features of the video image to construct one or more region of interest templates; and

compare the one or more templates with one or more feature maps of the video image.