AUTOMATICALLY CALIBRATING REGIONS OF INTEREST FOR VIDEO SURVEILLANCE

• US 20100110183A1
• Filed: 10/31/2008
• Published: 05/06/2010
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Pankanti, Sharathchandra Umapathirao, Kjeldsen, Frederik, Zhai, Yun, Fan, Quanfu, Yanagawa, Akira, Bobbitt, Russell Patrick
• Granted Patent: US 8,345,101 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

Techniques for automatically calibrating one or more regions of interest for video surveillance are provided. The techniques include at a user-defined frequency, determining if one or more regions of interest (ROIs) are present within a field of view of a camera, if one or more ROIs are present within the field of view of the camera, automatically calibrating the one or more ROIs within the field of view of the camera, and if one or more ROIs are not present within the field of view of the camera, sending an alert to a user.

Claims

1. A method for automatically calibrating one or more regions of interest for video surveillance, comprising the steps of:

at a user-defined frequency, determining if one or more regions of interest are present within a field of view of a camera;

if one or more regions of interest are present within the field of view of the camera, automatically calibrating the one or more regions of interest within the field of view of the camera; and

if one or more regions of interest are not present within the field of view of the camera, sending an alert to a user.

2. The method of claim 1, wherein sending an alert to a user comprises sending an alert to a user to manually adjust the camera.

3. The method of claim 1, wherein determining if one or more regions of interest are present within a field of view of a camera comprises:

acquiring a video image;

performing motion detection on the video image;

tracking one or more foreground moving objects in the video image;

extracting one or more features from the one of more foreground moving objects;

based on the one or more extracted features, generating a heat map for a camera field-of-view, wherein the heat map comprises motion probabilities and a temporal sequence order between one or more high motion interest regions; and

using the heat map to perform temporal matching between one or more visual activities and one or more received recorded activities.

4. The method of claim 3, further comprising outputting a set of one or more target regions of interest that correspond to one or more high-interest regions in the camera field-of-view.

5. The method of claim 3, further comprising, if the activity reflected in the heat map is inconsistent with an expected level of activity for the one or more regions of interest, sending an alert to a user.

6. The method of claim 1, wherein determining if one or more regions of interest are present within a field of view of a camera comprises using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

7. The method of claim 6, wherein using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view comprises:

collecting a set of one or more pre-selected region of interest models;

obtaining a video image;

extracting a set of one or more features from the video image;

using one or more spatial features of the video image to construct one or more region of interest templates; and

comparing the one or more templates with one or more feature maps of the video image.

8. The method of claim 7, further comprising:

locating one region in the camera field-of-view; and

using domain knowledge and the located region to locate one or more spatially related regions of interest.

9. The method of claim 7, further comprising outputting a set of one or more target regions of interest that correspond to one or more high-interest regions in the camera field-of-view.

10. The method of claim 1, further comprising validating that activity in a scanner region corresponds to transaction item entry events for a user-defined period of time.

11. The method of claim 10, further comprising, if the activity in a scanner region does not correspond to transaction item entry events for a user-defined period of time, sending an alert to a user.

12. The method of claim 1, wherein one or more regions of interest are drawn according to a location of one or more user-defined items.

13. The method of claim 12, wherein the one or more user-defined items comprise at least one of a checkout area, an entry area, an entry belt, a scan area, a scanner, a barcode scanner, an exit area, an exit belt, a cashier work area, a register, a customer area, a credit card terminal, a receipt printer, a lane open sign, a lane closed sign and a keyboard entry area.

14. The method of claim 1, further comprising processing the one or more regions of interest in an activity analysis module.

15. The method of claim 1, wherein determining if one or more regions of interest are present within a field of view of a camera comprises determining if one or more regions of interest at a retail checkout environment are present within the field of view of the camera.

16. A computer program product comprising a computer readable medium having computer readable program code for automatically calibrating one or more regions of interest for video surveillance, said computer program product including:

computer readable program code for, at a user-defined frequency, determining if one or more regions of interest are present within a field of view of a camera;

computer readable program code for, if one or more regions of interest are present within the field of view of the camera, automatically calibrating the one or more regions of interest within the field of view of the camera; and

computer readable program code for, if one or more regions of interest are not present within the field of view of the camera, sending an alert to a user.

17. The computer program product of claim 16, wherein the computer readable program code for determining if one or more regions of interest are present within a field of view of a camera comprises:

computer readable program code for acquiring a video image;

computer readable program code for performing motion detection on the video image;

computer readable program code for tracking one or more foreground moving objects in the video image;

computer readable program code for extracting one or more features from the one of more foreground moving objects;

computer readable program code for, based on the one or more extracted features, generating a heat map for a camera field-of-view, wherein the heat map comprises motion probabilities and a temporal sequence order between one or more high motion interest regions; and

computer readable program code for using the heat map to perform temporal matching between one or more visual activities and one or more received recorded activities.

18. The computer program product of claim 16, wherein the computer readable program code for determining if one or more regions of interest are present within a field of view of a camera comprises computer readable program code for using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

19. The computer program product of claim 18, wherein the computer readable program code for using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view comprises:

computer readable program code for collecting a set of one or more pre-selected region of interest models;

computer readable program code for obtaining a video image;

computer readable program code for extracting a set of one or more features from the video image;

computer readable program code for using one or more spatial features of the video image to construct one or more region of interest templates; and

computer readable program code for comparing the one or more templates with one or more feature maps of the video image.

20. The computer program product of claim 19, further comprising:

computer readable program code for locating one region in the camera field-of-view; and

computer readable program code for using domain knowledge and the located region to locate one or more spatially related regions of interest.

21. A system for automatically calibrating one or more regions of interest for video surveillance, comprising:

a memory; and

at least one processor coupled to said memory and operative to;

at a user-defined frequency, determine if one or more regions of interest are present within a field of view of a camera;

if one or more regions of interest are present within the field of view of the camera, automatically calibrate the one or more regions of interest within the field of view of the camera; and

if one or more regions of interest are not present within the field of view of the camera, send an alert to a user.

22. The system of claim 21, wherein in determining if one or more regions of interest are present within a field of view of a camera, the at least one processor coupled to said memory is further operative to:

acquire a video image;

perform motion detection on the video image;

track one or more foreground moving objects in the video image;

extract one or more features from the one of more foreground moving objects;

based on the one or more extracted features, generate a heat map for a camera field-of-view, wherein the heat map comprises motion probabilities and a temporal sequence order between one or more high motion interest regions; and

use the heat map to perform temporal matching between one or more visual activities and one or more received recorded activities.

23. The system of claim 21, wherein in determining if one or more regions of interest are present within a field of view of a camera, the at least one processor coupled to said memory is further operative to use one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view.

24. The system of claim 23, wherein in using one or more models of one or more target regions of interest to locate one or more similar regions of interest in a camera field-of-view, the at least one processor coupled to said memory is further operative to:

collect a set of one or more pre-selected region of interest models;

obtain a video image;

extract a set of one or more features from the video image;

use one or more spatial features of the video image to construct one or more region of interest templates; and

compare the one or more templates with one or more feature maps of the video image.

25. An apparatus for automatically calibrating one or more regions of interest for video surveillance, said apparatus comprising:

means for, at a user-defined frequency, determining if one or more regions of interest are present within a field of view of a camera;

means for, if one or more regions of interest are present within the field of view of the camera, automatically calibrating the one or more regions of interest within the field of view of the camera; and

means for, if one or more regions of interest are not present within the field of view of the camera, sending an alert to a user.