Method and system for automated labeling at scale of motion-detected events in video surveillance

• US 9,124,783 B2
• Filed: 09/27/2012
• Issued: 09/01/2015
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Camiolog, Inc.
• Original Assignee: Camiolog, Inc.
• Inventors: Maslan, Jonathan Carter
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US13/629,048
• Publication Number:
  US 20130083198A1
• Time in Patent Office: 1,069 Days
• Field of Search: 348/155
• US Class Current: 1/1
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A method and system are provided for automated labeling at scale of motion-detected events in video surveillance imagery that may be private to and viewable only by the owner of that imagery. Further provided are processing, organization, and archival aspects related to such automated labeling at scale.

Claims

1. A computer-implemented method for smart labeling at scale of motion detected events in video surveillance, comprising the steps of:

at a server, labeling at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and storing said label zone by;

receiving said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity;

upon authorizing and using said upload token, presenting said source image to said end user entity for labeling; and

receiving at least one label name of said at least one zone of said source image, correlating said at least one label name with said at least one zone for said source image, and storing said correlated labeled zone; and

at a server in real time, processing label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, said processing by;

receiving and decrypting an encrypted token that uniquely identifies said end user entity and said source camera;

upon authorizing said decrypted token, receiving at least a first incoming image frame, and at least a second incoming image frame from said source camera;

comparing one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame;

for each identified significant motion, determining a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays;

for each motion zone;

computing an amount of an overlap with a candidate labeled zone from said stored correlated labeled zones; and

measuring said overlap among said motion zone and said labeled zone, determining therefrom whether further granular analyses are required to be performed, and performing said further granular analyses when required;

generating and indexing a label hit for each matching motion zone with candidate labeled zone; and

generating tasks for each generated label hit;

wherein at least one of the steps is performed by a processor.

2. The computer-implemented method of claim 1, wherein said correlating said at least one label name with said at least one zone for said image source comprises translating said at least one labeled zone into percentage offsets from the edges of said source image.

3. The computer-implemented method of claim 1, wherein performing further granular analyses comprises:

performing another iteration by returning to said step of comparing objects wherein said comparing uses any of;

pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones.

4. The computer-implemented method of claim 1, wherein a task of said generated tasks is sending a notification to the end user entity.

5. The computer-implemented method of claim 1, wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of:

view, review, and rate the accuracy of said label hit.

6. An apparatus for smart labeling at scale of motion detected events in video surveillance, comprising:

a server adapted to label at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and to store said label zone by comprising;

a receiver adapted to receive said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity;

an authorizing processor adapted to authorize said token;

a presenter adapted to present, using said token, said source image to said end user entity for labeling;

a receiver adapted to receive at least one label name of said at least one zone of said source image, to correlate said at least one label name with said at least one zone for said image source, and store said correlated labeled zone; and

a server adapted to process, in real time, label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, by comprising;

a decryptor adapted to receive and decrypt an encrypted token that uniquely identifies said end user entity and said source camera;

a receiver adapted to receive, upon authorization of said decrypted token, at least a first incoming image frame and at least a second incoming image frame from said source camera;

a comparing component adapted to compare one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame;

a determiner adapted to determine, for each identified significant motion, a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays;

for each motion zone;

a computing component adapted to compute an amount of an overlap with a candidate labeled zone from said stored correlated labeled zones, and measure said overlap among said motion zone and said labeled zone, determine therefrom whether further granular analyses are required to be performed, and perform said further granular analyses when required; and

a generator adapted to generate and index a label hit for each matching motion zone with candidate labeled zone, and generate tasks for each generated label hit.

7. The apparatus of claim 6, wherein said correlate said at least one label name with said at least one zone for said image source comprises to translate said at least one labeled zone into percentage offsets from the edges of said source image.

8. The apparatus of claim 6, wherein to perform further granular analyses comprises to perform another iteration by returning to said step of comparing objects wherein said comparing uses any of:

pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones.

9. The apparatus of claim 6, wherein a task of said generated tasks is sending a notification to the end user entity.

10. The apparatus of claim 6, wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of:

view, review, and rate the accuracy of said label hit.

11. A non-transitory computer readable storage medium having stored thereon a computer program for smart labeling at scale of motion detected events in video surveillance, said computer program comprising a program code which, when executed by a processor, performs the steps of:

at a server, labeling at least one zone of a source image, said source image from a source camera of a plurality of source cameras, and storing said label zone by;

receiving said source image with a corresponding upload token, wherein said upload token contains data identifying said source camera and an end user entity;

upon authorizing and using said upload token, presenting said source image to said end user entity for labeling; and

receiving at least one label name of said at least one zone of said source image, correlating said at least one label name with said at least one zone for said source image, and storing said correlated labeled zone; and

at a server in real time, processing label hits for incoming images from a plurality of source cameras, said plurality comprising said source camera, said processing by;

receiving and decrypting an encrypted token that uniquely identifies said end user entity and said source camera;

upon authorizing said decrypted token, receiving at least a first incoming image frame, and at least a second incoming image frame from said source camera;

comparing one or more objects on said at least second incoming image frame with one or more objects on said at least first incoming image frame and identifying significant motion based on said comparison, wherein significant motion is identified in part by computing a delta of change between any of said objects on said at least second incoming image frame and said objects on said at least first incoming image frame;

for each identified significant motion, determining a motion zone, wherein said motion zone is the zone of said at least second incoming image frame on which said significant motion overlays;

for each motion zone;

computing an amount of an overlap with a candidate labeled zone from said stored correlated labeled zones; and

measuring said overlap among said motion zone and said labeled zone, determining therefrom whether further granular analyses are required to be performed, and performing said further granular analyses when required;

generating and indexing a label hit for each matching motion zone with candidate labeled zone; and

generating tasks for each generated label hit.

12. The non-transitory computer readable storage medium of claim 11, wherein said program code further comprises correlating said at least one label name with said at least one zone for said image source comprising translating said at least one labeled zone into percentage offsets from the edges of said source image.

13. The non-transitory computer readable storage medium of claim 11, wherein said program code further comprises performing further granular analyses by performing another iteration by returning to said step of comparing objects wherein said comparing uses any of:

pixel resolution in analysis of specific bounding boxes, image filtering parameters, and degree of computational effort in searching for contiguous zones.

14. The non-transitory computer readable storage medium of claim 11, wherein a task of said generated tasks is sending a notification to the end user entity.

15. The non-transitory computer readable storage medium of claim 11, wherein label hits are correlated each with any one of source images sourced from a particular camera of a particular user and wherein the user is able to perform any of:

view, review, and rate the accuracy of said label hit.