Image management system

• US 9,892,325 B2
• Filed: 09/11/2015
• Issued: 02/13/2018
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: iOmniscient Pty Ltd.
• Original Assignee: iOmniscient Pty Ltd.
• Inventors: Kanga, Rustom Adi, Li, Ivy Lai Chun
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/851,736
• Publication Number:
  US 20160078302A1
• Time in Patent Office: 886 Days
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

The present invention relates to systems and processes for the management of image data captured by a network covering a plurality of imaging nodes. The systems and processes may include a surveillance network arranged to receive surveillance streams from a plurality of checkpoints that are spatially separated along a route, extracting image data containing facial image content from the surveillance streams, and identifying individual people and determining an elapsed time between distinct checkpoints along the route, to track progress of people along the route.

Claims

1. A surveillance process comprising, by a computer system:

receiving surveillance streams from a plurality of checkpoints that are spatially separated along a route, each of the surveillance streams comprising a sequence of image frames,

automatically detecting image content in sub-frame sections of the surveillance streams and generating profiles from sub-frame sections that contain facial content for individual people,

storing the profiles in system memory with time data, derived from a corresponding image frame, that define a plurality of temporal references at each checkpoint location,

matching profiles derived from surveillance streams captured at distinct checkpoints along the route to determine the elapsed time at checkpoint locations, and

determining a dwell time for individuals, based on determining the elapsed times between the checkpoint locations for individual people from the time data and checkpoint location data stored with each set of matching profiles, whereby the determined dwell time may be utilized to inform one of operational and security processes associated with a location being surveyed by the surveillance process.

2. The process of claim 1 comprising, by the computing system, isolating sub-frame sections that contain facial characteristics of detected people and storing the isolated sub-frame sections in system memory.

3. The process of claim 1 comprising, by the computing system, quantifying facial characteristics of people detected in the surveillance streams and compiling profiles from the quantified facial characteristic data.

4. The process of claim 3 comprising, by the computing system, identifying a predefined subset of facial features for people and generating profiles comprising quantified facial data derived from the predefined subset of facial features.

5. The process of claim 3 comprising, by the computing system, generating the profiles from the relative position, size and/or shape of a detected person'"'"'s eyes, nose, cheekbones and/or jaw.

6. The process of claim 3 comprising matching profiles derived from surveillance streams captured at distinct checkpoints to track a person'"'"'s progress along the route.

7. The process of claim 1 comprising, by the computing system, normalizing sub-frame sections containing facial image content using a repository of facial images and identifying distinctive image characteristics from the sub-frame sections.

8. The process of claim 7 comprising, by the computing system, recording the distinctive image characteristics in profiles and matching the profiles derived from surveillance streams captured at distinct checkpoints to track a person'"'"'s progress along the route.

9. The process of claim 1 comprising, by the computing system, detecting a plurality of individual people within a single frame of a surveillance stream and generating profiles for detected people from corresponding sub-frame sections of the image frame.

10. The process of claim 1 comprising, by the computing system, identifying facial image content for an individual person captured in a plurality of consecutive image frames from a single surveillance stream and comparatively evaluating the facial image content captured in each of the consecutive frames.

11. The process of claim 10 comprising, by the computing system, compiling a profile from image data selectively extracted from one of the consecutive image frames.

12. The process of claim 10 comprising, by the computing system, compiling a consolidated profile from image data extracted from a plurality of the consecutive image frames.

31. The process of claim 1 comprising, by the computing system, determining dwell times for a plurality of individuals and determining an average dwell time based on the plurality of determined dwell times.

13. A surveillance process comprising:

receiving images of a person from a plurality of checkpoints at checkpoint positions that are spatially separated along a route,

matching facial image data derived from images captured at distinct checkpoints along the route to track progression of the person and the time spent at each checkpoint position, and

determining an elapsed time for the person between the distinct checkpoint positions using a temporal reference derived from corresponding images of the person, at the checkpoint positions, and

determining a dwell time for the person, based on the elapsed time, whereby the determined dwell time may be utilized to inform one of operational and security processes associated with a location being surveyed by the surveillance process.

14. The process of claim 13 comprising detecting image content within sub-frame sections of a surveillance stream and generating profiles from sub-frame sections that contain facial content for individual people.

15. The process of claim 14 comprising quantifying facial characteristics of people detected in the surveillance streams and compiling profiles from the quantified facial characteristic data.

16. The process of claim 15 comprising identifying a predefined subset of facial features for a detected person and generating a profile comprising quantified facial data derived from the predefined subset of facial features.

17. A surveillance system comprising a computing system that is configured to receive surveillance streams from a plurality of imaging nodes disposed at spatially separated checkpoint positions along a route and track the progress of people along the route using elapsed time between distinct checkpoints, the computing system having an image processing module that is configured to generate profiles for individual people captured within frames of the surveillance streams and match profiles derived from distinct surveillance streams to determine an elapsed time at checkpoints for a corresponding person, and to determine a dwell time for the corresponding person based on the elapsed times and the check point locations, whereby the determined dwell time may be utilized to inform one of operational and security processes associated with the location being surveyed by the surveillance system.

18. The system of claim 17 comprising a recording module that is configured to isolate sub-frame sections captured within the surveillance streams that contain facial characteristics of detected people, and store the isolated sub-frame sections in system memory with time data that defines a temporal reference for the corresponding image fame.

19. The system of claim 17 comprising a profiling engine that is configured to quantify facial characteristics of people captured within the surveillance streams and compile profiles from quantified facial characteristic data for each of the people.

20. The system of claim 18 comprising a profiling engine that is configured to identify a predefined subset of facial features captured in the isolated sub-frame sections and generate profiles comprising quantified facial data derived from the predefined subset of facial features.

21. The system of claim 19 wherein the profiling engine is configured to generate profiles from the relative position, size and/or shape of a detected person'"'"'s eyes, nose, cheekbones and/or jaw.

22. The system of claim 19 comprising a matching engine that is configured to compare profiles derived from surveillance streams captured at distinct checkpoints to track progress of people along the route.

23. The system of claim 18 comprising a profiling engine that is configured to normalize the isolated sub-frame sections using a repository of facial images, and identify distinctive image characteristics from the sub-frame sections.

24. The system of claim 23 comprising a matching engine that is configured to record the distinctive image characteristics in profiles, and match profiles derived from surveillance streams captured at distinct checkpoints to track progress of people along the route.

25. The system of claim 17 comprising a tracking engine that is configured to detect a plurality of individual people within a single frame of a surveillance stream, and generate profiles for detected people.

26. The system of claim 17 comprising a compliance engine that is configured to identify facial image content for an individual person captured in a plurality of consecutive image frames from a single surveillance stream at a checkpoint position, and comparatively evaluate the facial image content captured in each of the consecutive frames.

27. The system of claim 26 comprising a profiling engine that is configured to compile a profile from image data selectively extracted from one of the consecutive image frames.

28. The system of claim 26 comprising a profiling engine that is configured to compile a consolidated profile from image data extracted from a plurality of the consecutive image frames.

32. The system of claim 17, wherein the image processing module is arranged to determine dwell times for a plurality of people and to determine an average dwell time based on the dwell times for the plurality of people.

29. A surveillance system comprising:

a network of imaging nodes disposed along a route at spatially separated checkpoint positions, each of the imaging nodes having a camera that is configured to capture a localized surveillance stream comprising a sequence of image frames and a processing unit that is configured to generate profiles from the corresponding surveillance stream, the checkpoint positions and a plurality of temporal references associated with the image frames, and

a computing system that is configured to receive profiles from the imaging nodes and track the progress of individual people along the route using elapsed time between distinct checkpoint positions, the computing system having a processor that is configured to match profiles derived from distinct checkpoint positions and determines the elapsed time from the corresponding surveillance streams, the checkpoint positions and the temporal references associated with the image frames provided by the surveillance streams, and to determine dwell times or individuals, based on the elapsed times at the checkpoint locations, whereby the determined dwell times may be utilized to inform one of operational and security processes associated with a location being surveyed by the surveillance process.

30. A non-transitory computer readable medium with stored instructions that, when executed by a computing system, cause the computing system to perform a process comprising:

receiving images of a person from a plurality of checkpoint positions that are spatially separated along a route,

matching facial image data derived from images captured at distinct checkpoint positions along the route and temporal data associated with the image data to track progression of the person, and

determining an elapsed time for the person between the distinct checkpoints using the temporal reference derived from corresponding images of the person, the checkpoint position data, and determining a dwell time for the person based on the elapsed time at the checkpoint locations, whereby the determined dwell time may be utilized to inform one of operational and security processes associated with a location being surveyed.