SYSTEM AND PROCESS FOR DETECTING, TRACKING AND COUNTING HUMAN OBJECTS OF INTEREST

• US 20160314597A1
• Filed: 07/01/2016
• Published: 10/27/2016
• Est. Priority Date: 07/03/2007
• Status: Active Grant

• Current Assignee: Shoppertrak RCT LLC (Johnson Controls International plc)
• Original Assignee: Shoppertrak RCT Corporation (Johnson Controls International plc)
• Inventors: Wang, Zhiqian, Kastilahn, William C.
• Granted Patent: US 10,558,890 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method of identifying, tracking, and counting human objects of interest based upon at least one pair of stereo image frames taken by at least one image capturing device, comprising the steps of: obtaining said stereo image frames and converting each said stereo image frame to a rectified image frame using calibration data obtained for said at least one image capturing device; generating a disparity map based upon a pair of said rectified image frames; generating a depth map based upon said disparity map and said calibration data; identifying the presence or absence of said objects of interest from said depth map and comparing each of said objects of interest to existing tracks comprising previously identified objects of interest; for each said presence of an object of interest, adding said object of interest to one of said existing tracks if said object of interest matches said one existing track, or creating a new track comprising said object of interest if said object of interest does not match any of said existing tracks; updating each said existing track; and maintaining a count of said objects of interest in a given time period based upon said existing tracks created or modified during said given time period.

Claims

1. A method of identifying and tracking objects of interest, comprising the steps of:

obtaining at least one pair of stereo image frames from more than one image capturing device;

identifying with the processor a presence or absence of one or more objects of interest within the at least one pair of stereo image frames;

comparing with a processor each of the objects of interest to existing tracks comprising previously identified objects of interest;

if an object of interest is present and the object of interest matches one of the existing tracks, adding the object of interest to one of the existing tracks;

if an object of interest is present and the object of interest does not match any of the existing tracks, creating a new track comprising the object of interest;

each existing track and new track including location information indicative of a zone correlating to an area where the object of interest was detected; and

using the location information to track a movement of the objects of interest.

2. The method of claim 1, wherein each track further comprises a set of additional track attributes, including a status attribute for each of the tracks, the status attribute further including one value selected from three possible values:

active, inactive, or deleted, wherein the status attribute is set to be active if the track comprises the object of interest in a most recent frame, inactive if the track does not comprise any object of interest in the most recent frame, and deleted if the track is marked for deletion.

3. The method of claim 1, wherein the at least one of the more than one image capturing device includes a stereo camera.

4. The method of claim 1, wherein a direction and speed of the objects of interest can be determined.

5. The method of claim 1, including the step of determining whether the objects of interest are moving into or out of the area.

6. A method of detecting and tracking objects of interest, comprising the steps of:

obtaining stereo image frames from more than one image capturing device;

identifying, by a processor, a presence or absence of detected objects of interest within one of the stereo image frames;

comparing with the processor each of the detected objects of interest to existing tracks comprising previously identified objects of interest;

each existing track including location information indicative of a zone correlating to a first area where the previously identified object of interest associated with the existing track was detected;

each new track including location information indicative of a zone correlating to a second area where the detected object of interest associated with the new track was detected; and

using the location information to track a movement of the objects of interest.

7. The method of claim 6, wherein at least one of the more than one image capturing devices includes a stereo camera.

8. The method of claim 6, wherein a direction and speed of the objects of interest can be determined.

9. The method of claim 6, including the step of determining whether the objects of interest are moving into or out of the first area.

10. The method of claim 6, including the step of determining whether the objects of interest are moving into or out of the second area.

11. The method of claim 6, wherein the step of comparing the detected objects of interest to existing tracks includes using information representative of color for the detected objects of interest.

12. The method of claim 6, wherein the step of comparing the detected objects of interest to existing tracks includes using gray level intensity information.

13. An object identifying, tracking, and counting system based upon image frames taken at a facility, the system comprising:

at least one image capturing device adapted to obtain image frames from the facility;

a boot loader module adapted to load programs into memory of the system and to execute the programs in the system;

a counting module adapted to compute height maps, object detections, object tracks, and object counts based upon the image frames; and

wherein, the counting module is adapted to;

obtain at least one pair of image frames from the at least one image capturing device,

identify a presence or absence of objects of interest,

compare each of the objects of interest to existing tracks comprising previously identified objects of interest,

if an object of interest is present and the object of interest matches one of the existing tracks, adding the object of interest to one of the existing tracks,

if an object of interest is present and the object of interest does not match any of the existing tracks, creating a new track comprising the object of interest;

each existing track and new track including location information indicative of a zone correlating to an area where the object of interest was detected; and

using the location information to track a movement of the objects of interest.

14. The system described in claim 13, wherein the at least one image capturing device includes a stereo camera.

15. The system described in claim 13, wherein a direction and speed of the object of interest can be determined.

16. The system described in claim 13, wherein the system can determine whether the objects of interest are moving into or out of the area.