Method and system for full path analysis

• US 10,163,031 B2
• Filed: 09/05/2012
• Issued: 12/25/2018
• Est. Priority Date: 02/29/2012
• Status: Expired due to Fees

• Current Assignee: RetailNext, Inc.
• Original Assignee: RetailNext, Inc.
• Inventors: Shaw, George
• Primary Examiner(s): Harold, Jefferey F
• Assistant Examiner(s): Karwan, Sihar A
• Application Number:
  US13/603,832
• Publication Number:
  US 20130222599A1
• Time in Patent Office: 2,302 Days
• Field of Search: 356 401, 348159
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A space is modeled using images from a set of cameras in the space. An easy to use tool is provided that allows users to identify a reference location in an image and a corresponding reference location on a floor plan of the space. Based on these correspondences, a model is generated that can map a point in a camera'"'"'s view to a point on the floor plan, or vice-versa. Subjects moving through the space are identified and tracked using the model and attributes associated with a subject. In a specific implementation, the attributes include velocity, color, size, and position. In a specific implementation, the space is a retail store and the subjects are customers browsing through the retail store.

Claims

1. A method comprising:

identifying a base track of a tracked subject, the base track being identified as a first track in a first camera space based on images of the tracked subject captured by a first camera in a space, the first track being a first superimposed line in the first camera space that represents locations occupied by the tracked subject in the first camera space over a fixed period of time;

identifying a first candidate track of a first candidate subject from a plurality of candidate tracks captured by a second camera, the first candidate track being identified as a second track in a second camera space based on images of the first candidate subject captured by the second camera in the space, the second track being a second superimposed line in the second camera space that represents locations occupied by the first candidate subject in the second camera space over the fixed period of time;

identifying a second candidate track of a second candidate subject from the plurality of candidate tracks captured by the second camera, the second candidate track being identified as a third track in a second camera space based on images of the second candidate subject captured by the second camera in the space, the third track being a third superimposed line in the second camera space that represents locations occupied by the second candidate subject in the second camera space over the fixed period of time;

generating a model comprising camera-space data to real-space data correspondence that is independent of camera lens type, the camera-space data having coordinates measured in pixels and the real-space data having coordinates measured in lengths, the model being generated by mapping a set of calibration points identified in each camera-space of each camera and a set of corresponding calibration points identified in a floor plan representing the real-space;

applying the model comprising the camera-space to real-space correspondence mapping data to project the base track, the first candidate track, and the second candidate track onto a the floor plan of the space;

comparing a first set of attributes associated with the tracked subject, a second set of attributes associated with the first candidate subject and a third set of attributes associated with the second candidate subject, the comparing comprising;

scoring the projected base track and projected first candidate track by calculating a first plurality of differences between the first set of attributes and the second set of attributes and applying weights to the plurality of differences such that a color attribute difference is weighted more than the other attributes to determine a score for the projected first candidate track; and

scoring the projected base track and projected second candidate track by calculating a second plurality of differences between the first set of attributes and the third set of attributes and applying the same weights as applied to the first plurality of differences to determine a score for the projected second candidate track;

determining, based upon the comparison, that the first candidate subject is the same as the tracked subject when the score for the projected first candidate track is greater than a user-specified threshold; and

joining, based upon the determination that the first candidate subject is the same as the tracked subject, the projected first candidate track to the projected base track to create a continuous track of the tracked subject on the floor plan of the space.

2. The method of claim 1 wherein the mapping data is based on a plurality calibration points placed on each view of each camera and a plurality of corresponding calibration points on placed on the floor plan, and each calibration point and corresponding calibration point is placed by a user using a graphical user interface tool.

3. The method of claim 1 wherein the base, first candidate, and second candidate tracks are defined in a camera-space coordinate system and after the applying the model, the projected base, and first candidate, and second candidate tracks are defined in a real-space coordinate system.

4. The method of claim 1 wherein a first attribute of each of the first, second and third sets of attributes comprises color and the analyzing a set of attributes comprises:

measuring a difference between a color of the tracked subject and a color of the first candidate subject; and

measuring a difference between the color of the tracked subject and a color of the second candidate subject.

5. The method of claim 1 wherein a first attribute of each of the first, second and third sets of attributes comprises size and the analyzing a set of attributes comprises:

measuring a difference between a size of the tracked subject and a size of the first candidate subject; and

measuring a difference between the size of the tracked subject and a size of the second candidate subject.

6. The method of claim 1 wherein the size comprises a height and a width.

7. The method of claim 1 wherein a first attribute of each of the first, second and third sets of attributes comprises position and the analyzing a set of attributes comprises:

measuring a difference between a position of the projected base track and a position of the projected first candidate track; and

measuring a difference between the position of the projected base track and the position of the projected second candidate track.

8. The method of claim 1 wherein a first attribute of each of the first, second and third sets of attributes comprises velocity and the analyzing a set of attributes comprises:

measuring a difference between a velocity of the projected base track and a velocity of the projected first candidate track; and

measuring a difference between the velocity of the projected base track and a velocity of the projected second candidate track.

9. The method of claim 1 wherein the analyzing a set of attributes comprises:

scoring the projected base track and projected first candidate track by;

calculating a first difference in velocity between the tracked subject and first candidate subject and applying a first weight to the first difference;

calculating a second difference in color between the tracked subject and first candidate subject and applying a second weight to the second difference;

calculating a third difference in size between the tracked subject and first candidate subject and applying a third weight to the third difference; and

calculating a fourth difference in position between the tracked subject and first candidate subject and applying a fourth weight to the fourth difference; and

scoring the projected base track and projected second candidate track by;

calculating a fifth difference in velocity between the tracked subject and second candidate subject and applying the first weight to the fifth difference;

calculating a sixth difference in color between the tracked subject and second candidate subject and applying the second weight to the sixth difference;

calculating a seventh difference in size between the tracked subject and second candidate subject and applying the third weight to the seventh difference; and

calculating an eighth difference in position between the tracked subject and second candidate subject and applying the fourth weight to the eighth difference.