Video object fragmentation detection and management

• US 8,611,590 B2
• Filed: 12/23/2009
• Issued: 12/17/2013
• Est. Priority Date: 12/23/2008
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Pakulski, Peter Jan, Wedge, Daniel John, Partis, Ashley, See, David Kenji
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US12/645,611
• Publication Number:
  US 20100157089A1
• Time in Patent Office: 1,455 Days
• Field of Search: 382/103, 348169-172
• US Class Current: 382/103
• CPC Class Codes: G06T 7/248 involving reference images or patches

Abstract

Disclosed herein are a computer-implemented method and a camera system for determining a current spatial representation for a detection in a current frame of an image sequence. The method derives an expected spatial representation (820) for the detection based on at least one previous frame, generates a spatial representation (810) of the detection, and extends the spatial representation (810) to obtain an extended spatial representation (830), based on the expected spatial representation (820). The method determines a similarity measure between the extended spatial representation (830) and the expected spatial representation (820), and then determines the current spatial representation for the detection to based on the similarity measure.

Claims

1. A computer-implemented method of determining a current spatial representation for a detection in a current frame of an image sequence, said method comprising the steps of:

deriving an expected spatial representation for said detection based on at least one previous frame;

generating a spatial representation of said detection;

extending said spatial representation to obtain an extended spatial representation, based on the expected spatial representation;

determining a similarity measure between said extended spatial representation and said expected spatial representation; and

determining the current spatial representation for the detection based on said similarity measure.

2. The method according to claim 1, wherein said extending step includes at least one of:

extending at least one dimension of said spatial representation of said detection;

augmenting said spatial representation of said detection with an augmenting spatial representation; and

extending said expected spatial representation to obtain the extended spatial representation, based on said spatial representation of said detection.

3. The method according to claim 1, comprising the further step of:

associating said extended spatial representation with a track, if the determined similarity measure exceeds a predetermined threshold.

4. The method according to claim 1, comprising the further step of:

classifying said detection as one of a false positive and a true positive, based on the determined similarity measure.

5. The method according to claim 1, comprising the further step of:

determining an initial spatial similarity between said spatial representation of said detection and said expected spatial representation;

wherein said steps of extending said spatial representation and determining said similarity measure are performed if the initial spatial similarity exceeds a direct association threshold.

6. The method according to claim 5, wherein said steps of determining said initial spatial similarity and determining the similarity measure are performed using the same spatial similarity method.

7. The method according to claim 4, wherein said track is derived from at least one image sequence.

8. A computer-implemented method of associating at least one of a plurality of detections in a frame of an image sequence with a track, said method comprising the steps of:

for each one of said plurality of detections;

determining an initial spatial similarity measure between an expected spatial representation associated with said track and a spatial representation of said detection;

determining an extended similarity measure between said expected spatial representation of said track and an extended spatial representation of said detection; and

associating said detection with said track, based on at least one of;

(i) said initial spatial similarity measure exceeding a direct association threshold; and

(ii) said extended similarity measure exceeding a fragment similarity threshold.

9. The method according to claim 8, comprising the further steps of:

generating a plurality of association hypotheses, each association hypothesis including an association between said track and at least one detection from a set of detections derived from said frame, wherein generating each one of said plurality of association hypotheses includes the steps of;

selecting, from the set of detections, a combination of detections to be associated with that respective association hypothesis; and

calculating a matching similarity score, for that respective association hypothesis, between said track and a spatial representation of said selected combination of detections; and

selecting one of said plurality of association hypotheses, based on said respective matching similarity scores, wherein said plurality of detections correspond to said combination of detections associated with said selected association hypothesis.

10. The method according to claim 8, wherein:

each detection of said set of detections corresponds to at most one association hypothesis.

11. The method according to claim 8, wherein:

each one of said expected spatial representations corresponds to at most one association hypothesis.

12. The method according to claim 8, wherein:

calculating each matching similarity score is based on;

(i) a number of detections in the combination of detections associated with the respective association hypothesis, and

(ii) at least one of;

area, density, geometric distance, height, width or visual properties of the spatial representation of said selected combination of detections.

13. A camera system for classifying a detection in a frame of an image sequence as a fragment of a video object in said frame, based on a spatial similarity between said detection and an expected spatial representation for a track associated with said image sequence, said camera system comprising:

a lens system;

a camera module coupled to said lens system to capture at least one frame in said image sequence;

a storage device for storing a computer program; and

a processor for executing the program, said program comprising;

code for deriving an extended spatial representation from a spatial representation of said detection, wherein at least one dimension of said extended spatial representation is at least as large as a corresponding dimension of said expected spatial representation;

code for determining an extended spatial similarity measure between said extended spatial representation and said expected spatial representation; and

code for determining said detection to be a fragment of said video object, if the extended spatial similarity measure exceeds a fragment similarity threshold.

14. The camera system according to claim 13, wherein said program further comprises:

code for detecting said detection in said frame by utilizing a foreground separation method; and

code for determining said spatial representation of said detection.

15. A computer-implemented method of associating an association hypothesis with a track associated with an image, based on a plurality of fragments detected in said image, said method comprising the steps of:

generating a plurality of association hypotheses based on said track and said plurality of fragments detected in the image, each association hypothesis including an expectation associated with said track and a combination of at least one fragment associated with said expectation;

for each association hypothesis, performing the steps of;

determining a matching similarity score, based on said expectation and at least one characteristic of at least one fragment in the combination in the association hypothesis;

determining a value to adjust the matching similarity score corresponding to a number of fragments in the combination in the association hypothesis; and

adjusting the matching similarity score based on the determined value;

selecting an association hypothesis from said plurality of association hypotheses, based on the adjusted matching similarity scores of the association hypotheses; and

associating said selected association hypothesis with said track.

16. The method according to claim 15, wherein said at least one characteristic is selected from the group of characteristics consisting of:

area, density, geometric distance, height, width, and visual properties of said at least one fragment.

17. The method according to claim 16, wherein adjusting the matching similarity score includes applying a bias.

18. A computer-implemented method of associating an association hypothesis with a track associated with an image, based on a fragment detected in said image and a plurality of tracks associated with said image, said method comprising the steps of:

generating a plurality of association hypotheses based on said fragment and said plurality of tracks, each association hypothesis including said fragment detection and a combination of at least one expectation associated with said detection, wherein each expectation is associated with one of said tracks;

for each association hypothesis, performing the steps of;

determining a similarity score for each association hypothesis, based on said fragment detection and at least one characteristic of at least one track in the combination in the association hypothesis;

determining a value to adjust the determined similarity score corresponding to a number of track expectations included in the hypothesis; and

adjusting the similarity score based on the determined value;

selecting an association hypothesis, based on the similarity scores for the association hypotheses; and

associating said selected association hypothesis with said track.

19. A method of determining a detection to be a fragment of a video object in a video frame sequence, based on a spatial similarity between a track associated with said video frame sequence and said detection in a video frame of said video frame sequence, said method comprising the steps of:

deriving an extended spatial representation from a spatial representation of said detection, wherein at least one dimension of said extended spatial representation is at least as large as a corresponding dimension of an expected spatial representation of said track;

determining an extended spatial similarity measure between said extended spatial representation and said expected spatial representation; and

determining said detection to be a fragment of said video object when said extended spatial similarity measure exceeds an extended representation similarity threshold.