Multi-camera vehicle identification system

• US 9,449,258 B1
• Filed: 07/02/2015
• Issued: 09/20/2016
• Est. Priority Date: 07/02/2015
• Status: Active Grant

• Current Assignee: AGT International GMBH
• Original Assignee: AGT International GMBH
• Inventors: Palacio, Sebastian, Hohmann, Sebastian, Krauss, Stephan, Hirzel, Jan, Stricker, Didier, Schlattmann, Markus
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US14/790,323
• Time in Patent Office: 446 Days
• Field of Search: 382/103, 382/104, 382/107
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A target object captured by a first camera is identified in images captured by a second camera. A reference platoon comprising the target object and other objects is generated using first camera images. A reference group characterizing the objects in the reference platoon is generated by running a first set of trained classifiers over the reference platoon, the first set of trained classifiers trained to characterize objects captured by the first camera. Candidate platoons are generated using second camera images. Candidate groups characterizing objects in the candidate platoons are obtained by running an independently trained second set of classifiers over the candidate platoons, the second set characterizing objects captured by the second camera. Candidate groups are compared to the reference group, and a best matching candidate platoon is selected. The target object is identified in the selected candidate platoon based on the object'"'"'s position in the reference platoon.

Claims

1. A method of identifying in one or more images captured by a second camera a target object captured by a first camera, the method implemented on a processor and comprising:

storing in a non-transitory computer readable memory operatively coupled to the processor a first set of trained classifiers and a second set of trained classifiers, wherein the 1^st set specifies values corresponding to a first plurality of attributes usable for identifying objects captured by the first camera, and the second set differs from the first set and specifies values corresponding to a second plurality of attributes usable for identifying objects captured by the second camera, wherein the first set of trained classifiers and second set of trained classifiers are trained independently, and wherein the first plurality of attributes and second plurality of attributes have at least one attribute in common;

generating, by the processor, using one or more images captured by the 1^st camera, a reference platoon of n objects, the reference platoon comprising the target object and (n−

1) other objects;

generating, by the processor, a reference group by running the first set of trained classifiers over the reference platoon, the reference group being indicative of values of attributes specified by the 1^st set of trained classifiers and characterizing the objects in the reference platoon;

generating, by the processor, using one or more images captured by the 2^nd camera, a plurality of candidate platoons, each candidate platoon comprising n objects, wherein the one or more images are captured by the 2^nd camera in a time window corresponding to the time of capturing by the 1^st camera the one or more images used for generating the reference platoon;

generating, by the processor, a plurality of candidate groups, each candidate group obtained by running the 2^nd set of trained classifiers over a respective candidate platoon, each candidate group being indicative of values of attributes specified by the 2^nd set of trained classifiers and characterizing the objects in the corresponding candidate platoon;

selecting, by the processor, a candidate platoon corresponding to a candidate group best matching the reference group;

identifying, by the processor, the target object in the selected candidate platoon in accordance with a position of the target object in the reference platoon.

2. The method of claim 1, wherein the reference group is a reference sequence, and wherein the candidate groups are candidate sequences.

3. The method of claim 2 further comprising associating, by the processor, each candidate sequence with a distance metric indicative of the sequence level distance between the given candidate sequence and the reference sequence, and wherein the candidate sequence best matching the reference sequence is the candidate sequence associated with the lowest distance metric.

4. The method of claim 3, wherein the sequence level distance is equal to the sum of object level distances for each pair of corresponding objects in the candidate sequence-reference sequence pair, and wherein an object level distance for a given pair of objects is the sum of the attribute level distances for the given pair of objects.

5. The method of claim 4, wherein each attribute is associated with a given weight, and the sequence level distance is equal to the weighted sum of the object level distances, each object level distance equal to the weighted sum of the attribute level distances.

6. The method of claim 5, wherein the attribute weights are learned by minimizing the error on object level distances for a given pair of matching sequences.

7. The method of claim 1 wherein at least some of the objects are vehicles, and at least one attribute is selected from the group consisting of:

attributes related to vehicle size, attributes related to vehicle color, attributes related to vehicle type, attributes related to vehicle shape, and attributes related to vehicle aspect ratio.

8. The method of claim 1, wherein one of the first or second cameras is configured for license plate recognition.

9. The method of claim 1, wherein n is selected from the group consisting of:

a predetermined value determined according to the degree of similarity of the objects in proximity to the target object, a predetermined value determined according to the frequency of passing objects in proximity to the target object, and a configurable value selected by a user.

10. The method of claim 1, wherein the objects are vehicles behind and/or in front of the target vehicle in the same lane as the target vehicle, and wherein the value for n is in the range of 3-9.

11. The method of claim 1, wherein each classifier within a set of classifiers is trained independently.

12. A method of identifying in one or more images captured by a second camera a target platoon of n objects corresponding to a reference platoon of n objects generated using images captured by a first camera, the method implemented by a processor and comprising:

storing in a non-transitory computer readable memory operatively coupled to the processor a first set of trained classifiers and a second set of trained classifiers, wherein the 1^st set specifies values corresponding to a first plurality of attributes usable for identifying objects captured by the first camera, and the 2^nd set differs from the first set and specifies values corresponding to a second plurality of attributes usable for identifying objects captured by the 2^nd camera, wherein the first set of trained classifiers and second set of trained classifiers are trained independently, and wherein the first plurality of attributes and second plurality of attributes have at least one attribute in common;

generating, by the processor, a reference group by running the first set of trained classifiers over the reference platoon, the reference group being indicative of values of attributes specified by the 1^st set of trained classifiers and characterizing the objects in the reference platoon;

generating, by the processor, using one or more images captured by the 2^nd camera, a plurality of candidate platoons, each candidate platoon comprising n objects, wherein the one or more images are captured by the 2^nd camera in a time window corresponding to the time of capturing by the 1^st camera the one or more images used for generating the reference platoon;

generating, by the processor, a plurality of candidate groups, each candidate group obtained by running the 2^nd set of trained classifiers over a respective candidate platoon, each candidate group being indicative of values of attributes specified by the 2^nd set of trained classifiers and characterizing the objects in the corresponding candidate platoon;

selecting, by the processor, a candidate platoon corresponding to a candidate group best matching the reference group;

identifying, by the processor, the selected candidate platoon as the target platoon.

13. The method of claim 12, wherein the reference group is a reference sequence, and wherein the candidate groups are candidate sequences.

14. The method of claim 12 further comprising identifying the m^th object in the target platoon as the same object as the m^th object in the reference platoon, wherein m is less than or equal to n.

15. A system for identifying an object in a group of objects appearing in a plurality of cameras comprising:

a first camera;

a second camera;

a memory; and

a processing unit communicatively coupled to the first camera, the second camera, and the memory, the processing unit comprising a processor configured to;

store in the memory a first set of trained classifiers and a second set of trained classifiers, wherein the 1^st set specifies values corresponding to a first plurality of attributes usable for identifying objects captured by the first camera, and the 2^nd set differs from the first set and specifies values corresponding to a second plurality of attributes usable for identifying objects captured by the 2^nd camera, wherein the first set of trained classifiers and second set of trained classifiers are trained independently, and wherein the first plurality of attributes and second plurality of attributes have at least one attribute in common;

generate, using one or more images captured by the 1^st camera, a reference platoon of n objects, the reference platoon comprising a target object and (n−

1) other objects;

generate a reference group by running the first set of trained classifiers over the reference platoon, the reference group being indicative of values of attributes classified by the 1^st set of trained classifiers and characterizing the objects in the reference platoon;

generate, using one or more images captured by the 2^nd camera, a plurality of candidate platoons, each candidate platoon comprising n objects, wherein the one or more images are captured by the 2^nd camera in a time window corresponding to the time of capturing by the 1^st camera the one or more images used for generating the reference platoon;

generate a plurality of candidate groups, each candidate group obtained by running the 2^nd set of trained classifiers over a respective candidate platoon, each candidate group being indicative of values of attributes classified by the 2^nd set of trained classifiers and characterizing the objects in the corresponding candidate platoon;

select a candidate platoon corresponding to a candidate group best matching the reference group;

identify the target object in the selected candidate platoon in accordance with a position of the target object in the reference platoon.

16. The system of claim 15, wherein the reference group is a reference sequence, and wherein the candidate groups are candidate sequences.

17. The system of claim 16, wherein the processor is further configured to associate each candidate sequence with a distance metric indicative of the sequence level distance between the given candidate sequence and the reference sequence, wherein the candidate sequence best matching the reference sequence is the candidate sequence associated with the lowest distance metric.

18. The system of claim 17, wherein the processor is further configured to associate each attribute with a given weight, wherein the sequence level distance is equal to the weighted sum of the object level distances, each object level distance equal to the weighted sum of the attribute level distances, wherein the weights are learned by minimizing the error on object level distances for a given pair of matching sequences.

19. The system of claim 15 wherein at least some of the objects are vehicles, and at least one attribute is selected from the group consisting of:

attributes related to vehicle size, attributes related to vehicle color, attributes related to vehicle type, attributes related to vehicle shape, and attributes related to vehicle aspect ratio.

20. A non-transitory storage medium comprising instructions that when executed by a processor, cause the processor to:

store in a memory a first set of trained classifiers and a second set of trained classifiers, wherein the 1^st set specifies values corresponding to a first plurality of attributes usable for identifying objects captured by a first camera, and the 2^nd set differs from the first set and specifies values corresponding to a second plurality of attributes usable for identifying objects captured by a 2^nd camera, wherein the first set of trained classifiers and second set of trained classifiers are trained independently, and wherein the first plurality of attributes and second plurality of attributes have at least one attribute in common;

generate, using one or more images captured by the 1^st camera, a reference platoon of n objects, the reference platoon comprising a target object and (n−

1) other objects;

generate a reference group by running the first set of trained classifiers over the reference platoon, the reference group being indicative of values of attributes classified by the 1^st set of trained classifiers and characterizing the objects in the reference platoon;

generate, using one or more images captured by the 2^nd camera, a plurality of candidate platoons, each candidate platoon comprising n objects, wherein the one or more images are captured by the 2^nd camera in a time window corresponding to the time of capturing by the 1^st camera the one or more images used for generating the reference platoon;

generate a plurality of candidate groups, each candidate group obtained by running the 2^nd set of trained classifiers over a respective candidate platoon, each candidate group being indicative of values of attributes classified by the 2^nd set of trained classifiers and characterizing the objects in the corresponding candidate platoon;

select a candidate platoon corresponding to a candidate group best matching the reference group;

identify the target object in the selected candidate platoon in accordance with a position of the target object in the reference platoon.

21. The medium of claim 20, wherein the reference group is a reference sequence, and wherein the candidate groups are candidate sequences.