Adaptive template object classification system with a template generator

• US 7,486,802 B2
• Filed: 06/07/2004
• Issued: 02/03/2009
• Est. Priority Date: 06/07/2004
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Hougen, Darrell
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US10/862,487
• Publication Number:
  US 20050271254A1
• Time in Patent Office: 1,702 Days
• Field of Search: 382/104, 382/103, 382/105, 382/209, 382217-225, 348113-120, 348169-172, 701/1, 701/28, 701/200, 701/207, 701/223, 701/301
• US Class Current: 382/104
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method of performing object classification within a collision warning and countermeasure system (10) of a vehicle (12) includes the generation of an object detection signal in response to the detection of an object (52). An image detection signal is generated and includes an image representation of the object (52). The object detection signal is projected on an image plane (51) in response to the image detection signal to generate a fused image reference. A candidate template (55) is generated in response to the fused image reference. The candidate template (55) is validated. The object (52) is classified in response to the candidate template (55).

Claims

1. A method of performing object classification in a collision warning and countermeasure system of a vehicle, said method comprising the steps of:

detecting at least one object and generating at least one object detection signal in response to the detection;

generating at least one image detection signal including an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to thereby generate at least one fused image reference;

comparing at least one said fused image reference to at least one validated object template that has been selected from a stored plurality of validated object templates;

if at least one said validated object template is determined to sufficiently match at least one said fused image reference, classifying at least one said object according to the at least one matching validated object template; and

if no said validated object template is determined to sufficiently match at least one said fused image reference, generating at least one candidate object template according to at least one said fused image reference, validating at least one said candidate object template, and classifying at least one said object according to at least one validated candidate object template.

2. A method as in claim 1, said method comprising the steps of:

comparing said at least one fused image reference with said plurality of validated object templates to thereby select an approximately matching object template;

determining a match level between at least one said fused image reference and said matching object template; and

adjusting a score level of said matching object template in response to at least one said fused image reference.

3. A method as in claim 1, said method comprising the steps of:

detecting a first object and generating a first object detection signal;

generating a first candidate object template in response to said first object detection signal;

detecting a second object and generating a second object detection signal; and

classifying said second object in response to said first candidate object template.

4. A method as in claim 1, said method further comprising the step of refining an image location of at least one said candidate object template.

5. A method as in claim 4, wherein refining said image location comprises performing double window operations.

7. A method as in claim 1, said method further comprising the step of scoring at least one said candidate object template, and said scoring comprising at least one of:

determining a frequency occurrence level;

comparing pixel intensity values;

comparing pixel contrasts between windows; and

comparing a candidate object template with a validated object template.

6. A method of performing object classification in a collision warning and countermeasure system of a vehicle, said method comprising the steps of:

detecting at least one object and generating at least one object detection signal in response to the detection;

generating at least one image detection signal comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one candidate template in response to said at least one fused image reference;

validating said at least one candidate template;

classifying at least one of said at least one object in response to said at least one candidate template; and

refining an image location of said at least one candidate template;

wherein refining said image location comprises performing double window operations; and

wherein performing said double window operations comprises;

generating an inner window corresponding to the size and shape of said at least one object;

generating an outer window larger than and surrounding said inner window;

evaluating pixel intensities within said inner window to generate an inner window pixel value;

evaluating pixel intensities within said outer window to generate an outer window pixel value;

comparing said inner window pixel value with said outer window pixel value; and

adjusting the image location of said at least one candidate template in response to the comparison.

8. An object classification system for a collision warning and countermeasure system of a vehicle, said object classification system comprising:

at least one non-vision object detection sensor operable to detect at least one object and generate at least one object detection signal in response to the detection;

at least one image-generating sensor operable to generate at least one image detection signal including an image representation of said at least one object;

a fusion device operable to project said at least one object detection signal on an image plane in response to said at least one image detection signal to thereby generate at least one fused image reference;

a template storage device operable to store a plurality of validated object templates;

a comparator operable to compare at least one said fused image reference to at least one validated object template that has been selected from said plurality of validated object templates;

a template generator operable to generate at least one candidate object template according to at least one said fused image reference if no said validated object template is determined to sufficiently match at least one said fused image reference;

a template validating device operable to validate at least one said candidate object template; and

a controller operable to classify at least one said object according to at least one matching validated object template or at least one validated candidate object template.

9. An object classification system as in claim 8, wherein said fusion device is operable to project said at least one object detection signal according to a registration model.

10. An object classification system as in claim 9, wherein said registration model includes relative positions of said at least one non-vision object detection sensor, said at least one image-generating sensor, and ground.

11. An object classification system as in claim 9, wherein said template storage device is operable to also store a plurality of validated candidate object templates.

12. An object classification system as in claim 11, wherein said plurality of validated object templates are stored with an associated frequency occurrence level.

13. An object classification system as in claim 11, wherein said controller is operable to periodically remove a validated object template from said template storage device when an associated frequency occurrence level is below a predetermined value.

14. An object classification system as in claim 8, wherein said comparator is operable to compare at least one said fused image reference with said plurality of validated object templates to thereby select an approximately matching object template.

15. An object classification system as in claim 14, wherein said controller is operable to increase the frequency occurrence level of said matching object template when a match level is greater than a predetermined level.

16. An object classification system as in claim 8, wherein said system further comprises a candidate template processor operable to orient at least one said candidate object template at an unused fused image location within at least one said fused image reference.

17. An object classification system as in claim 8, wherein said system further comprises a candidate template processor operable to refine an image location of at least one said candidate object template.

18. An object classification system as in claim 17, wherein said candidate template processor is operable to use double window operations in refining an image location.

19. A method of operating a collision warning and countermeasure system of a vehicle, said method comprising the steps of:

detecting at least one object and generating at least one object detection signal in response to the detection;

generating at least one image detection signal including an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal and according to a registration model to thereby generate at least one fused image reference;

comparing at least one said fused image reference to at least one validated object template that has been selected from a stored plurality of validated object templates;

if at least one said validated object template is determined to sufficiently match at least one said fused image reference, classifying at least one said object according to the at least one matching validated object template;

if no said validated object template is determined to sufficiently match at least one said fused image reference, generating at least one candidate object template according to at least one said fused image reference, scoring at least one said candidate object template, validating at least one said candidate object template according to said scoring, and classifying at least one said object according to at least one validated candidate object template; and

performing a countermeasure according to the classification of at least one said object.

20. A method as in claim 19, said method further comprising the steps of:

detecting a first object and generating a first object detection signal;

generating a first candidate object template in response to said first object detection signal;

detecting a second object and generating a second object detection signal; and

classifying said second object in response to said first candidate object template.