Object classification system for a vehicle

• US 7,702,425 B2
• Filed: 06/07/2004
• Issued: 04/20/2010
• Est. Priority Date: 06/07/2004
• Status: Expired due to Fees

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Hougen, Darrell
• Primary Examiner(s): Tran; Dalena
• Application Number:
  US10/862,486
• Publication Number:
  US 20050273212A1
• Time in Patent Office: 2,143 Days
• Field of Search: 701/45, 701/1, 382/103, 382/276, 382/254, 382/256, 382/260, 280/728.1, 280/735, 280/734
• US Class Current: 701/1
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

A method of classifying objects within a vehicle (12) includes the detection of an object and the generation of an object detection signal in response to the detection. A range signal is generated in response to the object detection signal. An image detection signal is generated having an image representation of the object. The object detection signal is projected on an image plane (122) in response to the image detection signal to generate a fused image reference (120). A search region (144) is generated on the image plane (122) in the vicinity of the fused image reference (120). A vector of feature values is calculated via generation of a sum-array (152) and in response to the range signal and the image detection signal. The search region (144) is classified in response to the vector of feature values.

Claims

1. A method for classifying objects within a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal using at least one image generating sensor comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

classifying each of said at least one search region in response to said at least one vector of feature values;

wherein said at least one vector of feature values is calculated using a plurality of functions;

wherein said plurality of functions comprise a common set of parameters selected from at least one of attributed data, inner window specifications, or outer window specifications;

wherein said attributed data comprises a set of sum-arrays; and

wherein said set of sum-arrays are formed in response to a set of features selected from at least one of intensity data, squared intensity data, x-derivative data, and y-derivative data.

2. A method for classifying objects within a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal using at least one image generating sensor comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

classifying each of said at least one search region in response to said at least one vector of feature values;

wherein said at least one vector of feature values is generated in response to at least one feature definition file.

3. A method for classifying objects within a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal using at least one image generating sensor comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

classifying each of said at least one search region in response to said at least one vector of feature values;

wherein said at least one vector of feature values is generated in response to at least one feature definition file using a single feature evaluation function.

4. A method as in claim 3, wherein the use of said single feature evaluation function comprises the step of comparing two portions of said at least one search region with corresponding values within said at least one feature definition file.

5. A method for classifying objects within a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal using at least one image generating sensor comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal;

classifying each of said at least one search region in response to said at least one vector of feature values;

scoring each of said at least one search region to generate at least one classification score; and

performing at least one countermeasure in response to said at least one classification score.

6. An object classification system for a vehicle, said system comprising:

at least one non-vision object detection sensor for detecting at least one object and generating at least one object detection signal and at least one range signal in response to the detection;

at least one image-generating sensor for generating at least one image detection signal comprising an image representation of said at least one object;

a fusion device for projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

a search region generator for generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

a feature vector calculator for calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

an object classifier for classifying each of said at least one search region in response to said at least one vector of feature values;

wherein said feature vector calculator is operable to generate said at least one vector of feature values in response to at least one feature definition file.

7. An object classification system for a vehicle, said system comprising:

at least one non-vision object detection sensor for detecting at least one object and generating at least one object detection signal and at least one range signal in response to the detection;

at least one image-generating sensor for generating at least one image detection signal comprising an image representation of said at least one object;

a fusion device for projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

a search region generator for generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

a feature vector calculator for calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

an object classifier for classifying each of said at least one search region in response to said at least one vector of feature values;

wherein said feature vector calculator is operable to generate said at least one vector of feature values in response to at least one feature definition file using a single feature evaluation function.

8. A method for operating a collision warning and countermeasure system of a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision object detection sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal;

classifying each of said at least one search region in response to said at least one vector of feature values;

scoring each of said at least one search region to generate at least one classification score; and

performing at least one countermeasure in response to said at least one classification score.

9. A method as in claim 8, said method further comprising the step of incrementing a classification score value upon consistent classification of said at least one object.

10. A method as in claim 8, said method further comprising the step of selectively determining at least one point near a center of said at least one object, which is consistent with a potential threatening object.

11. A method for classifying objects within a vehicle, said method comprising the steps of:

detecting at least one object using at least one non-vision object detection sensor and generating at least one object detection signal in response to the detection;

generating at least one range signal in response to said at least one object detection signal;

generating at least one image detection signal comprising an image representation of said at least one object;

projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

classifying each of said at least one search region in response to said at least one vector of feature values.

12. A method as in claim 11, wherein said at least one vector of feature values is calculated using a plurality of functions.

13. A method as in claim 12, wherein said plurality of functions are formed in response to double-window parameters.

14. A method as in claim 13, wherein said double-window parameters comprise at least one parameter selected from pixel intensity values, pixel contrast values between windows, and edge values.

15. A method as in claim 12, wherein said plurality of functions comprise a common set of parameters selected from at least one of attributed data, inner window specifications, or outer window specifications.

16. A method as in claim 15, wherein said attributed data comprises a set of sum-arrays.

17. An object classification system for a vehicle, said system comprising:

at least one non-vision object detection sensor for detecting at least one object and generating at least one object detection signal and at least one range signal in response to the detection;

at least one image-generating sensor for generating at least one image detection signal comprising an image representation of said at least one object;

a fusion device for projecting said at least one object detection signal on an image plane in response to said at least one image detection signal to generate at least one fused image reference;

a search region generator for generating at least one search region on said image plane in vicinity of each of said at least one fused image reference;

a feature vector calculator for calculating at least one vector of feature values via generation of at least one sum-array and in response to said at least one range signal and said at least one image detection signal; and

an object classifier for classifying each of said at least one search region in response to said at least one vector of feature values.

18. A system as in claim 17, wherein said feature vector calculator is operable to use a plurality of functions in calculating said at least one vector of feature values.

19. A system as in claim 18, wherein said plurality of functions are formed in response to double-window parameters.

20. A system as in claim 18, wherein said plurality of functions comprise a common set of parameters selected from at least one of attributed data, inner window specifications, or outer window specifications.