Hitch angle detection for trailer backup assist system using multiple imaging devices

• US 10,011,228 B2
• Filed: 04/25/2016
• Issued: 07/03/2018
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Hu, Zheng, Lavoie, Erick Michael
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US15/137,589
• Publication Number:
  US 20170174130A1
• Time in Patent Office: 799 Days
• Field of Search: None
• CPC Class Codes: B60C 5/00 Inflatable pneumatic tyres or inner tubes B60C1/00, B60C9/00 - B60C17/00 take precedence
• CPC Class Codes: B60D 1/06 Ball-and-socket hitches , e.g. constructional details, auxiliary devices, their arrangement on the vehicle
• CPC Class Codes: B60D 1/145 consisting of an elongated single bar or tube
• CPC Class Codes: B60D 1/245 for facilitating push back or parking of trailers
• CPC Class Codes: B60D 1/36 for facilitating connection, e.g. hitch catchers , visual guide means, signalling aids B60D1/465 takes precedence; vehicle signalling in general B60Q; optical arrangements specially adapted for viewing trailer-hitches B60R1/003
• CPC Class Codes: B60D 1/62 involving supply lines, electric circuits, or the like
• CPC Class Codes: B60Q 5/00 Arrangement or adaptation of acoustic signal devices
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B62D 13/06 for backing a normally drawn trailer
• CPC Class Codes: B62D 15/024 Other means for determination of steering angle without directly measuring it, e.g. deriving from wheel speeds on different sides of the car
• CPC Class Codes: B62D 15/0285 Parking performed automatically
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A trailer backup assist system is provided herein. A plurality of imaging devices is configured to capture rear-vehicle images. A controller is configured to receive output from each imaging device. The controller processes the output from each imaging device to track a number of trailer features. The controller determines a confidence score for the output of each imaging device. The confidence score is determined based on the number of trailer features being tracked and a tracking quality of each trailer feature. The controller also determines a hitch angle between a vehicle and a trailer based on the output associated with whichever imaging device yielded the highest confidence score.

Claims

1. A trailer backup assist system comprising:

a plurality of imaging devices configured to capture rear-vehicle images; and

a controller configured to;

receive output from each imaging device;

determine a confidence score for the output of each imaging device; and

determine a hitch angle between a vehicle and a trailer based on the output associated with whichever imaging device yielded the highest confidence score;

wherein the plurality of imaging devices comprises a first imaging device coupled to a tailgate of the vehicle, a second imaging device coupled to a cabin of the vehicle, a third imaging device coupled to a side view mirror disposed on one side of the vehicle, and a fourth imaging device coupled to a side view mirror disposed on another side of the vehicle; and

wherein the controller processes the output from the second, third, and fourth imaging devices to identify one or more trailer features when the trailer is located at a maximum controllable hitch angle and a zero hitch angle relative to the vehicle, wherein the maximum controllable hitch angle is defined as the hitch angle at which a full turn in one of a right and a left direction occurs.

2. The trailer backup assist system of claim 1, wherein if one of the second, third, and fourth imaging devices yields the highest confidence score, the controller determines the hitch angle by tracking the identified one or more trailer features in current images relative to the corresponding location of the identified one or more trailer features at the maximum controllable hitch angle and the zero hitch angle.

3. The trailer backup assist system of claim 2, wherein if the one or more identified trailer features being tracked is associated with a trailer position located between the maximum controllable hitch angle and the zero hitch angle, the hitch angle is determined through linear interpolation, and wherein if the one or more identified trailer features being tracked is associated with a trailer position located beyond the maximum controllable hitch angle, the hitch angle is determined through linear extrapolation.

4. The trailer backup assist system of claim 1, wherein the controller is configured to process the output of each imaging device to track a number of trailer features, and wherein the confidence score is determined based on the number of trailer features being tracked and a tracking quality of each trailer feature.

5. The trailer backup assist system of claim 4, wherein the confidence score is determined by the following equation:

S=Σ

_k=1ⁿQ(k)

wherein S is the confidence score, Q is the tracking quality, k is a feature index, and n is a total number of features.

6. The trailer backup assist system of claim 4, wherein the controller determines the hitch angle if the highest confidence score exceeds a predetermined threshold value.

7. A trailer backup assist system comprising:

a plurality of imaging devices configured to capture rear-vehicle images; and

a controller configured to;

receive output from each imaging device;

process the output from each imaging device to track a number of trailer features;

determine a confidence score for the output of each imaging device, wherein the confidence score is determined based on the number of trailer features being tracked and a tracking quality of each trailer feature; and

determine a hitch angle between a vehicle and a trailer based on the output associated with whichever imaging device yielded the highest confidence score;

wherein the plurality of imaging devices comprises a first imaging device coupled to a tailgate of the vehicle, a second imaging device coupled to a cabin of the vehicle, a third imaging device coupled to a side view mirror disposed on one side of the vehicle, and a fourth imaging device coupled to a side view mirror disposed on another side of the vehicle; and

wherein the controller processes the output from the second, third, and fourth imaging devices to identity one or more trailer features when the trailer is located at a maximum controllable hitch angle and a zero hitch angle relative to the vehicle, wherein the maximum controllable hitch angle is defined as the hitch angle at which a full turn in one of a right and a left direction occurs.

8. The trailer backup assist system of claim 7, wherein if one of the second, third, and fourth imaging devices yields the highest confidence score, the controller determines the hitch angle by tracking the identified one or more trailer features in current images relative to the corresponding location of the identified one or more trailer features at the maximum controllable hitch angle and the zero hitch angle.

9. The trailer backup assist system of claim 8, wherein if the one or more identified trailer features being tracked is associated with a trailer position located between the maximum controllable hitch angle and the zero hitch angle, the hitch angle is determined through linear interpolation, and wherein if the one or more identified trailer features being tracked is associated with a trailer position located beyond the maximum controllable hitch angle, the hitch angle is determined through linear extrapolation.

10. The trailer backup assist system of claim 7, wherein the confidence score is determined by the following equation:

S=Σ

_k=1ⁿQ(k)

wherein S is the confidence score, Q is the tracking quality, k is a feature index, and n is a total number of features.

11. A method comprising:

providing a plurality of imaging devices configured to capture rear-vehicle images; and

supplying output from each imaging device to a controller, the controller configured to;

process the output from each imaging device to track a number of trailer features;

determine a confidence score for the output of each imaging device, wherein the confidence score is determined based on the number of trailer features being tracked and a tracking quality of each trailer feature; and

determine a hitch angle between a vehicle and a trailer based on the output associated with whichever imaging device yielded the highest confidence score;

wherein the plurality of imaging devices comprises a first imaging device coupled to a tailgate of the vehicle, a second imaging device coupled to a cabin of the vehicle, a third imaging device coupled to a side view mirror disposed on one side of the vehicle, and a fourth imaging device coupled to a side view mirror disposed on another side of the vehicle; and

wherein the controller processes the output from the second, third, and fourth imaging devices to identify one or more trailer features when the trailer is located at a maximum controllable hitch angle and a zero hitch angle relative to the vehicle, wherein the maximum controllable hitch angle is defined as the hitch angle at which a fill turn in one of a right and a left direction occurs.

12. The method of claim 11, wherein if one of the second, third, and fourth imaging devices yields the highest confidence score, the controller determines the hitch angle by tracking the identified one or more trailer features in current images relative to the corresponding location of the identified one or more trailer features at the maximum controllable hitch angle and the zero hitch angle.

13. The method of claim 12, wherein if the one or more identified trailer features being tracked is associated with a trailer position located between the maximum controllable hitch angle and the zero hitch angle, the hitch angle is determined through linear interpolation, and wherein if the one or more identified trailer features being tracked is associated with a trailer position located beyond the maximum controllable hitch angle, the hitch angle is determined through linear extrapolation.

14. The method of claim 11, wherein the confidence score is determined by the following equation:

S=Σ

_k=1ⁿQ(k)

wherein S is the confidence score, Q is the tracking quality, k is a feature index, and n is a total number of features.