System and method for trailer alignment

• US 10,744,943 B1
• Filed: 04/08/2019
• Issued: 08/18/2020
• Est. Priority Date: 04/08/2019
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Jales Costa, Bruno Sielly, Nagraj Rao, Nikhil, Nariyambut Murali, Vidya, Niewiadomski, Luke
• Primary Examiner(s): Aziz, Adnan
• Application Number:
  US16/377,388
• Time in Patent Office: 498 Days
• CPC Class Codes: B60D 1/06 Ball-and-socket hitches , e.g. constructional details, auxiliary devices, their arrangement on the vehicle
• CPC Class Codes: B60D 1/065 characterised by the hitch mechanism
• CPC Class Codes: B60D 1/36 for facilitating connection, e.g. hitch catchers , visual guide means, signalling aids B60D1/465 takes precedence; vehicle signalling in general B60Q; optical arrangements specially adapted for viewing trailer-hitches B60R1/003
• CPC Class Codes: B60D 1/363 Hitch guiding or catching elements, e.g. V-shaped plates partially surrounding a coupling member for guiding the other coupling member
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/003 for viewing trailer hitches
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/808 for facilitating docking to a trailer
• CPC Class Codes: B60Y 2300/28 related to towing or towed situations
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A vehicle maneuvering system comprises at least one image device configured to capture image data and a controller. The controller is configured to identify a coupler position of a trailer in the image data and control motion of the vehicle navigating a hitch ball of the vehicle toward the coupler position. The controller is further configured to monitor a coupler distance extending from the coupler position and the hitch ball. In response to the coupler distance being less than or equal to the distance threshold, the controller is configured to classify a plurality of portions of the image data as trailer portions and non-trailer portions and identify the coupler position by processing the image via a feature extraction operation.

Claims

1. A vehicle maneuvering control system, comprising:

at least one image device configured to capture image data; and

a controller configured to;

identify a coupler position of a trailer in the image data;

control motion of the vehicle navigating a hitch of the vehicle toward the coupler position;

monitor a coupler distance extending from the coupler position to the hitch; and

in response to the coupler distance being less than or equal to a distance threshold;

classify a plurality of portions of the image data as trailer portions and non-trailer portions;

identify a trailer heading direction based on the trailer portions;

filter a plurality of the trailer portions of the image data to determine a plurality of nearest trailer portions from the trailer portions, wherein the nearest portions are filtered based on a minimum distance between the trailer portions as detected in the image data and the hitch;

identify the coupler position by processing the image data via a feature extraction operation, wherein the feature extraction operation processes the nearest trailer portions to identify the coupler position; and

in response to the feature extraction operation detecting the coupler position with a predetermined confidence level, control the navigation of the vehicle to the coupler position identified based on the feature extraction operation.

2. The system according to claim 1, wherein the controller is further configured to:

in response to the feature extraction operation failing to achieve the predetermined confidence level, detect the coupler position, based on the trailer heading direction and the trailer portions nearest the vehicle.

3. The system according to claim 1, wherein the coupler position is approximated when beyond the distance threshold by processing the image data via a trailer pattern processing method.

4. The system according to claim 1, wherein the coupler position is detected via the feature extraction operation in the form of a Hough transform in response to the coupler position within the distance threshold.

5. The system according to claim 1, wherein the nearest trailer portions are filtered by a Kalman filter.

6. The system according to claim 1, wherein the non-trailer portions are further classified as ground portions and object portions.

7. The system according to claim 6, wherein the object portions are tracked in relation to a ground plane identified based on the ground portions.

8. The system according to claim 6, wherein the trailer portions are classified as trailer tongue portions, and wherein the controller is further configured to:

receive motion data for the vehicle via at least one motion sensor; and

track the tongue portions relative to the ground plane based on the motion data.

9. The system according to claim 1, wherein the classification of the plurality of portions of the image data is processed by comparing pixel values in image data to identify a plurality of edges corresponding to objects.

10. A method for controlling an alignment between a hitch of a vehicle and a coupler of a trailer comprising:

identifying a coupler position of the coupler in image data;

controlling a motion of the vehicle navigating the hitch of the vehicle toward the coupler position;

monitoring a coupler distance between the coupler position and the hitch to a distance threshold; and

in response to the coupler distance being less than or equal to the distance threshold;

classifying a plurality of pixels of the image data as trailer pixels and non-trailer pixels;

identifying a trailer heading direction based on the trailer pixels;

filtering the trailer pixels of the image data to determine a plurality of nearest trailer portions from the trailer pixels, wherein the nearest trailer portions are filtered based on a minimum distance between the trailer pixels as detected in the image data and the hitch,

identifying the coupler position of the coupler via a feature extraction operation, wherein the feature extraction operation processes the nearest trailer portions to identify the coupled position; and

in response to the feature extraction operation detecting the coupler position at a predetermined confidence level, navigating to the coupler position identified based on the feature extraction operation.

11. The method according to claim 10, wherein in response to the feature extraction operation failing to achieve the predetermined confidence level, detecting the coupler position, based on the trailer heading direction and the trailer pixels nearest the vehicle.

12. The method according to claim 10, wherein the coupler position is first approximated by processing the image data via a trailer pattern processing method.

13. The method according to claim 10, wherein the coupler position is detected via the feature extraction operation in the form of a Hough transform.

14. The method according to claim 10, wherein the nearest trailer portions are filtered by a Kalman filter.

15. A control system for vehicle navigation comprising:

a controller in communication with at least one imaging device configured to capture image data, wherein the controller is configured to;

monitor a coupler distance between a hitch of the vehicle and a coupler of a trailer; and

in response to the coupler distance being less than or equal to a distance threshold;

classify a plurality of portions of the image data as trailer portions and non-trailer portions;

identify a trailer heading direction based on the trailer portions;

process the image data via a Kalman filter identifying a plurality of nearest trailer portions from the trailer portions, wherein the nearest portions are identified based on a nearest distance between the trailer portions as detected in the image data and the hitch;

identify the coupler position via a Hough transform, wherein the Hough transition processes the nearest trailer portions to identify the coupler position;

in response to detecting the coupler position via the Hough transform at a predetermined confidence level, control a navigation to the coupler position identified via the Hough transform; and

in response to the Hough transform failing to achieve the predetermined confidence level, detect the coupler position, based on the heading direction and the trailer portions nearest the vehicle.

16. The system according to claim 15, wherein the coupler distance is first approximated by processing the image data via a trailer pattern processing method when the vehicle is outside the distance threshold.