SMART TRAILER HITCH CONTROL USING HMI ASSISTED VISUAL SERVOING

• US 20160378118A1
• Filed: 06/23/2015
• Published: 12/29/2016
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zeng, Shuqing, Tong, Wei, Wang, Jinsong, Zhang, Wende
• Granted Patent: US 9,696,723 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60D 1/36 for facilitating connection, e.g. hitch catchers , visual guide means, signalling aids B60D1/465 takes precedence; vehicle signalling in general B60Q; optical arrangements specially adapted for viewing trailer-hitches B60R1/003
• CPC Class Codes: B60D 1/62 involving supply lines, electric circuits, or the like
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/202 displaying a blind spot scene on the vehicle part responsible for the blind spot
• CPC Class Codes: B60R 2300/808 for facilitating docking to a trailer
• CPC Class Codes: B62D 13/00 Steering specially adapted for trailers combined traction and steering hitches B60D
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

A method for autonomously aligning a tow hitch ball on a towing vehicle and a trailer drawbar on a trailer through a human-machine interface (HMI) assisted visual servoing process. The method includes providing rearview images from a rearview camera. The method includes touching the tow ball on a display to register a location of the tow ball in the image and touching the drawbar on the display to register a location of a target where the tow ball will be properly aligned with the drawbar. The method provides a template pattern around the target on the image and autonomously moves the vehicle so that the tow ball moves towards the target. The method predicts a new location of the target as the vehicle moves and identifies the target in new images as the vehicle moves by comparing the previous template pattern with an image patch around the predicted location.

Claims

1. A method for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said method comprising:

providing rearview images from a rearview camera on the vehicle that show the tow ball and the drawbar;

displaying the images on a touch screen display;

touching the tow ball on the display to register a position of the tow ball in the image;

touching the drawbar on the display to register a position of a target where the tow ball will be aligned with the drawbar;

providing, by a controller including a processor and memory, a template pattern around the registered target in the image;

autonomously moving the vehicle, by the controller, so that the tow ball moves towards the target;

predicting, by the controller, a position of the target in new images as the vehicle moves;

estimating, by the controller, a new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target;

determining, by the controller, a distance between the tow ball and the new location of the target; and

stopping the vehicle, by the controller, when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

2. The method according to claim 1 wherein autonomously moving the vehicle includes autonomously providing vehicle brake, throttle and steering commands.

3. The method according to claim 1 wherein autonomously moving the vehicle includes recalculating vehicle speed and steering based on the new location of the target.

4. The method according to claim 1 wherein autonomously moving the vehicle includes autonomously steering the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

5. The method according to claim 4 wherein converting image coordinates of the position of the tow ball and the target to world coordinates includes using a homography matrix.

6. The method according to claim 1 wherein predicting a position of the vehicle includes using vehicle speed, vehicle steering angle and hitch trajectory.

7. The method according to claim 1 wherein estimating a new location of the target in the new images includes matching the template pattern to a same pattern in the window.

8. The method according to claim 1 wherein estimating a new location of the target includes employing a Kalman-Bucy filtering process.

9. The method according to claim 8 wherein the Kalman-Bucy filtering process uses a road wheel angle input, a vehicle speed input, a Kalman gain value, and a measurement of the position of the target.

10. The method according to claim 1 further comprising providing image rectification of new images as they are obtained.

11. The method according to claim 1 further comprising initiating the alignment maneuver after the target has been registered by putting the vehicle in a reverse gear.

12. A method for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said method comprising:

providing rearview images from a rearview camera on the vehicle that show the tow ball and the drawbar;

registering a position of the tow ball in the image;

registering a position of a target where the tow ball will be aligned with the drawbar;

autonomously moving the vehicle, by a controller including a processor and memory, so that the tow ball moves towards the target;

determining, by the controller, a distance between the tow ball and a new location of the target; and

stopping the vehicle, by the controller, when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

13. The method according to claim 12 further comprising providing by the controller a template pattern around the registered target in the image, predicting by the controller a position of the target in new images as the vehicle moves, and estimating by the controller the new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target.

14. The method according to claim 13 wherein estimating the new location of the target in the new images includes matching the template pattern to a same pattern in the window.

15. The method according to claim 12 wherein autonomously moving the vehicle includes autonomously steering the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

16. A system for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said system comprising:

a rearview camera on the vehicle for providing rearview images that show the tow ball and the drawbar;

a touch screen display for displaying the images;

means for touching the tow ball on the display to register a position of the tow ball in the image;

means for touching the drawbar on the display to register a position of a target where the tow ball will be aligned with the drawbar; and

a controller including a processor and memory, said controller configured for;

providing a template pattern around the target in the image;

autonomously moving the vehicle so that the tow ball moves towards the target;

predicting a position of the target in new images as the vehicle moves;

estimating a new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target;

determining a distance between the tow ball and the new location of the target; and

stopping the vehicle when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

17. The system according to claim 16 wherein the controller autonomously steers the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

18. The system according to claim 17 wherein converting image coordinates of the position of the tow ball and the target to world coordinates includes using a homography matrix.

19. The system according to claim 16 wherein estimating a new location of the target employs a Kalman-Bucy filtering process.

20. The system according to claim 19 wherein the Kalman-Bucy filtering process uses a road wheel angle input, a vehicle speed input, a Kalman gain value, and a measurement of the position of the target.