TRAILER ANGLE DETECTION USING REAR BACKUP CAMERA

• US 20190016264A1
• Filed: 07/12/2018
• Published: 01/17/2019
• Est. Priority Date: 07/14/2017
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Potnis, Anuj S., Koravadi, Krishna
• Granted Patent: US 10,532,698 B2
• Time in Patent Office: Days
• CPC Class Codes: B60D 1/245 for facilitating push back or parking of trailers
• CPC Class Codes: B60D 1/62 involving supply lines, electric circuits, or the like
• CPC Class Codes: B60D 1/64 Couplings or joints therefor
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/003 for viewing trailer hitches
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/808 for facilitating docking to a trailer
• CPC Class Codes: B62D 13/06 for backing a normally drawn trailer
• CPC Class Codes: B62D 15/023 by measuring on the king pin
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/75 involving models

Abstract

A method for determining trailer angle includes determining angles of a trailer hitch relative to a data recording device via an angle measuring device while the trailer hitch pivots about a hitch ball of the device. While the trailer hitch pivots about the hitch ball of the device, trailer hitch information is determined responsive to processing of captured image data and responsive to determined angles of the trailer hitch. After determining trailer hitch information, data representative of the determined information is provided to an electronic device, which is disposed at a vehicle having a rear backup camera that has a field of view that encompasses a hitch ball of the vehicle and at least a portion of a hitch of a trailer hitched to the vehicle. Using the provided data, and responsive to processing of captured image data, an angle of the trailer relative to the vehicle is determined.

Claims

1. A method for determining trailer angle, said method comprising:

providing a data recording device;

wherein the data recording device has a camera disposed thereat;

wherein the camera has a field of view that encompasses a hitch ball of the data recording device;

wherein the data recording device comprises a processor operable to process image data captured by the camera;

hitching a trailer hitch at the hitch ball of the data recording device and pivoting the trailer hitch about the hitch ball of the data recording device;

capturing image data via the camera while the trailer hitch pivots about the hitch ball of the data recording device;

determining angles of the trailer hitch relative to the data recording device via an angle measuring device while the trailer hitch pivots about the hitch ball of the data recording device;

while the trailer hitch pivots about the hitch ball of the data recording device, determining trailer hitch information responsive to processing by the processor of image data captured by the camera and responsive to determined angles of the trailer hitch relative to the data recording device as determined by the angle measuring device;

after determining trailer hitch information, providing data representative of the determined trailer hitch information to an electronic device;

equipping a vehicle with the electronic device, wherein the equipped vehicle has a hitch ball at a rear portion of the equipped vehicle, and wherein the equipped vehicle is equipped with a rear backup camera;

hitching a hitch of a trailer to the hitch ball at the rear portion of the equipped vehicle;

wherein the rear backup camera has a field of view that encompasses the hitch ball at the rear portion of the equipped vehicle and at least a portion of the hitch of the trailer hitched to the hitch ball at the rear portion of the equipped vehicle;

capturing image data via the rear backup camera and processing, via a processor at the equipped vehicle, image data captured by the rear backup camera as the equipped vehicle and trailer move along a road; and

using the data representative of the determined trailer hitch information that was provided to the electronic device, and responsive to processing by the processor of image data captured by the rear backup camera, determining an angle of the trailer relative to the equipped vehicle as the equipped vehicle and trailer move along the road.

2. The method of claim 1, wherein determining the angle of the trailer relative to the equipped vehicle comprises use of a neural network.

3. The method of claim 2, wherein the neural network receives image data captured by the rear backup camera and processes the received image data via at least one convolution layer, at least one activation layer and at least one fully connected layer.

4. The method of claim 2, wherein determining the angle of the trailer comprises use of a convolution neural network having a plurality of micro convolution neural networks, and wherein said method comprises detecting a type of the trailer using the micro convolution neural networks.

5. The method of claim 1, wherein the electronic device is part of the rear backup camera.

6. The method of claim 1, comprising (a) pre-processing image data captured by the camera of the data recording device to transform the captured image data to be representative of a viewing angle above the hitch ball of the data recording device, and wherein pre-processed image data is input to the data recording device when determining trailer hitch information, and (b) pre-processing image data captured by the rear backup camera of the equipped vehicle to transform the captured image data to be representative of a viewing angle above the hitch ball of the equipped vehicle, and wherein pre-processed image data is input to the electronic device for processing with the data representative of the determined trailer hitch information that was provided to the electronic device.

7. The method of claim 1, wherein determining trailer hitch information synchronizing determined angles of the trailer hitch relative to the data recording device with the image data captured by the camera.

8. The method of claim 1, wherein determining trailer hitch information comprises correlating image data captured by the camera with determined angles of the trailer hitch as determined by the angle measuring device.

9. The method of claim 1, wherein video images derived from image data captured by the rear backup camera are displayed at a display screen of the equipped vehicle for viewing by the driver during a reversing maneuver of the equipped vehicle.

10. The method of claim 9, wherein image data captured by the rear backup camera is processed to detect objects rearward of the equipped vehicle during the reversing maneuver of the equipped vehicle.

11. The method of claim 1, wherein the rear backup camera comprises part of a multi-camera vision system of the equipped vehicle, and wherein image data captured by the rear backup camera and other exterior viewing cameras of the multi-camera vision system are processed for display of video images surrounding the equipped vehicle during a parking maneuver of the equipped vehicle.

12. The method of claim 1, wherein the trailer hitched at the rear of the equipped vehicle comprises a targetless trailer.

13. The method of claim 1, wherein the angle measuring device comprises an electromechanical device that is disposed at a hitch joint between the trailer hitch and the hitch ball of the data recording device.

14. A method for determining trailer angle, said method comprising:

providing a data recording device;

wherein the data recording device has a camera disposed thereat;

wherein the camera has a field of view that encompasses a hitch ball of the data recording device;

wherein the data recording device comprises a processor operable to process image data captured by the camera;

hitching a trailer hitch at the hitch ball of the data recording device and pivoting the trailer hitch about the hitch ball of the data recording device;

capturing image data via the camera while the trailer hitch pivots about the hitch ball of the data recording device;

determining angles of the trailer hitch relative to the data recording device via an angle measuring device while the trailer hitch pivots about the hitch ball of the data recording device;

while the trailer hitch pivots about the hitch ball of the data recording device, determining trailer hitch information responsive to processing by the processor of image data captured by the camera and responsive to determined angles of the trailer hitch relative to the data recording device as determined by the angle measuring device;

wherein determining trailer hitch information comprises synchronizing determined angles of the trailer hitch relative to the data recording device with the image data captured by the camera;

after determining trailer hitch information, providing data representative of the determined trailer hitch information to an electronic device;

equipping a vehicle with the electronic device, wherein the equipped vehicle has a hitch ball at a rear portion of the equipped vehicle, and wherein the equipped vehicle is equipped with a rear backup camera;

hitching a hitch of a trailer to the hitch ball at the rear portion of the equipped vehicle;

wherein the rear backup camera has a field of view that encompasses the hitch ball at the rear portion of the equipped vehicle and at least a portion of the hitch of the trailer hitched to the hitch ball at the rear portion of the equipped vehicle;

capturing image data via the rear backup camera and processing, via a processor at the equipped vehicle, image data captured by the rear backup camera as the equipped vehicle and trailer move along a road;

using the data representative of the determined trailer hitch information that was provided to the electronic device, and responsive to processing by the processor of image data captured by the rear backup camera, determining an angle of the trailer relative to the equipped vehicle as the equipped vehicle and trailer move along the road;

wherein determining the angle of the trailer relative to the equipped vehicle comprises use of a neural network; and

wherein video images derived from image data captured by the rear backup camera are displayed at a display screen of the equipped vehicle for viewing by the driver during a reversing maneuver of the equipped vehicle.

15. The method of claim 14, wherein the neural network receives image data captured by the rear backup camera and processes the received image data via at least one convolution layer, at least one activation layer and at least one fully connected layer.

16. The method of claim 14, wherein image data captured by the rear backup camera is processed to detect objects rearward of the equipped vehicle during the reversing maneuver of the equipped vehicle.

17. The method of claim 14, wherein the electronic device is part of the rear backup camera.

18. A method for determining trailer angle, said method comprising:

providing a data recording device;

wherein the data recording device has a camera disposed thereat;

wherein the camera has a field of view that encompasses a hitch ball of the data recording device;

wherein the data recording device comprises a processor operable to process image data captured by the camera;

hitching a trailer hitch at the hitch ball of the data recording device and pivoting the trailer hitch about the hitch ball of the data recording device;

capturing image data via the camera while the trailer hitch pivots about the hitch ball of the data recording device;

determining angles of the trailer hitch relative to the data recording device via an angle measuring device while the trailer hitch pivots about the hitch ball of the data recording device;

wherein the angle measuring device comprises an electromechanical device that is disposed at a hitch joint between the trailer hitch and the hitch ball of the data recording device;

while the trailer hitch pivots about the hitch ball of the data recording device, determining trailer hitch information responsive to processing by the processor of image data captured by the camera and responsive to determined angles of the trailer hitch relative to the data recording device as determined by the angle measuring device;

after determining trailer hitch information, providing data representative of the determined trailer hitch information to an electronic device;

equipping a vehicle with the electronic device, wherein the equipped vehicle has a hitch ball at a rear portion of the equipped vehicle, and wherein the equipped vehicle is equipped with a rear backup camera;

hitching a hitch of a trailer to the hitch ball at the rear portion of the equipped vehicle, wherein the trailer hitched to the hitch ball at the rear portion of the equipped vehicle comprises a targetless trailer;

wherein the rear backup camera has a field of view that encompasses the hitch ball at the rear portion of the equipped vehicle and at least a portion of the hitch of the trailer hitched to the hitch ball at the rear portion of the equipped vehicle;

capturing image data via the rear backup camera and processing, via a processor at the equipped vehicle, image data captured by the rear backup camera as the equipped vehicle and trailer move along a road;

using the data provided to the electronic device, and responsive to processing by the processor of image data captured by the rear backup camera, determining an angle of the trailer relative to the equipped vehicle as the equipped vehicle and trailer move along the road; and

wherein the rear backup camera comprises part of a multi-camera vision system of the equipped vehicle, and wherein image data captured by the rear backup camera and other exterior viewing cameras of the multi-camera vision system are processed for display of video images surrounding the equipped vehicle during a parking maneuver of the equipped vehicle.

19. The method of claim 18, wherein determining the angle of the trailer comprises use of a neural network, and wherein the neural network receives image data captured by the rear backup camera and processes the received image data via at least one convolution layer, at least one activation layer and at least one fully connected layer.

20. The method of claim 18, wherein determining trailer hitch information comprises correlating image data captured by the camera with determined angles of the trailer hitch as determined by the angle measuring device.