APPLICATION OF AUXILIARY LIGHTING IN AUTOMATIC HITCH OPERATION
1. A vehicle hitching assistance system, comprising:
- a controller;
acquiring image data from the vehicle;
identifying a trailer within a specified area of the image data and then identifying a coupler of the trailer, the specified area being less than a total field of the image data;
outputting a steering signal to the vehicle to cause the vehicle to steer to align a hitch ball of the vehicle with the coupler.
A vehicle hitching assistance system includes a controller acquiring image data from the vehicle and identifying a trailer within a specified area of the image data and then identifying a coupler of the trailer. The specified area is less than a total field of the image data. The controller then outputs a steering signal to the vehicle to cause the vehicle to steer to align a hitch ball of the vehicle with the coupler.
- 1. A vehicle hitching assistance system, comprising:
a controller; acquiring image data from the vehicle; identifying a trailer within a specified area of the image data and then identifying a coupler of the trailer, the specified area being less than a total field of the image data; outputting a steering signal to the vehicle to cause the vehicle to steer to align a hitch ball of the vehicle with the coupler.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
- 13. A vehicle, comprising:
a steering system; at least one exterior light mounted on and directed away from a rear of the vehicle; and a controller; initiating illumination of the at least one exterior light; acquiring image data from the vehicle; identifying a trailer within the image data; and outputting a steering signal to the vehicle steering system to an align a hitch ball of the vehicle with a coupler of the trailer.
- View Dependent Claims (14, 15, 16, 17, 18, 19)
- 20. A method for assisting a vehicle in hitching with a trailer, comprising:
acquiring image data for a field of view away from a rear of the vehicle; identifying a trailer within a specified area less than the field of view of the image data and then identifying a coupler of the trailer; and outputting a steering signal to cause the vehicle to steer to an align a hitch ball of the vehicle with the coupler.
The present invention generally relates to a vehicle hitch assistance system. In particular, the system provides the user with various options for assisting in hitching a vehicle with a trailer and targets for initial alignment of the vehicle prior to assistance in hitching.
Hitching a trailer to a vehicle can be a difficult and time-consuming experience. In particular, aligning a vehicle hitch ball with the desired trailer hitch can, depending on the initial location of the trailer relative to the vehicle, require repeated forward and reverse driving coordinated with multiple steering maneuvers to appropriately position the vehicle. Further, through a significant portion of the driving needed for appropriate hitch ball alignment, the trailer hitch cannot be seen, and the hitch ball can, under ordinary circumstance, never actually be seen by the driver. This lack of sight lines requires inference of the positioning of the hitch ball and hitch based on experience with a particular vehicle and trailer, and can still require multiple instances of stopping and stepping out of the vehicle to confirm alignment or to note an appropriate correction for a subsequent set of maneuvers. Even further, the closeness of the hitch ball to the rear bumper of the vehicle means that any overshoot can cause a collision of the vehicle with the trailer. Accordingly, further advancements may be desired.
According to one aspect of the disclosure, a vehicle hitching assistance system includes a controller acquiring image data from the vehicle and identifying a trailer within a specified area of the image data and then identifying a coupler of the trailer. The specified area is less than a total field of the image data. The controller then outputs a steering signal to the vehicle to cause the vehicle to steer to align a hitch ball of the vehicle with the coupler.
Embodiments of the first aspect of the disclosure can include any one or a combination of the following features or aspects:
the controller may acquire the image data from an imaging system included with the vehicle, the imaging system having at least one camera, the total field of the image data corresponding with a total field of view of the at least one camera;
the controller may output the steering signal to a steering system included with the vehicle, and the controller may derive the steering signal based on at least a maximum steering angle of the steering system;
the specified area of the image data can be a target area disposed within a central portion of the image data;
the specified area of the image data can be within a designated boundary comprising respective portions based on a resolution of the image data, a proportion of the trailer relative to the total field, and a known steering limit of the vehicle;
the respective portions of the designated boundary may be based on a correlation of the total field of the image data with an area of an assumed ground plane on which the vehicle is positioned visible within the total field;
the area of the assumed ground plane may include a maximum coupler detection distance corresponding with the resolution of the image data, a minimum trailer identification distance corresponding with the proportion of the trailer relative to the total field, and left and right maximum steerable paths extending from the vehicle in a reversing direction corresponding with the known steering limit of the vehicle;
the controller may further output a video image displayable on a human-machine interface within the vehicle including the image data and a graphic overlay of the specified area on the image data in a proportionally correlated manner;
the controller may output the graphic overlay in the video image upon activation of the system;
the controller may receive an input from the human-machine interface corresponding with a user indication of a trailer within the image data and may output the graphic overlay in the video image only after receiving the user indication of the trailer within the image data and failing to identify a trailer within the specified area of the image data;
the controller may further cause the vehicle to illuminate one or more exterior lights directed toward a rear of the vehicle prior to acquiring the image data from the vehicle; and
the controller may identify the trailer within the specified area of the image data and then identify a coupler of the trailer and output a steering signal to the vehicle to cause the vehicle to steer to an align a hitch ball of the vehicle with the coupler as a part of a first hitch assist mode implemented when the controller determines that a sensing condition and a visibility condition are met, further implement a second hitch assistance mode when one of the sensing condition and the visibility condition are not met, receive a selection signal from the vehicle corresponding with a user selection of a mode before implementing either the first or second hitch assistance mode, and cause the vehicle to present an indication that the first hitch assist mode may not be selected when one of the sensing condition and the visibility condition are not met.
According to another aspect of the disclosure, a vehicle includes a steering system, at least one exterior light mounted on and directed away from a rear of the vehicle, and a controller. The controller causes the at least one exterior light to illuminate, acquiring image data from the vehicle, identifies a trailer within the image data, and outputs a steering signal to the vehicle steering system to an align a hitch ball of the vehicle with a coupler of the trailer.
According to another aspect of the disclosure, a method for assisting a vehicle in hitching with a trailer includes acquiring image data for a field of view away from a rear of the vehicle, identifying a trailer within a specified area less than the field of view of the image data and then identifying a coupler of the trailer, and outputting a steering signal to cause the vehicle to steer to an align a hitch ball of the vehicle with the coupler.
These and other aspects, objects, and features of the present invention will be understood and appreciated by those skilled in the art upon studying the following specification, claims, and appended drawings.
In the drawings:
For purposes of description herein, the terms “upper,” “lower,” “right,” “left,” “rear,” “front,” “vertical,” “horizontal,” “interior,” “exterior,” and derivatives thereof shall relate to the device as oriented in
Referring generally to
With respect to the general operation of the hitch assist system 10, as illustrated in the system diagram of
As further shown in
With continued reference to
As also illustrated in
Additionally, the hitch assist system 10 may communicate with human-machine interface (“HMI”) 40 for the vehicle 12. The HMI 40 may include a vehicle display 44, such as a center-stack mounted navigation or entertainment display (
Still referring to the embodiment shown in
System 10 can also incorporate an imaging system 18 that includes one or more exterior cameras, which in the illustrated examples include rear camera 48, center high-mount stop light (CMHSL) camera 50, and side-view cameras 52a and 52b, although other arrangements including additional or alternative cameras are possible. In one example, imaging system 18 can include rear camera 48 alone or can be configured such that system 10 utilizes only rear camera 48 in a vehicle with multiple exterior cameras. In another example, the various cameras 48, 50, 52a, 52b included in imaging system 18 can be positioned to generally overlap in their respective fields of view, which may correspond with rear camera 48, center high-mount stop light (CMHSL) camera 50, and side-view cameras 52a and 52b, respectively. In this manner, image data 55 from two or more of the cameras can be combined in image processing routine 64, or in another dedicated image processor within imaging system 18, into a single image. In an extension of such an example, the image data 55 can be used to derive stereoscopic image data that can be used to reconstruct a three-dimensional scene of the area or areas within overlapped areas of the various fields of view 49, 51, 53a, 53b, including any objects (obstacles or coupler 14, for example) therein. In an embodiment, the use of two images including the same object can be used to determine a location of the object relative to the two image sources, given a known spatial relationship between the image sources. In this respect, the image processing routine 64 can use known programming and/or functionality to identify an object within image data 55 from the various cameras 48, 50, 52a, and 52b within imaging system 18. In either example, the image processing routine 64 can include information related to the positioning of any cameras 48, 50, 52a, and 52b present on vehicle 12 or utilized by system 10, including relative to the center 36 (
The image processing routine 64 can be specifically programmed or otherwise configured to locate coupler 14 within image data 55. In the example of
After the trailer 16 is identified, controller 26 may then identify the coupler 14 of that trailer 16 within the image data 55 based, similarly, on stored or otherwise known visual characteristics of coupler 14 or couplers in general. In another embodiment, a marker in the form of a sticker or the like may be affixed with trailer 16 in a specified position relative to coupler 14 in a manner similar to that which is described in commonly-assigned U.S. Pat. No. 9,102,271, the entire disclosure of which is incorporated by reference herein. In such an embodiment, image processing routine 64 may be programmed with identifying characteristics of the marker for location in image data 55, as well as the positioning of coupler 14 relative to such a marker so that the location 28 of coupler 14 can be determined based on the marker location. Additionally or alternatively, controller 26 may seek confirmation of the determined coupler 14, via a prompt on touchscreen 42 similar to the box 17 used to prompt for confirmation the trailer 16. If the coupler 14 determination is not confirmed, further image processing may be provided, or user-adjustment of the position 28 of coupler 14 may be facilitated, either using touchscreen 42 or another input to allow the user to move the depicted position 28 of coupler 14 on touchscreen 42, which controller 26 uses to adjust the determination of position 28 of coupler 14 with respect to vehicle 12 based on the above-described use of image data 55. In various examples, controller 26 may initially rely on the identification of trailer 16 for the initial stages of an automated hitching operation, with the path 32 being derived to move the hitch ball 34 toward a centrally-aligned position with respect to trailer 16 with the path 32 being refined once the coupler 14 is identified. Such an operational scheme can be implemented when it is determined that trailer 16 is at a far enough distance from vehicle 12 to begin backing without knowing the precise endpoint 35 of path 32 and can be useful when trailer 16 is at a distance where the resolution of the image data 55 makes it possible to accurately identify trailer 16, but at which the coupler 14 cannot be precisely identified. In this manner, initial rearward movement of vehicle 12 can allow for calibration of various system 12 inputs or measurements that can improve the accuracy of distance measurements, for example, that can help make coupler 14 identification more accurate. Similarly, movement of vehicle 12 resulting in a change to the particular image within the data 55 that can improve the resolution or move the coupler 14 relative to the remaining portions of trailer 16 such that it can be more easily identified.
In this manner, the initial determination of the position 28 of trailer 16 to an accepted level of accuracy is needed for execution of the path derivation routine 66 and subsequent automated backing of vehicle 12 along the path 32. Various characteristics or limitations of system 10 may impact the ability of system 10 to identify the trailer 16 (as well as the coupler 14, whenever such identification is carried out) in the data 55 received from imaging system 18 under certain conditions or in certain settings. Still further, various vehicle 12 or other system 10 characteristics may impact the ability of system 10 to navigate to reach a trailer 16 that is, nevertheless, present within the image data 55. Depending on the particular configuration of system 10, such characteristics can be partially driven by the imaging system 18 used by system 10. The imaging system 18 may be limited in its ability to identify a trailer 16 and/or coupler 14 within the entire field of the image data 55. In an example, it may be assumed, at least for simplicity of illustration, that system 10 only uses rear camera 48 for trailer 16 and coupler 14 detection, with rear camera 48 having a field of view 49 that is included in its entirety in the “total field” of the image data 55 (notably, if additional cameras 50, 52a, 52b are used, the total field of the image data 55 would include the entire assembled image from all such utilized cameras). The imaging system 18 limitations may limit system 10 functionality to only a limited distance between trailer coupler 14 and the vehicle 12, as different factors may limit the ability of controller 26 in identifying a trailer 16 or its coupler 14 when the trailer 16 and vehicle 12 are too close together or too far apart. For example, the resolution of the various cameras 48, 50, 52a, 52b in imaging system 18 may impact the ability to identify any trailers 16 or couplers 14 beyond a maximum distance R1 from vehicle 12 with the particular value of R1 being influenced by ambient conditions, including available light and/or weather conditions (e.g., rain or snow).
Additionally, a minimum distance R2 for trailer 16 or coupler 14 detection may be realized because certain implementations of system 10 may rely on dynamic readings (such as of the ground surface behind vehicle 12 or other features visible around coupler 14) to calibrate system 10 and or to track vehicle 12 speed in reversing and to track the position of coupler 14 during system 10 operation. In particular, in the above example where only rear camera 48 is used by system 10, it may be necessary to detect motion within the field of view 49 to identify distance to the coupler 14 and to provide accurate tracking and boundary resolution (an aspect of image processing routine 64). Further, the operating routine 68 may include a longitudinal control algorithm that relies on precise control of the vehicle 12, and a minimum amount of travel distance corresponding with R2 in an example, is required to calibrate certain braking and powertrain variables to achieve such vehicle control. Still further, if a trailer 16 is too close to vehicle 12, various features of the trailer 16 may appear as trailers themselves to the image processing routine 64, meaning that to assist system 10, the trailer 16 should be beyond the minimum distance R2 such that a proportionality of features, including of trailer 16 itself as well as of trailer 16 relative to the total field of image data 55, is optimized for image processing routine 64 functionality.
Additionally, other limitations of system 10 functionality may add constraints to the acceptable zone of operation. In this respect, system 10 may not be capable of maneuvering vehicle 12 towards all locations in an initial view of the rear camera 48 (i.e., during trailer 16 or coupler 14 identification). In particular, system 10 is restricted in its ability to reach a potential target position due, but not limited, to a lateral span that is a function of a distance range and the steering angle δ limitations of vehicle 12. In one aspect, the maximum steering angle δmax of the vehicle 12 determines the lateral range, as a function of distance Dc to coupler 14, as discussed further below. In general, an implementation of system 10 may restrict maneuvering of vehicle 12 to a single reversing motion that, while potentially including steering in both the left and right directions, does not incorporate forward driving of vehicle 12 between successive instances of reverse driving, for example. In this manner, the maximum lateral distance that can be traversed by vehicle 12 in an automated hitching operation is limited by the maximum steering angle δmax. As the vehicle 12 travels laterally by turning the steered wheels 76 and reversing, the lateral limits of system operability 10 are determined as, essentially, a theoretical hitch ball 34 path extending rearward of the vehicle corresponding with steering of vehicle 12 at the maximum steering angle under reversing of vehicle to either side. In this manner, the lateral limits of system 10 may extend outwardly from vehicle 12, with increasing distance away from vehicle 12. In a further aspect, the steering angle δ may be limited to an angle δa that is lower than maximum steering angle δmax based on predetermined constraints for allowable swing of the front end of vehicle 12. In this manner, the lateral limits of system 10 functionality may be further limited.
Because of these limitations, the present system 10 may be configured to only function with trailers 16 and associated couplers 14 positioned inside a “valid” region of space relative to the vehicle 12. The region is determined by the factors listed above, and, potentially, any additional factors that affect the system 10 capability. To ensure such positioning of vehicle 12 relative to trailer 16, system 10 can be generally configured to direct the user to position vehicle 12 relative to trailer 16 such that trailer 16 is within such a valid area of the field of view of the utilized cameras, such as field of view 49 of rear camera 48, and the corresponding image data 55. As shown in
When initiated, system 10 can automatically attempt to identify a trailer 16 within the area of target 45 while prompting the driver to position vehicle 12 such that the coupler 14 and/or trailer 16 is within the area of target 45. When a trailer 16, including its coupler 14, are detected (which would generally coincide with positioning thereof within the area of target 45, system 10 can indicate such an identification, as discussed above, by highlighting the trailer with box 17 (
As shown in
When collected, the position information can then be used in light of the position 28 of coupler 14 within the field of view of the image data 55 to determine or estimate the height Hc of coupler 14. Once the positioning Dc, αc of coupler 14 has been determined and, optionally, confirmed by the user, controller 26 can take control of at least the vehicle steering system 20 to control the movement of vehicle 12 along the desired path 32 to align the vehicle hitch ball 34 with coupler 14, as discussed further below.
Continuing with reference to
in which the wheelbase W is fixed and the steering angle δ can be controlled by controller 26 by communication with steering system 20, as discussed above. In this manner, when the maximum steering angle δmax is known, the smallest possible value for the turning radius ρmin is determined as:
Path derivation routine 66 can be programmed to derive vehicle path 32 to align a known location of the vehicle hitch ball 34 with the estimated position 28 of coupler 14 that takes into account the determined minimum turning radius ρmin to allow path 32 to use the minimum amount of space and maneuvers. In this manner, path derivation routine 66 can use the position of vehicle 12, which can be based on the center 36 of vehicle 12, a location along the rear axle, the location of the dead reckoning device 24, or another known location on the coordinate system 82, to determine both a lateral distance to the coupler 14 and a forward or rearward distance to coupler 14 and derive a path 32 that achieves the needed lateral and forward-backward movement of vehicle 12 within the limitations of steering system 20. The derivation of path 32 further takes into account the positioning of hitch ball 34, based on length L, relative to the tracked location of vehicle 12 (which may correspond with the center 36 of mass of vehicle 12, the location of a GPS receiver, or another specified, known area) to determine the needed positioning of vehicle 12 to align hitch ball 34 with coupler 14. It is noted that hitch assist system 10 can compensate for horizontal movement Δx of coupler 14 in a driving direction away from axle 84 by determining the movement of coupler 14 in the vertical direction Ay that will be needed to receive hitch ball 34 within coupler 14. Such functionality is discussed further in co-pending, commonly-assigned U.S. patent application Ser. No. 14/736,391 and Ser. No. 16/038,462, the entire disclosures of which are hereby incorporated by reference herein.
As discussed above, once the desired path 32, including endpoint 35, has been determined using either of the offset determination schemes discussed above, controller 26 is then allowed to at least control the steering system 20 of vehicle 12 with the powertrain control system 72 and the brake control system 70 (whether controlled by the driver or by controller 26, as discussed below) controlling the velocity (forward or rearward) of vehicle 12. In this manner, controller 26 can receive data regarding the position of vehicle 12 during movement thereof from positioning system 22 while controlling steering system 20, as needed to maintain vehicle 12 along path 32. In particular, the path 32, having been determined based on the vehicle 12 and the geometry of steering system 20, can adjust the steering angle δ, as dictated by path 32, depending on the position of vehicle 12 therealong. It is additionally noted that in an embodiment, the path 32 may comprise a progression of steering angle δ adjustment that is dependent on the tracked vehicle position.
As illustrated in
As discussed above, system 10 requires the availability of a number of measurements obtained using imaging system 18 and, optionally, various sensors 54 and devices 22, as well as reliable control of the steering, 20, powertrain 72 and braking 70 systems to implement the image processing 64, path derivation 66, and operating routines 68 for control the backing of vehicle 12 according to the process described above. Accordingly, the inability of system 10 to obtain any such measurements or to reliably control any of the involved vehicle systems can impact the ability of system 10 to reliably carry out the above hitching process. Accordingly, system 10 may also be configured to provide multiple levels of hitching assistance depending on both user preference, as well as measurement availability and control reliability. For example, a user may not feel comfortable relinquishing control (completely or at all) of vehicle 12, but may still prefer some level of guidance in aligning hitch ball 26 with coupler 14. In other examples, visibility by way of available light or weather may impact the ability of system 10 locate or track coupler 14, even when trailer 16 is within the above-described acceptable zone, or system 10 may determine that for various reasons, the steering 20 and braking 70 systems cannot be reliably controlled. Generally, control of the steering 20 system may be impacted by vehicle 12 being positioned on a transverse slope, which may cause wheel slip to cause vehicle 12 to travel on an unexpected path, and moisture, for example, may cause the brake system 70 functionality. Still further, positioning of vehicle 12 on soft ground or on an upward or downward slope may make powertrain 72 or braking 70 system control operate out of an optimal range. These or other conditions may diminish to the point where reliable control by system 10 is not available.
To address any of the above, or other similar potential, scenarios, system 10 may include additional functionality according to
In an additional level of functionality, when coupler 14 can be identified by system 10, but steering system 20, braking system 70 and powertrain system 72 cannot be controlled with acceptable reliability, system may offer an “ideal” path 100 based on the user of the image processing 64 and path derivation 66 routines that can represent a path 32 determined by system 10 to align hitch ball 34 with coupler 14. When present, the user can adjust the steering input for vehicle 12 such that the hitch ball path 94 aligns with the ideal path 98 during reversing, while also controlling steering and braking until system 10 presents the braking indication 96 or the driver determines that appropriate longitudinal positioning has been achieved. Still further, system 10 may provide automatic steering of vehicle 12 by control of steering system 20 to maintain vehicle 12 along the determined path 32 while the user controls the operation of powertrain 72 and braking 70 system to control the speed of vehicle 12. This functionality can be used, for example, where visibility or ground conditions, for example allow for coupler 14 detection, but not ground tracking, or where powertrain 72 and/or brake system 70 cannot be controlled with the required accuracy, as discussed above. Finally, when the above-described conditions are met, the operability described above, including full control of vehicle 12 can be achieved.
System 10, when providing the various levels of functionality discussed above, can additionally operate according to the scheme 200 depicted in the flow chart of
As shown in
As shown in
In another example, system 10 can operate by the scheme 400 of
As shown in
The additional illumination provided by the illustrated lights 114a, 114b, 116, 118 may facilitate the identification by system 10 of any trailers 16 within the field of view 49 of camera 48, for example, by improving the contrast and/or resolution of the image data 55 available to system 10, particularly in the area where the illumination of such lights overlaps. As shown in
Turning now to
Turning now to
If the coupler 14 can be identified (step 608) in the image data 55, the height Hc distance Dc, and offset angle αc of coupler 14, as identified in step 606, can then be determined using the available image data 55 (step 606) as discussed above, including using image processing routine 64. As discussed above, image processing routine 64 can be programmed or otherwise configured to identify coupler 14 of trailer 16 within image data 55 (step 606). In this manner, after the results of the initial scene scan (step 604) are analyzed, controller 26 can determine if coupler 14 has been confirmed by the user (such as by way of HMI 40). If coupler 14 has not been confirmed or if a determined coupler 14 has been rejected, the scene scan (step 604) can be continued, including while instructing driver to move vehicle 12 to better align with trailer 16, including by positioning the trailer 16 and/or coupler 14 within any of the above-descried targets 45, until coupler 14 is identified.
When coupler 14 has been identified and confirmed, the path derivation routine 66 can be used to determine the vehicle path 32 to align hitch ball 34 with coupler 14 in step 610. In this manner, the positioning Dh, βh of coupler 14 is extracted from the image data 55 and used to place the coupler 14 within the stored data relating the image coordinates with the real-world coordinates of the area surrounding vehicle 12. In doing so, controller 26 uses path derivation routine 66 to determine path 32 to align hitch ball 34 with the predicted position 28 of coupler 14 to an engaging position over hitch ball 34, as described above with respect to
Once the path 32 has been derived, hitch assist system 10 can ask the user U to relinquish control of at least the steering wheel of vehicle 12 (and, optionally, the throttle 73 and brake, in the implementation of hitch assist system 10 described above wherein controller 26 assumes control of powertrain control system 72 and brake control system 70 during execution of operating routine 68). When it has been confirmed that user U is not attempting to control steering system 20 (for example, using torque sensor 80, as discussed above), controller 26 begins to move vehicle 12 along the determined path 32. Hitch assist system 10 then controls steering system 20 (step 612) to maintain vehicle 12 along path 32 as either user U or controller 26 controls the velocity of vehicle 12 using powertrain control system 72 and braking control system 70. As discussed above, controller 26 or the user can control at least steering system 20, while tracking the position Dc, αc of coupler 14 until vehicle 12 reaches endpoint 35 (step 614), wherein the vehicle 12 hitch ball 34 reaches the desired position 38d for the desired alignment with coupler 14, at which point operating routine 68 can end (step 618), either by controlling brake system 70 to cause vehicle 12 to stop (which can be done progressively as vehicle 12 approaches such a point), or by issuing a command to the user to stop vehicle 12 (which can also be done progressively or by a countdown as vehicle 12 approaches the desired location) before deactivating hitch assist system 10. Vehicle 12 can then be driven normally with system 10 remains idle until a reactivation input 620 is received, at which point the above-described method restarts at the scanning step 604.
It is to be understood that variations and modifications can be made on the aforementioned structure without departing from the concepts of the present invention, and further it is to be understood that such concepts are intended to be covered by the following claims unless these claims by their language expressly state otherwise.