Tire-road friction estimation and mapping

• US 10,773,725 B1
• Filed: 08/22/2018
• Issued: 09/15/2020
• Est. Priority Date: 08/25/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ilievski, Filip, Milovich, Matisse J., Miller, Christopher M., Dan, Budhadipta, Graves, Jack E., Cull, Evan C., Perali, Irene, Misra, Mira S., Nabanja, Sheila P., Ion, Lucian, Reuther, James J., Cull, Christy F.
• Primary Examiner(s): Tran, Khoi H
• Assistant Examiner(s): Hannan, B M M
• Application Number:
  US16/108,792
• Time in Patent Office: 755 Days
• Field of Search: 701 80, 701117
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2530/20 Tyre data
• CPC Class Codes: B60W 2552/35 Road bumpiness, e.g. potholes
• CPC Class Codes: B60W 2552/40 Coefficient of friction
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 40/068 Road friction coefficient
• CPC Class Codes: B60W 60/0015 specially adapted for safety
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section

Abstract

A method includes determining temperature values for roadway areas ahead of a vehicle, determining lubricant state values for the roadway areas, and determining lubricant thickness values for the roadway areas. The method also includes determining a tire-road friction estimate for each of the roadway areas using the temperature values, the lubricant state values, and the lubricant thickness values, and defining a friction map that relates the tire-road friction estimates to the roadway areas. The method also includes determining a motion plan based at least in part on the friction map, and controlling the vehicle based on the motion plan.

Claims

1. A method, comprising:

obtaining images using sensors of a vehicle, wherein the images show a roadway that is located ahead of the vehicle;

dividing the images into image sections that correspond to a plurality of roadway areas from the roadway;

determining temperature values for the plurality of roadway areas using respective ones of the image sections from the images obtained by the sensors;

determining lubricant state values for the plurality of roadway areas using respective ones of the image sections from the images obtained by the sensors;

determining lubricant thickness values for the plurality of roadway areas using respective ones of the image sections from the images obtained by the sensors;

determining tire-road friction estimates for the plurality of roadway areas using the temperature values, the lubricant state values, and the lubricant thickness values;

determining a trajectory for the vehicle using the tire-road friction estimates for the plurality of roadway areas, wherein the trajectory traverses at least some roadway areas from the plurality of roadway areas; and

controlling actuator systems of the vehicle such that the vehicle follows the trajectory.

2. The method of claim 1, wherein determining the temperature values for the plurality of roadway areas includes obtaining infrared temperature measurements for the plurality of roadway areas.

3. The method of claim 1, wherein determining the lubricant state values is performed by determining emissivity values for the plurality of roadway areas and comparing the emissivity values for the plurality of roadway areas to known emissivity values.

4. The method of claim 3, wherein determining the emissivity values for the plurality of roadway areas is performed using long wave infrared (LWIR) signals.

5. The method of claim 1, wherein determining the lubricant thickness values for the plurality of roadway areas is performed by differential spectroscopy using near infrared (NIR) signals.

6. The method of claim 1, further comprising:

determining lubricant composition values for the plurality of roadway areas, wherein the lubricant composition values are determined based on reflectance and determining the tire-road friction estimates for the plurality of roadway areas is performed using the lubricant composition values.

7. The method of claim 1, further comprising:

determining surface roughness values for the plurality of roadway areas, wherein the surface roughness values for the plurality of roadway areas are determined using a structured light signal and determining the tire-road friction estimates is performed using the surface roughness values.

8. The method of claim 1, further comprising:

determining a humidity value, wherein determining the tire-road friction estimates is performed using the humidity value.

9. The method of claim 1, wherein determining the trajectory includes:

determining a first candidate trajectory that traverses a first group of roadway areas from the plurality of roadway areas,

determining a second candidate trajectory that traverses a second group of roadway areas from the plurality of roadway areas, and

selecting one of the first candidate trajectory or the second candidate motion trajectory as the trajectory based in part on the tire-road friction estimates for the first group of roadway areas and the tire-road friction estimates for the second group of roadway areas.

10. The method of claim 1, wherein the lubricant state values each correspond to at least one of a liquid state, a semi-solid state, or a solid state.

11. The method of claim 1, wherein determining the trajectory for the vehicle includes determining vehicle handling limits for the plurality of roadway areas based on the tire-road friction estimates for the plurality of roadway areas.

12. The method of claim 1, wherein the emissivity values for the plurality of roadway areas are determined using long wave infrared (LWIR) signals from the images, the lubricant thickness values for the plurality of roadway areas are determined by differential spectroscopy using near infrared (NIR) signals from the images, the lubricant composition values for the plurality of roadway areas are determined using the images based on reflectance.

13. A system, comprising:

a sensor group that is mounted to a vehicle and includes;

one or more near infrared (NIR) light sources that output electromagnetic radiation in at least a first NIR wavelength band and a second NIR wavelength band,

a first NIR camera that is sensitive to electromagnetic radiation in the first NIR wavelength band and outputs a first NIR raster image representing a roadway ahead of the vehicle,

a second NIR camera that is sensitive to electromagnetic radiation in the second NIR wavelength band and outputs a second NIR raster image representing the roadway ahead of the vehicle,

a first long wave infrared (LWIR) camera that is sensitive to electromagnetic radiation in a first LWIR wavelength band and outputs a first LWIR raster image representing the roadway ahead of the vehicle, and

a second LWIR camera that is sensitive to electromagnetic radiation in a second LWIR wavelength band and outputs a second LWIR raster image representing the roadway ahead of the vehicle; and

a friction estimation unit that defines image portions corresponding to a plurality of roadway areas of each of the first NIR raster image, the second NIR raster image, the first LWIR raster image, and the second LWIR raster image, and determines tire-road friction estimates for the plurality of roadway areas using corresponding ones of the image portions.

14. The system of claim 13, wherein the friction estimation unit determines temperature values for the plurality of roadway areas using at least one of the first NIR raster image or the second NIR raster image, the friction estimation unit determines lubricant state values for the plurality of roadway areas using the first LWIR raster image and the second LWIR raster image, the friction estimation unit determines lubricant thickness values for the plurality of roadway areas using the first NIR raster image and the second NIR raster image, and the friction estimation unit determines the tire-road friction estimates the plurality of roadway areas based on the temperature values, the lubricant state values, and the lubricant thickness values.

15. The system of claim 13, wherein the one or more NIR light sources include a single NIR light source that outputs electromagnetic radiation in the first NIR wavelength band and the second NIR wavelength band.

16. The system of claim 13, wherein the one or more NIR light sources include a first NIR light source that outputs electromagnetic radiation only in the first NIR wavelength band and a second NIR light source that outputs electromagnetic radiation only in the second NIR wavelength band.

17. The system of claim 13, further comprising:

an automated control unit that determines a motion plan for the vehicle based on vehicle handling limits for the plurality roadway areas based on tire-road friction estimates for the plurality of roadway areas and controls actuator systems of the vehicle according to the motion plan.

18. A system, comprising:

a sensor group that is mounted to a vehicle and includes;

one or more near infrared (NIR) light sources that output electromagnetic radiation in at least a first NIR wavelength band and a second NIR wavelength band,

a first NIR imaging device that is sensitive to electromagnetic radiation in the first NIR wavelength band and outputs a first NIR raster image representing a roadway ahead of the vehicle, wherein the first NIR imaging device includes a first NIR filter that passes only electromagnetic radiation in the first NIR wavelength band, and the first NIR wavelength band includes a wavelength of 905 nm,

a second NIR imaging device that is sensitive to electromagnetic radiation in the second NIR wavelength band and outputs a second NIR raster image representing the roadway ahead of the vehicle, wherein the second NIR imaging device includes a second NIR filter that passes only electromagnetic radiation in the second NIR wavelength band, and the second NIR wavelength includes a wavelength of 1310 nm,

a first long wave infrared (LWIR) imaging device that is sensitive to electromagnetic radiation in a first LWIR wavelength band and outputs a first LWIR raster image representing the roadway ahead of the vehicle, wherein the first LWIR imaging device includes a first LWIR filter that passes only electromagnetic radiation in the first LWIR wavelength band, and the first LWIR wavelength band includes a wavelength of 8000 nm, and

a second LWIR imaging device that is sensitive to electromagnetic radiation in a second LWIR wavelength band and outputs a second LWIR raster image representing the roadway ahead of the vehicle, wherein the second LWIR imaging device includes a second LWIR filter that passes only electromagnetic radiation in second first LWIR wavelength band, and the second LWIR wavelength band includes a wavelength of 11000 nm; and

a friction estimation unit that defines image portions corresponding to roadway areas of each of the first NIR raster image, the second NIR raster image, the first LWIR raster image, and the second LWIR raster image, and determines tire-road friction estimates for the roadway areas using corresponding ones of the image portions.

19. The system of claim 18, wherein the friction estimation unit determines temperature values for the plurality of roadway areas using at least one of the first NIR raster image or the second NIR raster image, the friction estimation unit determines lubricant state values for the plurality of roadway areas using the first LWIR raster image and the second LWIR raster image, the friction estimation unit determines lubricant thickness values for the plurality of roadway areas using the first NIR raster image and the second NIR raster image, and the friction estimation unit determines the tire-road friction estimates for the plurality of roadway areas based on the temperature values, the lubricant state values, and the lubricant thickness values.

20. A method, comprising:

obtaining images from infrared cameras that are mounted to a vehicle, wherein the images show a plurality of roadway areas that are located ahead of the vehicle;

determining parameter values for the plurality of roadway areas based on the images output by the infrared cameras, wherein the parameter values include temperature values, lubricant state values, and lubricant thickness values;

determining tire-road friction estimates that include an individual tire-road friction estimates for roadway areas from the plurality of roadway areas using the parameter values;

determining a trajectory for the vehicle based on the tire-road friction estimates, wherein the trajectory for the vehicle traverses at least some roadway areas from the plurality of roadway areas; and

controlling actuator systems of the vehicle such that the vehicle follows the trajectory.

21. The method of claim 20, wherein the parameter values for the plurality of roadway areas also include lubricant composition values for the plurality of roadway areas.

22. The method of claim 20, wherein determining the trajectory for the vehicle includes determining vehicle handling limits for the plurality of roadway areas based on the tire-road friction estimates for the plurality of roadway areas.