LIDAR and vision vehicle sensing

• US 10,629,072 B2
• Filed: 10/20/2016
• Issued: 04/21/2020
• Est. Priority Date: 10/20/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Felix, Rodrigo
• Primary Examiner(s): Edwards, Jerrah
• Application Number:
  US15/298,389
• Publication Number:
  US 20180114436A1
• Time in Patent Office: 1,279 Days
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 50/00 Details of control systems for road vehicle drive control not related to the control of a particular sub-unit , e.g. process diagnostic or vehicle driver interfaces
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/015 with provision for distinguishing between two or more types of vehicles, e.g. between motor-cars and cycles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: G08G 1/056 with provision for distinguishing direction of travel

Abstract

A computer is programmed to identify a vehicle in an infrared image, determine a body type of the vehicle based on the infrared image, and predict coordinates of one or more vehicle elements based at least on the determined body type. The computer is further programmed to then perform a plurality of LIDAR sensor sweeps and, based on the LIDAR sensor sweeps, determine whether the vehicle elements are at the predicted coordinates.

Claims

1. A computer, comprising a processor and a memory, the memory storing instructions such that the computer is programmed to:

determine a body type of a first vehicle based on an infrared image received from an infrared sensor;

predict coordinates of one or more first vehicle elements based at least on the determined body type;

actuate a LIDAR sensor to sweep the one or more first vehicle elements; and

based on LIDAR sensor sweeps received from the LIDAR sensor, operate a second vehicle by actuating an actuator of the second vehicle upon determining that the one or more first vehicle elements are at the predicted coordinates.

2. The computer of claim 1, further programmed to determine the body type of the first vehicle based at least in part on a shape of the first vehicle determined from the infrared image, wherein the shape includes at least one of a first vehicle height, a first vehicle length, a first vehicle width, and a distance between the one or more first vehicle elements.

3. The computer of claim 1, wherein the body type of the first vehicle is one of convertible, compact, mid-size sedan, full-size sedan, sport utility, minivan, van, and pickup truck.

4. The computer of claim 1, wherein the one or more first vehicle elements include a front windshield, a front light, a window, a rear light, a pillar, a wheel well, and a wheel.

5. The computer of claim 1, further programmed to:

receive two or more infrared images;

identify a direction of movement of the first vehicle based at least in part on the received two or more infrared images; and

predict coordinates of the one or more first vehicle elements further based on the identified direction of movement.

6. The computer of claim 1, further programmed to:

project a size of the first vehicle based on the infrared image; and

predict coordinates of the one or more first vehicle elements further based on the projected size and a size associated with the determined body type.

7. The computer of claim 1, further programmed to:

predict a size of one of the one or more first vehicle elements; and

based on the LIDAR sensor sweeps, determine whether each one or more first vehicle elements has a size matching its predicted size.

8. The computer of claim 1, further programmed to:

predict a distance between two first vehicle elements based at least in part on the determined body type; and

based on the LIDAR sensor sweeps, further determine whether the two first vehicle elements are at the predicted distance from one another.

9. The computer of claim 1, further programmed to:

predict an orientation of the first vehicle with respect to a second vehicle based on the infrared image; and

predict coordinates of the one or more first vehicle elements of the first vehicle further based on the predicted orientation of the first vehicle with respect to the second vehicle.

10. The computer of claim 1, further programmed to:

determine a sweep area based at least in part on the predicted coordinates of the one or more first vehicle elements of the first vehicle; and

receive LIDAR sweep data from the LIDAR sensor by;

transmitting LIDAR beams to the determined sweep area; and

receiving reflections of the transmitted LIDAR beams.

11. The computer of claim 10, wherein the computer is further programmed to receive reflections of the transmitted LIDAR beams at least by:

receiving a first reflection from a transparent vehicle element of the one or more first vehicle elements; and

receiving a second reflection from an opaque vehicle element of the one or more first vehicle elements.

12. The computer of claim 11, further programmed to determine whether the one or more first vehicle elements are at the predicted coordinates by detecting dimensions of the transparent vehicle element based at least in part on the first reflection and the second reflection.

13. A method, comprising:

determining a body type of a first vehicle based on an infrared image;

predicting coordinates of one or more first vehicle elements based at least on the determined body type;

actuating a LIDAR sensor to sweep the one or more first vehicle elements; and

based on LIDAR sensor sweeps received from the LIDAR sensor, operating a second vehicle by actuating an actuator of the second vehicle upon determining that the one or more first vehicle elements are at the predicted coordinates.

14. The method of claim 13, wherein determining the body type of the first vehicle is based at least in part on a shape of the first vehicle determined from the infrared image.

15. The method of claim 13, further comprising:

determining a sweep area based at least in part on the predicted coordinates of the one or more first vehicle elements of the first vehicle; and

performing the plurality of LIDAR sensor sweeps by at least;

transmitting LIDAR beams to the determined sweep area; and

receiving reflections of the transmitted LIDAR beams.

16. The method of claim 15, wherein the sweep area includes at least the one or more first vehicle elements of the first vehicle.

17. The method of claim 13, further comprising:

receiving two or more infrared images;

identifying a direction of movement of the first vehicle based at least in part on the received two or more infrared images; and

predicting coordinates of the one or more first vehicle elements further based on the identified direction of movement.

18. The method of claim 13, further comprising:

projecting a size of the first vehicle based on the infrared image; and

predicting coordinates of the one or more first vehicle elements further based on the projected size and a size associated with the determined body type.

19. The method of claim 13, further comprising:

predicting a distance between two first vehicle elements based at least in part on the determined body type; and

based on the LIDAR sensor sweeps, further determining whether the two first vehicle elements are at the predicted distance from one another.

20. A computer, programmed to:

identify a vehicle in an infrared image;

determine a body type of the vehicle based on the infrared image;

predict coordinates of one or more vehicle elements based at least on the determined body type;

receive LIDAR sweep data from a LIDAR sensor;

receive reflections of the transmitted LIDAR beams at least by receiving a first reflection from a transparent vehicle element of the one or more vehicle elements, receiving a second reflection from an opaque vehicle element of the one or more first vehicle elements; and

based on the LIDAR sensor sweeps, determine whether the one or more first vehicle elements are at the predicted coordinates by detecting dimensions of the transparent vehicle element based at least in part on the first reflection and the second reflection.