Road sign recognition

• US 10,528,055 B2
• Filed: 11/03/2016
• Issued: 01/07/2020
• Est. Priority Date: 11/03/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Kappauf, Collin, Richardson, Andrew
• Primary Examiner(s): Ingram, Thomas
• Application Number:
  US15/342,405
• Publication Number:
  US 20180120857A1
• Time in Patent Office: 1,160 Days
• CPC Class Codes: G01C 21/362 received from an external device or application, e.g. PDA, mobile phone or calendar application
• CPC Class Codes: G01C 21/3811 Point data, e.g. Point of Interest [POI]
• CPC Class Codes: G01C 21/3837 Data obtained from a single source
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G01C 21/3896 Transmission of map data from central databases
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/0236 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

Vehicles can be equipped with computing devices, networks, sensors and controllers to determine, modify, and/or augment a digital map of the surrounding real world. Computing devices included in a vehicle can determine road sign locations in LIDAR data. The computing devices can identify road signs in one or more video images based on the road sign locations and include the road sign in a digital map based on the road sign identity and the road sign location.

Claims

1. A method, comprising:

determining a road sign location in LIDAR data;

identifying a road sign in one or more video images based on the road sign location determined from the LIDAR data, wherein the video images are from a video camera and the LIDAR data are from a LIDAR sensor, wherein identifying the road sign in the video images based on the road sign location includes isolating video images of the road sign from the video images based on the road sign location, and the isolated video images are projected homographically so as to appear as if the video camera acquired the video images of the road sign location at a right angles to the primary face of the road signs; and

including the road sign in a digital map based on the road sign identity and the road sign location.

2. The method of claim 1, further comprising:

determining the road sign location by;

determining locations of isolated high intensity returns in the LIDAR data; and

determining the road sign location by clustering and filtering the locations of isolated high intensity returns in the LIDAR data.

3. The method of claim 2, wherein high intensity returns are created by retroreflective materials.

4. The method of claim 1, wherein identifying the road sign in the isolated video images based on the road sign location includes performing character recognition and sign matching on the isolated video images.

5. The method of claim 1, further comprising:

uploading the road sign to a computing device and including the road sign in a digital map based on the road sign identity and the road sign location.

6. The method of claim 5, further comprising:

downloading the digital map from the computing device and piloting a vehicle based on the digital map.

7. The method of claim 1, further comprising:

determining the road sign location in two or more sets of LIDAR data.

8. The method of claim 1, further comprising:

piloting a vehicle based on the digital map.

9. An apparatus, comprising:

a processor; and

a memory, the memory storing instructions executable by the processor to;

determine a road sign location in LIDAR data;

identify a road sign in one or more video images based on the road sign location determined from the LIDAR data, wherein the video images are from a video camera and the LIDAR data are from a LIDAR sensor, wherein identifying the road sign in the video images based on the road sign location includes isolating video images of the road sign from the video images based on the road sign location, and the isolated video images are projected homographically so as to appear as if the video camera acquired the video images of the road sign location at a right angles to the primary face of the road signs; and

include the road sign in a digital map based on the road sign identity and the road sign location.

10. The apparatus of claim 9, further comprising:

determine the road sign location by;

determining locations of isolated high intensity returns in the LIDAR data; and

determining the road sign location by clustering and filtering the locations of isolated high intensity returns in the LIDAR data.

11. The apparatus of claim 10, wherein the high intensity returns are created by retroreflective materials.

12. The apparatus of claim 9, wherein identify the road sign in the isolated video images based on the road sign location includes performing character recognition and sign matching on the isolated video images.

13. The apparatus of claim 9, further comprising:

upload the road sign to a computing device and include the road sign in a digital map based on the road sign identity and the road sign location.

14. The apparatus of claim 13, further comprising:

download the digital map from the computing device and pilot a vehicle based on the digital map.

15. The apparatus of claim 9, further comprising:

determine the road sign location in two or more sets of LIDAR data.

16. The apparatus of claim 9, further comprising:

pilot a vehicle based on the digital map.