Object identification and material assessment using optical profiles

• US 10,623,716 B2
• Filed: 08/09/2019
• Issued: 04/14/2020
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: 4D Intellectual Properties, LLC
• Original Assignee: 4D Intellectual Properties, LLC
• Inventors: Retterath, James E.
• Primary Examiner(s): Habib, Irfan
• Application Number:
  US16/537,331
• Publication Number:
  US 20190364262A1
• Time in Patent Office: 249 Days
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format
• CPC Class Codes: H04N 23/671 in combination with active ranging signals, e.g. using light or sound signals emitted toward objects

Abstract

An image processing system with a camera system and a processing system analyzes a series of images of a scene to assess an optical property of an object in the scene. A surface of an object that is differentiable from other objects and has a common point that can be identified and analyzed in the series of images captured from different distances of the camera system to the object and different angular orientations of the camera system relative to the object. A set of characteristic values of the image pixels corresponding to the common point is determined that include a point intensity value, a distance from the camera system to the common point, a normal vector for the common point, and an angular orientation between an optical path of the image pixel and the normal vector. The set of characteristics values are used to create an optical profile of the common point that is compared to a set of predefined characteristic profiles to identify the object. In embodiments, the camera system is an active camera, and the wavelength includes near infrared.

Claims

1. An image processing system that analyzes a series of images of a scene to assess an optical property of an object in the scene comprising:

an active camera system configured to capture and store at least two successive frames of data of the scene to be used to generate each image of the series of images, wherein each frame is captured during a capture cycle in one or more frame buffers;

a processing system operably coupled to the active camera system to generate the series of images of the scene, the processing system configured to;

analyze the at least two successive frames of data to determine a minimum intensity value of image pixels due to ambient light in the scene and a maximum intensity value of image pixels corresponding to the scene and generate an image of the series of images based on a difference between the minimum intensity values and the maximum intensity values of the image pixels;

identify at least one object in the series of images that has a surface that is differentiable from other objects such that a common point on the surface of the object can be identified and analyzed in the series of images captured from different distances of the active camera system to the object and different angular orientations of the active camera system relative to the object;

analyze the series of images that include the common point to determine a set of characteristic values of the image pixels corresponding to the common point, including;

a reflected intensity value that is based on the maximum intensity value relative to the minimum intensity value;

a distance from the active camera system to the common point;

a normal vector for the common point; and

an angular orientation between an optical path of the image pixel and the normal vector;

create an optical profile for the common point based on the set of characteristic values of the image pixels corresponding to the common point;

compare the optical profile for the common point to a set of predefined characteristic profiles of optical properties of objects; and

identify the object based on the characteristic profile that best matches the optical profile for at least one common point of the object.

2. The system of claim 1, wherein the object is a surface of a roadway and the library contains characteristic profiles for an asphalt surface, an asphalt surface with surface moisture, an asphalt surface with surface ice, a concrete surface, a concrete surface with surface moisture, and a concrete surface with surface ice.

3. The system of claim 1, wherein the object is a road sign and the library contains characteristic profiles for road signs having different ASTM sheeting types.

4. The system of claim 1, wherein the object is a pavement marker.

5. The system of claim 1, wherein the optical profile is based on images that are selected from one of:

images at different distances with the same angular orientation, images at the same distance with different angular orientations, or images at different distances with different angular orientations.

6. The system of claim 1, wherein the optical profile is used to determine the smoothness rating of an asphalt surface or a concrete surface.

7. The system of claim 1, wherein the active camera includes a light source having a wavelength spectrum that is selected from one of a visible spectrum or a near infra-red spectrum.

8. The system of claim 1, wherein the active camera includes a light source that is selected from one of a laser, a light-emitting-diode, or a vehicle headlamp.

9. An image processing system that analyzes images of a scene to assess an optical property of an object in the scene comprising:

a camera system configured to capture and store a series of images, wherein each image is captured during a capture cycle in one or more frame buffers;

a processing system operably coupled to the camera system to generate a series of images of the scene, the processing system configured to;

identify at least one object in the series of images that has a surface that is differentiable from other objects such that a common point on the surface of the object can be identified and analyzed in the series of images captured from different distances of the camera system to the object and different angular orientations of the camera system relative to the object;

analyze the series of images that include the common point to determine a set of characteristic values of the image pixels corresponding to the common point, including;

a point intensity value;

a distance from the camera system to the common point;

a normal vector for the common point; and

an angular orientation between an optical path of the image pixel and the normal vector;

create an optical profile for the common point based on the set of characteristic values of the image pixels corresponding to the common point;

compare the optical profile for the common point to a set of predefined characteristic profiles of optical properties of objects; and

identify the object based on the characteristic profile that best matches the optical profile for at least one common point of the object.

10. The system of claim 9, wherein the camera system is an active camera.

11. The system of claim 10, wherein the active camera includes a light source having a wavelength spectrum that is selected from one of a visible spectrum or a near infra-red spectrum.

12. The system of claim 10, wherein the active camera includes a light source that is selected from one of a laser, a light-emitting-diode, or a vehicle headlamp.

13. The system of claim 9, wherein the object is a surface of a roadway and the library contains characteristic profiles for an asphalt surface, an asphalt surface with surface moisture, an asphalt surface with surface ice, a concrete surface, a concrete surface with surface moisture, and a concrete surface with surface ice.

14. The system of claim 9, wherein the object is a pavement marker.

15. The system of claim 9, wherein the optical profile is based on images that are selected from one of:

images at different distances with the same angular orientation, images at the same distance with different angular orientations, or images at different distances with different angular orientations.

16. The system of claim 9, wherein the optical profile is used to determine the smoothness rating of an asphalt surface or a concrete surface.

17. A method of using an image processing system to generate a series of images of a scene to assess an optical property of an object in the scene, the method comprising:

using a camera system to;

capture each of the series of images of the scene wherein the camera system is in motion relative to the scene during a capture cycle; and

store each image of the series of images of the scene captured during the capture cycle in one or more frame buffers; and

using a processing system to;

access the series of images stored in the one or more frame buffers;

identify at least one object in the series of images that has a surface that is differentiable from other objects such that a common point on the surface of the object can be identified and analyzed in the series of images captured from different distances of the camera system to the object and different angular orientations of the camera system relative to the object;

analyze the series of images that include the common point to determine a set of characteristic values of the image pixels corresponding to the common point, including;

a point intensity value;

a distance from the camera system to the common point;

a normal vector for the common point; and

an angular orientation between an optical path of the image pixel and the normal vector;

create an optical profile for the common point based on the set of characteristic values of the image pixels corresponding to the common point;

compare the optical profile for the common point to a set of predefined characteristic profiles of optical properties of objects; and

identify the object based on the characteristic profile that best matches the optical profile for at least one common point of the object.

18. The method of claim 17, wherein the camera system is an active camera that includes a light source and each of the images of the scene captured during the capture cycle include light energy emitted from the active camera.

19. The method of claim 18, wherein the light source has a wavelength spectrum that is selected from one of a visible spectrum or a near infra-red spectrum and the light energy emitted from active camera captured during the capture cycle includes light energy from the wavelength spectrum.

20. The method of claim 18, wherein the light source is selected from one of a laser, a light-emitting-diode, or a vehicle headlamp.

21. The method of claim 17, wherein the object is a surface of a roadway and the library contains characteristic profiles for an asphalt surface, an asphalt surface with surface moisture, an asphalt surface with surface ice, a concrete surface, a concrete surface with surface moisture, and a concrete surface with surface ice.

22. The method of claim 17, wherein the object is a pavement marker.

23. The method of claim 17, wherein the optical profile is based on images that are selected from one of:

images at different distances with the same angular orientation, images at the same distance with different angular orientations, or images at different distances with different angular orientations.

24. The method of claim 17, wherein the optical profile is used to determine the smoothness rating of an asphalt surface or a concrete surface.

25. An image processing system that analyzes a series of images of a scene to assess an optical property of an object in the scene comprising:

an active camera system that includes a light source having a wavelength spectrum of at least 400-1000 nm configured to;

capture each of the series of images of the scene wherein the camera system is in motion relative to the scene during a capture cycle; and

store each image of the series of images of the scene captured during the capture cycle in one or more frame buffers

a processing system operably coupled to the active camera system to generate the series of images of the scene, the processing system configured to;

analyze the at least two successive frames of data to determine a minimum intensity value of image pixels due to ambient light in the scene and a maximum intensity value of image pixels corresponding to the scene and generate an image of the series of images based on a difference between the minimum intensity values and the maximum intensity values of the image pixels;

identify at least one object in the series of images that has a surface that is differentiable from other objects such that a common point on the surface of the object can be identified and analyzed in the series of images captured from different distances of the active camera system to the object and different angular orientations of the active camera system relative to the object;

analyze the series of images that include the common point to determine a set of characteristic values of the image pixels corresponding to the common point, including;

a reflected intensity value that is based on the maximum intensity value relative to the minimum intensity value;

a distance from the active camera system to the common point;

a normal vector for the common point; and

an angular orientation between an optical path of the image pixel and the normal vector;

create an optical profile for the common point based on the set of characteristic values of the image pixels corresponding to the common point;

compare the optical profile for the common point to a set of predefined characteristic profiles of optical properties of objects; and

identify the object based on the characteristic profile that best matches the optical profile for at least one common point of the object.

26. The system of claim 25, wherein the object is one of a surface of a roadway, a road sign, and a pavement marker.

27. The system of claim 25, wherein the optical profile is based on images that are selected from one of:

images at different distances with the same angular orientation, images at the same distance with different angular orientations, or images at different distances with different angular orientations.