System and Method for Field Calibrating Video and Lidar Subsystems using Facial Features

• US 20170227628A1
• Filed: 09/21/2016
• Published: 08/10/2017
• Est. Priority Date: 08/15/2012
• Status: Active Grant

• Current Assignee: Aeva, Inc.
• Original Assignee: Digital Signal Corporation (StereoVision Imaging, Inc.)
• Inventors: Zheleznyak, Anatoley T., Roller, Chris, Rosenbush, Gavin, Sebastian, Richard L.
• Granted Patent: US 10,048,359 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4972 Alignment of sensor

Abstract

A system uses range and Doppler velocity measurements from a lidar subsystem and images from a video subsystem to estimate a six degree-of-freedom trajectory of a target. The video subsystem and the lidar subsystem may be aligned with one another by mapping the measurements of various facial features obtained by each of the subsystems to one another.

Claims

1. A system for calibrating a lidar subsystem, the system comprising:

a lidar subsystem configured to;

direct at least two beams toward a target, wherein the at least two beams comprises a first beam directed toward the target via a first scanning mirror and a second beam directed toward the target via a second scanning mirror,

generate a first set of three-dimensional measurements for a plurality of points on the target for the first beam, and

generate a second set of three-dimensional measurements for a plurality of points on the target for the second beam; and

a processor configured to;

receive, from the lidar subsystem, the first set of three-dimensional measurements and the second set of three-dimensional measurements,

compare the first set of three-dimensional measurements with the second set of three-dimensional measurements;

determine one or more offsets in locations of a plurality of facial features between the first set of three-dimensional measurements and the second set of three-dimensional measurements based on the comparison; and

determine one or more calibration adjustments to be made between the first scanning mirror and the second scanning mirror based on the one or more offsets.

2. The system of claim 1, wherein the one or more offsets are determined at a plurality of scanning positions.

3. The system of claim 1, wherein the processor is further configured to:

adjust one or more transformation parameters that transform a coordinate system of the first set of three-dimensional measurements to a coordinate system of the second set of three-dimensional measurements based on the one or more calibration adjustments.

4. A system for calibrating a lidar subsystem and a video subsystem, the system comprising:

a lidar subsystem configured to direct at least two beams toward a target and that generates a set of three-dimensional measurements for a plurality of points on the target for each of the at least two beams;

a video subsystem configured to provide at least one image of the target; and

a processor configured to;

receive, from the lidar subsystem, the set of three-dimensional measurements,

receive, from the video subsystem, the at least one image of the target,

determine locations of a plurality of symmetrical facial features based on the at least one image of the target and a 3D symmetry place, wherein the 3D symmetry plane is determined based on the set of three-dimensional measurements,

determine a noseridge relative to the 3D symmetry plane,

determine a horizontal offset between the locations of the plurality of symmetrical facial features and the 3D symmetry plane,

correct the horizontal offset by a shift between the 3D horizontal offset and the noseridge,

determine a first location of lip line based on the at least one image of the target;

determine a second location of the lip line based on the set of three-dimensional measurements,

determine a vertical offset based on the first location and the second location, and

determine one or more calibration adjustments based on the corrected horizontal offset and the vertical offset.

5. The system of claim 4, wherein the plurality of symmetrical facial features comprises eyes and nostrils.