Method and device for localizing a sensor in a vehicle

• US 11,138,753 B2
• Filed: 03/12/2020
• Issued: 10/05/2021
• Est. Priority Date: 04/04/2019
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG
• Original Assignee: Aptiv Technologies Limited (BorgWarner Incorporated)
• Inventors: Barth, Alexander, Schiebener, David, Lasley, Andrew J., Wilke, Detlef
• Primary Examiner(s): Philippe, Gims S
• Application Number:
  US16/816,705
• Publication Number:
  US 20200320730A1
• Time in Patent Office: 572 Days
• Field of Search: 348 42
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/001 Vehicle control means, e.g. steering-wheel or column
• CPC Class Codes: B60R 2011/0015 Back-rests
• CPC Class Codes: B60R 2011/0017 Head-rests
• CPC Class Codes: B60R 2011/0022 Pillars
• CPC Class Codes: B60R 2011/0028 Ceiling, e.g. roof rails
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method for localizing a sensor in a vehicle includes using a first sensor mounted on a vehicle to capture at least one image of a moveable element of the vehicle, the moveable element having a predetermined spatial relationship to a second sensor mounted on the vehicle, the moveable element being moveable relative to the first sensor; determining spatial information on the moveable element on the basis of the at least one image; and localizing the second sensor on the basis of the spatial information by a transformation rule representing the predetermined spatial relationship between the moveable element and the second sensor.

Claims

1. A method for localizing a sensor in a vehicle, the method comprising:

capturing at least one image of a moveable element of the vehicle by a first sensor that is mounted on the vehicle,

the moveable element having a predetermined spatial relationship to a second sensor that is mounted on the vehicle, the moveable element being moveable relative to the first sensor;

determining spatial information on the moveable element on the basis of the at least one image; and

localizing the second sensor on the basis of the spatial information by a transformation rule representing the predetermined spatial relationship between the moveable element and the second sensor.

2. The method of claim 1, comprising matching a geometrical model of the moveable element to the at least one image by using a matching algorithm, wherein the spatial information is determined on the basis of the matched model.

3. The method of claim 2, wherein the matching algorithm comprises at least one of a particle algorithm, a grid-search algorithm and a regression algorithm.

4. The method of claim 2, wherein the matching comprises:

generating a plurality of sample points for the geometrical model, each of the sample points having a spatial position;

determining, for at least some of the sample points, a plurality of sample pixels of the at least one image, each of the pixels having a respective pixel value;

computing a rating function on the basis of the sample pixels; and

determining the matched model on the basis of the rating function.

5. The method according to claim 4, wherein a first group of the sample points are located on the model, and a second group of the sample points are located outside the model, wherein difference pixel values are computed for sample points between the first group and the second group, and wherein the rating function is computed on the basis of the difference values.

6. The method according to claim 1, comprising processing at least a portion of the at least one image by a neural network for obtaining an output,

wherein the spatial information is determined on the basis of the output.

7. The method according to claim 1, wherein the at least one image comprises three-dimensional image data.

8. The method according to claim 1, wherein the spatial information comprises at least one of a position of the moveable element and an orientation of the moveable element.

9. The method according to claim 1, wherein the moveable element comprises at least a portion of one of the following:

a steering element of the vehicle,

a housing of the second sensor,

a steering element column of the vehicle, or

a headrest of a vehicle seat.

10. The method according to claim 1, wherein the first sensor is mounted on one of the following portions of the vehicle:

an inner side of a roof,

a pillar of a vehicle body,

a headrest of a seat, or

a seat backrest.

11. The method according to claim 1, wherein the second sensor is mounted on a steering wheel column of the vehicle, the steering wheel column being moveable relative to the first sensor and comprising the moveable element,

wherein the second sensor is configured to take images of a vehicle seat in which at least a portion of a passenger'"'"'s body is expected to be located when the passenger is sitting on the vehicle seat.

12. The method according to claim 1, wherein the moveable element comprises a reflective surface having reflective properties being adapted to the first sensor.

13. The method according to claim 12, wherein the reflective properties are limited to a range of wavelengths invisible to human eyes.

14. A non-transitory computer readable medium comprising instructions, which when executed by a processing unit, cause the processing unit to carry out the method of claim 1.

15. A device for localizing a sensor in a vehicle, the device comprising:

an input for receiving at least one image captured by a first sensor mounted on the vehicle, and

a processing unit configured to determine a location of a second sensor mounted on the vehicle by;

capturing at least one image of a moveable element of the vehicle by the first sensor,

the moveable element having a predetermined spatial relationship to the second sensor, the moveable element being moveable relative to the first sensor;

determining spatial information on the moveable element on the basis of the at least one image; and

localizing the second sensor on the basis of the spatial information by a transformation rule representing the predetermined spatial relationship between the moveable element and the second sensor, wherein the processing unit is further configured to process images captured by the second sensor with respect to the location of the second sensor or to output the location.

16. A system comprising the device of claim 15, a first sensor and a second sensor, wherein at least the first sensor is connected to the device.