Stereo distance measuring apparatus, stereo distance measuring method, and computer readable medium

• US 10,308,374 B2
• Filed: 10/03/2017
• Issued: 06/04/2019
• Est. Priority Date: 10/19/2016
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Subaru Corp.
• Inventors: Onomura, Yoichi, Yamane, Akihiro, Sakaguchi, Akitoshi
• Primary Examiner(s): Hsieh, Ping Y
• Application Number:
  US15/724,165
• Publication Number:
  US 20180105286A1
• Time in Patent Office: 609 Days
• CPC Class Codes: B64D 45/08 optical
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: G01C 11/12 the pictures being supported in the same relative position as when they were taken
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 3/155 Control of the image-sensor operation, e.g. image processing within the image-sensor

Abstract

A stereo distance measuring apparatus includes a first optical camera and a second optical camera, a position-posture calculator, an image corrector, and a stereo distance measuring unit. The first and second optical cameras are provided in an elastic structure part of an aircraft and separated from each other. The first and second optical cameras constitute a stereo camera. The position-posture calculator calculates, on the basis of a displacement amount of the elastic structure part, first and second position-posture information regarding positions and postures of the first and second optical cameras, respectively. The image corrector generates first and second corrected images by correcting, on the basis of the first and second position-posture information, positions and orientations of images taken by the first and second optical cameras, respectively. The stereo distance measuring unit calculates a distance to a photographing target, on the basis of the first and second corrected images.

Claims

1. A stereo distance measuring apparatus comprising:

a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft, the first optical camera and the second optical camera constituting a stereo camera;

a position-posture calculator configured to calculate

     1) first position-posture information regarding a position and a posture of the first optical camera based on a first displacement amount of the first elastic structure part of the first wing and

     2) second position-posture information regarding a position and a posture of the second optical camera based on a second displacement amount of the second elastic structure part of the second wing;

an image corrector configured to

     1) generate a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera, and

     2) generate a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera; and

a stereo distance measuring unit configured to calculate a distance to a photographing target, on a basis of the first corrected image and the second corrected image, wherein

the position-posture calculator includes;

a distance measuring unit that is provided in a highly-rigid part of the aircraft, and configured to measure a distance and a direction from the highly-rigid part to the first optical camera and a distance and a direction from the highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the first elastic structure part and the second elastic structure part;

a position calculator configured to calculate a first position vector to the first optical camera on a basis of the distance and the direction from the highly-rigid part to the first optical camera, and calculate a second position vector to the second optical camera on a basis of the distance and the direction from the highly-rigid part to the second optical camera;

a displacement amount calculator configured to calculate, on a basis of the first position vector, the first displacement amount of the first elastic structure part at which the first optical camera is provided, and calculate, on a basis of the second position vector, the second displacement amount of the second elastic structure part at which the second optical camera is provided; and

a posture calculator configured to calculate a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.

2. The stereo distance measuring apparatus according to claim 1, wherein

the first optical camera and the second optical camera are separately provided at respective tips of the first and second wings of the aircraft, and

the aircraft has a body provided with the distance measuring unit.

3. The stereo distance measuring apparatus according to claim 2, further comprising

a storage that stores in advance one of a database and a relational expression that express a relationship of the first displacement amount of the first elastic structure part at a first position at which the first optical camera is disposed versus an angle of the first position, and a relationship of the second displacement amount of the second elastic structure part at a second position at which the second optical camera is disposed versus an angle of the second position, wherein

the posture calculator is configured to calculate the first posture vector of the first optical camera and the second posture vector of the second optical camera by using one of the database and the relational expression that is stored in the storage.

4. The stereo distance measuring apparatus according to claim 1, further comprising

a storage that stores in advance one of a database and a relational expression that express a relationship of the first displacement amount of the first elastic structure part at a first position at which the first optical camera is disposed versus an angle of the first position, and a relationship of the second displacement amount of the second elastic structure part at a second position at which the second optical camera is disposed versus an angle of the second position, wherein

the posture calculator is configured to calculate the first posture vector of the first optical camera and the second posture vector of the second optical camera by using one of the database and the relational expression that is stored in the storage.

5. A stereo distance measuring apparatus comprising:

a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft, the first optical camera and the second optical camera constituting a stereo camera;

a position-posture calculator configured to calculate

     1) first position-posture information regarding a position and a posture of the first optical camera based on a first displacement amount of the first elastic structure part of the first wing and

     2) second position-posture information regarding a position and a posture of the second optical camera based on a second displacement amount of the second elastic structure part of the second wing;

an image corrector configured to

     1) generate a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera, and

     2) generate a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera; and

a stereo distance measuring unit configured to calculate a distance to a photographing target, on a basis of the first corrected image and the second corrected image, wherein

the position-posture calculator includes;

a displacement amount measuring unit configured to measure the first displacement amount of the first elastic structure part at which the first optical camera is provided, and measure the second displacement amount of the second elastic structure part at which the second optical camera is provided; and

a position-posture vector calculator configured to calculate a first position vector and a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second position vector and a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.

6. The stereo distance measuring apparatus according to claim 5, wherein

the first optical camera and the second optical camera are separately provided at respective tips of the first and second wings of the aircraft,

the first displacement amount includes a displacement amount, in a wing length direction, of the tip at which the first optical camera is provided and a displacement amount, in a wing thickness direction, of the tip at which the first optical camera is provided,

the second displacement amount includes a displacement amount, in a wing length direction, of the tip at which the second optical camera is provided and a displacement amount, in a wing thickness direction, of the tip at which the second optical camera is provided, and

the position-posture vector calculator includes:

a position calculator configured to calculate the first position vector on a basis of the displacement amount in the wing length direction and the displacement amount in the wing thickness direction that are included in the first displacement amount, and calculate the second position vector on a basis of the displacement amount in the wing length direction and the displacement amount in the wing thickness direction that are included in the second displacement amount; and

a posture calculator configured to calculate the first posture vector on a basis of the displacement amount in the wing thickness direction included in the first displacement amount, and calculate the second posture vector on a basis of the displacement amount in the wing thickness direction included in the second displacement amount.

7. The stereo distance measuring apparatus according to claim 6, further comprising

a storage that stores in advance one of a database and a relational expression that express a relationship of the first displacement amount of the first elastic structure part at a first position at which the first optical camera is disposed versus an angle of the first position, and a relationship of the second displacement amount of the second elastic structure part at a second position at which the second optical camera is disposed versus an angle of the second position, wherein

the posture calculator is configured to calculate the first posture vector of the first optical camera and the second posture vector of the second optical camera by using one of the database and the relational expression that is stored in the storage.

8. A stereo distance measuring method comprising:

calculating, with a stereo distance measuring apparatus including a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft,

     1) first position-posture information regarding a position and a posture of the first optical camera based on a first displacement amount of the first elastic structure part and

     2) second position-posture information regarding a position and a posture of the second optical camera based on a second displacement amount of the second elastic structure part, the first optical camera and the second optical camera constituting a stereo camera;

generating, with the stereo distance measuring apparatus, a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera, and a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera;

calculating, with the stereo distance measuring apparatus, a distance to a photographing target, on a basis of the first corrected image and the second corrected image;

measuring a distance and a direction from a highly-rigid part to the first optical camera and a distance and a direction from a highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the first elastic structure part and the second elastic structure part;

calculating a first position vector to the first optical camera on a basis of the distance and the direction from the highly-rigid part to the first optical camera, and calculate a second position vector to the second optical camera on a basis of the distance and the direction from the highly-rigid part to the second optical camera;

calculating, on a basis of the first position vector, the first displacement amount of the first elastic structure part at which the first optical camera is provided, and calculate, on a basis of the second position vector, the second displacement amount of the second elastic structure part at which the second optical camera is provided; and

calculating a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.

9. A non-transitory computer readable medium having a stereo distance measuring program causing a computer to implement a method, the method comprising:

calculating, with a stereo distance measuring apparatus including a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft,

     1) first position-posture information regarding a position and a posture of the first optical camera based on a first displacement amount of the first elastic structure part and

     2) second position-posture information regarding a position and a posture of the second optical camera based on a second displacement amount of the second elastic structure part, the first optical camera and the second optical camera constituting a stereo camera;

generating, with the stereo distance measuring apparatus, a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera, and a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera; and

calculating, with the stereo distance measuring apparatus, a distance to a photographing target, on a basis of the first corrected image and the second corrected image;

measuring a distance and a direction from a highly-rigid part to the first optical camera and a distance and a direction from a highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the first elastic structure part and the second elastic structure part;

calculating a first position vector to the first optical camera on a basis of the distance and the direction from the highly-rigid part to the first optical camera, and calculate a second position vector to the second optical camera on a basis of the distance and the direction from the highly-rigid part to the second optical camera;

calculating, on a basis of the first position vector, the first displacement amount of the first elastic structure part at which the first optical camera is provided, and calculate, on a basis of the second position vector, the second displacement amount of the second elastic structure part at which the second optical camera is provided; and

calculating a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.

10. A stereo distance measuring apparatus comprising:

a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft, the first optical camera and the second optical camera constituting a stereo camera;

a distance measuring unit that is provided in a highly-rigid part of the aircraft, and configured to measure a first distance and a first direction from the highly-rigid part to the first optical camera and a second distance and a second direction from the highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the elastic structure part; and

circuitry configured to

calculate, on a basis of the first distance and the first direction from the highly-rigid part to the first optical camera, first position-posture information regarding a position and a posture of the first optical camera, and calculate, on a basis of the second distance and the second direction from the highly-rigid part to the second optical camera, second position-posture information regarding a position and a posture of the second optical camera,

generate a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera, and generate a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera,

calculate a distance to a photographing target, on a basis of the first corrected image and the second corrected image,

measure a distance and a direction from a highly-rigid part to the first optical camera and a distance and a direction from a highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the first elastic structure part and the second elastic structure part,

calculate a first position vector to the first optical camera on a basis of the distance and the direction from the highly-rigid part to the first optical camera, and calculate a second position vector to the second optical camera on a basis of the distance and the direction from the highly-rigid part to the second optical camera,

calculate, on a basis of the first position vector, a first displacement amount of the first elastic structure part at which the first optical camera is provided, and calculate, on a basis of the second position vector, a second displacement amount of the second elastic structure part at which the second optical camera is provided, and

calculate a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.

11. A stereo distance measuring apparatus comprising:

a first optical camera provided in a first elastic structure part of a first wing of an aircraft and a second optical camera provided in a second elastic structure part of a second wing of the aircraft, the first optical camera and the second optical camera constituting a stereo camera;

a displacement amount measuring unit configured to measure a first displacement amount of the first elastic structure part at which the first optical camera is provided, and measure a second displacement amount of the second elastic structure part at which the second optical camera is provided; and

circuitry configured to

calculate, on a basis of the first displacement amount, first position-posture information regarding a position and a posture of the first optical camera,

calculate, on a basis of the second displacement amount, second position-posture information regarding a position and a posture of the second optical camera,

generate a first corrected image by correcting, on a basis of the first position-posture information, a position and an orientation of an image taken by the first optical camera,

generate a second corrected image by correcting, on a basis of the second position-posture information, a position and an orientation of an image taken by the second optical camera,

calculate a distance to a photographing target, on a basis of the first corrected image and the second corrected image,

measure a distance and a direction from a highly-rigid part to the first optical camera and a distance and a direction from a highly-rigid part to the second optical camera, the highly-rigid part having rigidity that is higher than rigidity of the first elastic structure part and the second elastic structure part,

calculate a first position vector to the first optical camera on a basis of the distance and the direction from the highly-rigid part to the first optical camera, and calculate a second position vector to the second optical camera on a basis of the distance and the direction from the highly-rigid part to the second optical camera,

calculate, on a basis of the first position vector, the first displacement amount of the first elastic structure part at which the first optical camera is provided, and calculate, on a basis of the second position vector, the second displacement amount of the second elastic structure part at which the second optical camera is provided, and

calculate a first posture vector of the first optical camera on a basis of the first displacement amount, and calculate a second posture vector of the second optical camera on a basis of the second displacement amount,

wherein the first position-posture information includes the first position vector and the first posture vector, and

wherein the second position-posture information includes the second position vector and the second posture vector.