Image acquiring device and image acquiring system

• US 10,602,129 B2
• Filed: 04/08/2016
• Issued: 03/24/2020
• Est. Priority Date: 09/28/2011
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Topcon
• Original Assignee: Kabushiki Kaisha Topcon
• Inventors: Ohtomo, Fumio, Otani, Hitoshi, Momiuchi, Masayuki, Osaragi, Kazuki
• Primary Examiner(s): Hodges, Susan E.
• Application Number:
  US15/094,190
• Publication Number:
  US 20160227197A1
• Time in Patent Office: 1,446 Days
• Field of Search: 348 46
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

An image acquiring device comprises a first camera 14 for acquiring video images, consisting of frame images continuous in time series, a second camera 15 being in a known relation with the first camera and used for acquiring two or more optical spectral images of an object to be measured, and an image pickup control device 21, and in the image acquiring device, the image pickup control device is configured to extract two or more feature points from one of the frame images, to sequentially specify the feature points in the frame images continuous in time series, to perform image matching between the frame images regarding the frame images corresponding to the two or more optical spectral images based on the feature points, and to synthesize the two or more optical spectral images according to the condition obtained by the image matching.

Claims

1. An image acquiring device, comprising:

an optical axis;

a half-mirror to divide the optical axis into an image optical axis and a spectral optical axis;

a first camera having a first image pickup element and for acquiring video images, consisting of frame images of real images continuous in time series;

a second camera having a second image pickup element, being in a known relation with said first camera, for changing between set wavelengths, for acquiring an optical spectral image of an object to be measured for each wavelength as each wavelength is changed over and for acquiring two or more optical spectral images of the object to be measured;

an image pickup control device; and

a GPS device,

wherein the image optical axis passes perpendicularly through a center of the first image pickup element, and the spectrum optical axis passes perpendicularly through a center of the second image pickup element, such that a position of all pixels of image pickup elements within the first image pickup element corresponds to a position of all pixels of image pickup elements within the second image pickup element in a one-to-one relation,

and digital photogrammetry is performed based on real images acquired at two points by said first camera and position information determined by said GPS device,

wherein in said image pickup control device, said first camera acquires said frame images synchronizing with an acquisition of said optical spectral images by said second camera,

and wherein said image pickup control device is configured to extract two or more feature points from one of said frame images, sequentially specify said feature points in the frame images continuous in time series, perform image matching between the frame images regarding the frame images corresponding to said two or more optical spectral images based on said feature points, acquire an image matching condition, synthesize said two or more optical spectral images according to said image matching condition, and produce a hyper-spectral image in which each pixel of said first image pickup element or second image pickup element has three-dimensional position information acquired by said digital photogrammetry and optical spectral information, and wherein the real image is a non-filtered digital image.

2. An image acquiring device according to claim 1, wherein said image pickup control device performs relative positioning of each of the optical spectral images based on a position of said feature point on each frame image on said frame images.

3. An image acquiring device according to claim 1, wherein said image pickup control device extracts at least five feature points from a first frame images of the video images taken by said first camera, specifies said at least five feature points in n-th frame image by image tracking, performs stereo-matching based on said feature points of both real images, and prepares a three-dimensional model of said object to be measured.

4. An image acquiring device according to claim 3, wherein said image pickup control device combines information of wavelength components of said synthesized optical spectral images to said three-dimensional model.

5. An image acquiring device according to claim 1, wherein the optical spectral images acquired by said second camera are acquired as two-dimensional images relating to a predetermined wavelength range.

6. An image acquiring device according to claim 1, wherein said optical spectral images are acquired for each of said frame images by said second camera and the image tracking is carried out for said frame images.

7. An image acquiring device according to claim 1, wherein said second camera has an interference filter which a plurality of interference membranes with different wavelength selection characteristics are formed on and which changes over said interference membranes on an optical axis and a diaphragm which changes an incident angle of a light to said interference filter, wherein set wavelengths are sequentially selected by associated operation of said interference filter with said diaphragm.