Image acquiring device and image acquiring system

• US 9,544,575 B2
• Filed: 09/12/2012
• Issued: 01/10/2017
• Est. Priority Date: 09/28/2011
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Topcon
• Original Assignee: Kabushiki Kaisha Topcon
• Inventors: Ohtomo, Fumio, Otani, Hitoshi, Momiuchi, Masayuki, Osaragi, Kazuki
• Primary Examiner(s): Hallenbeck-Huber, Jeremaiah C
• Assistant Examiner(s): Hodges, Susan E
• Application Number:
  US13/610,926
• Publication Number:
  US 20130076862A1
• Time in Patent Office: 1,581 Days
• Field of Search: 356/328, 356/326, 356/305, 356/303, 356/456, 348/46, 392/191
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

An image acquiring device comprises a first camera 14 for acquiring video images, consisting of frame images continuous in time series, a second camera 15 being in a known relation with the first camera and used for acquiring two or more optical spectral images of an object to be measured, and an image pickup control device 21, and in the image acquiring device, the image pickup control device is configured to extract two or more feature points from one of the frame images, to sequentially specify the feature points in the frame images continuous in time series, to perform image matching between the frame images regarding the frame images corresponding to the two or more optical spectral images based on the feature points, and to synthesize the two or more optical spectral images according to the condition obtained by the image matching.

Claims

1. An image acquiring device, comprising:

a first camera for acquiring video images, consisting of frame images continuous in time series;

a second camera having an optical axis in common with said first camera and being in a known relation with said first camera and used for synchronizing with said frame images and acquiring two or more optical spectral images of an object to be measured by different wavelengths; and

an image pickup control device, wherein each pixel of image pickup elements of the first camera corresponds to each pixel of image pickup elements of the second camera in a one-to-one relation and said second camera comprises an optical system which has a tele-centric optical system as a wavelength changing means and an interference filter tilted with respect to an optical axis of said second camera, a diaphragm of said tele-centric optical system is provided so as to be movable along a direction perpendicular to said optical axis, and by moving said diaphragm and changing an angle of luminous fluxes entering said interference filter, said second camera acquires two or more optical spectral images in a predetermined wavelength range including two or more different wavelengths, wherein said image pickup control device extracts two or more feature points from first frame images, sequentially specifies said feature points in the frame images continuous in time series, performs image matching between the frame images regarding second frame images corresponding to said two or more optical spectral images based on said feature points, performs photogrammetry based on said first frame images and said second frame images, synthesizes said two or more optical spectral images with different wavelengths according to the condition obtained by said image matching, and associates distance data of each pixel acquired by photogrammetry with each pixel of synthesized optical spectral images.

2. An image acquiring device according to claim 1, wherein said image pickup control device performs relative positioning of each of the optical spectral images based on a position of said feature point on each frame image.

3. An image acquiring device according to claim 1, wherein said image pickup control device extracts at least five feature points from said first frame images of the video images taken by said first camera, specifies said at least five feature points in said second frame image by image tracking, performs stereo-matching based on said feature points of both images, and prepares a three-dimensional model of said object to be measured.

4. An image acquiring device according to claim 3, wherein said image pickup control device combines information of wavelength components of said synthesized optical spectral images to said three-dimensional model.

5. An image acquiring device according to claim 1, wherein the optical spectral images acquired by said second camera are acquired as two-dimensional images relating to a predetermined wavelength range.

6. An image acquiring system, comprising:

a flying object;

a GPS device mounted on said flying object;

a control device for controlling flight of said flying object;

a first camera mounted on said flying object and for acquiring video images made up by frame images continuous to each other in time series;

a second camera having an optical axis in common with said first camera and provided in a known relation with said first camera and for acquiring two or more optical spectral images with different wavelengths of an object to be measured; and

an image pickup control device, wherein each pixel of image pickup elements of the first camera corresponds to each pixel of image pickup elements of the second camera in a one-to-one relation and said flying object moves from a first point to a second point, said GPS device measures a position of the first point and a position of the second point in geocentric coordinate system, said first camera acquires a still image of the first point, acquires video images during the moving from the first point to the second point, and further, acquires still images at the second point, wherein said second camera comprises an optical system which has a tele-centric optical system as a wavelength changing means and an interference filter tilted with respect to an optical axis of said second camera, wherein a diaphragm of said tele-centric optical system is provided so as to be movable along a direction perpendicular to said optical axis, and by moving said diaphragm and changing an angle of luminous fluxes entering said interference filter, said second camera acquires two or more optical spectral image in a predetermined wavelength range including two or more different wavelengths, and wherein said image pickup control device extracts two or more feature points from the still image at the first point, performs video image tracking from the video image during the moving from the first point to the second point, specifies said feature points in the still image at the second point, performs stereo-matching of the still image at the first point with the still image at the second point based on said feature points, prepares a three-dimensional model based on positions of said first point and said second point in the geocentric coordinate system, and said image pickup control device synthesizes two or more optical spectral images with different wavelengths, prepares an optical spectral synthetic image based on the condition of stereo-matching, synthesizes said three-dimensional model with said optical spectral synthetic image, and prepares a four-dimensional model having three-dimensional position data and optical spectral information of said object to be measured.

7. An image acquiring system according to claim 6, wherein said second camera acquires two or more optical spectral images with different wavelengths during a hovering flight at the first point and acquires two or more optical spectral images with different wavelengths during a hovering flight at the second point, said image pickup control device synthesizes two or more optical spectral images of the first point, prepares a first optical spectral synthetic image, synthesizes two or more optical spectral images of the second point, prepares a second optical spectral synthetic image, synthesizes said three-dimensional model with at least one of said first optical spectral synthetic image and said second optical spectral synthetic image, and prepares a four-dimensional model having three-dimensional positional data and optical spectral information of said object to be measured.

8. An image acquiring system according to claim 7, wherein said first camera acquires a same position video image in said hovering flight, said image pickup control device performs synchronization on frame image of said same position video image, acquires an optical spectral image by said second camera, performs image tracking between frame images, performs image matching of two frame images corresponding to two optical spectral images continuous in terms of time, synthesizes said two optical spectral images under the condition obtained by said image matching, sequentially repeats said image matching of said frame images and synthesizing of the optical spectral images, and synthesizes all optical spectral images acquired during said hovering flight.

9. An image acquiring system according to claim 6, wherein said second camera acquires two or more optical spectral images with different wavelengths in a predetermined wavelength range during the moving from the first point to the second point.

10. An image acquiring system according to claim 9, wherein said image pickup control device acquires an optical spectral image which synchronizes with the frame image of said video image by said second camera, performs image tracking between the frame images, performs image matching of two frame images corresponding to two optical spectral images continuous in terms of time, synthesizes said two optical images under the condition as obtained in said image matching, sequentially repeats said image matching of said frame image and synthesizing of the optical spectral image, and synthesizes all optical spectral images acquired during the course of said moving.