Imaging device and imaging method

• US 10,365,090 B2
• Filed: 12/18/2017
• Issued: 07/30/2019
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Watanabe, Mikio, Irie, Fuminori
• Primary Examiner(s): Dagnew, Mekonnen D
• Application Number:
  US15/845,233
• Publication Number:
  US 20180106605A1
• Time in Patent Office: 589 Days
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/26 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G01M 5/0008 of bridges
• CPC Class Codes: G01M 5/0091 by using electromagnetic excitation or detection
• CPC Class Codes: G01N 2021/8861 Determining coordinates of flaws
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G01N 21/9515 Objects of complex shape, e.g. examined with use of a surface follower device measuring contours and curvatures G01B11/24
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

In an aspect of the invention, first and second images having parallax are acquired by an imaging unit (202) including a first imaging part (202A) and a second imaging part (202B), and an angle between an imaging direction of the imaging unit (202) and a normal direction of a flat surface of an object to be inspected having the flat surface is calculated on the basis of the acquired first and second images. The pan/tilt of the imaging unit (202) are controlled by a pan/tilt control unit (210), but images for inspection showing the object to be inspected are taken and acquired by the imaging unit in a case in which the calculated angle is controlled to an angle within an allowable angle range.

Claims

1. An imaging device that images an object to be inspected including a flat surface, the imaging device comprising:

an imaging unit that includes a plurality of imaging parts taking a plurality of images having parallax;

a pan/tilt mechanism that rotates the imaging unit in each of a pan direction and a tilt direction;

an angle calculation unit that calculates an angle between an imaging direction of the imaging unit and a normal direction of the flat surface of the object to be inspected on the basis of the plurality of images taken by the imaging unit and showing the object to be inspected;

a pan/tilt control unit that controls the pan/tilt mechanism and sets the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected to an angle within an allowable angle range; and

an imaging control unit that makes at least one imaging part of the imaging unit take images for inspection showing the object to be inspected to acquire the images for inspection in a case in which the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected is an angle within the allowable angle range.

2. The imaging device according to claim 1,

wherein the angle calculation unit includes a corresponding point detection unit that detects three or more corresponding points, which are not positioned on the same straight line, from the plurality of images and a spatial coordinate calculation unit that calculates spatial coordinates of the three or more corresponding points detected by the corresponding point detection unit, and

the angle calculation unit estimates the flat surface of the object to be inspected from the calculated spatial coordinates of the three or more corresponding points, and calculates the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected on the basis of the normal direction of the estimated flat surface.

3. The imaging device according to claim 1,

wherein the allowable angle range is an angle range in which definition of the images for inspection at an end portion of an angle of view most distant from the imaging unit is the minimum definition required for the inspection accuracy of the object to be inspected, in a case in which the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected is the maximum angle within the allowable angle range.

4. The imaging device according to claim 1,

wherein the imaging unit includes an imaging optical system of which an imaging magnification is adjustable, and

the allowable angle range is set in accordance with the imaging magnification of the imaging optical system.

5. The imaging device according to claim 1, further comprising:

a live-view image display unit that displays a live-view image taken by at least one imaging part of the imaging unit;

a pan/tilt operating unit that outputs pan/tilt commands to the pan/tilt mechanism by a manual operation; and

a determination unit that determines whether or not the angle calculated by the angle calculation unit exceeds the allowable angle range,

wherein the pan/tilt control unit controls the pan/tilt mechanism on the basis of the pan/tilt commands output from the pan/tilt operating unit, and stops the pan/tilt mechanism in a case in which the determination unit determines that the angle calculated by the angle calculation unit exceeds the allowable angle range.

6. The imaging device according to claim 1, further comprising:

a live-view image display unit that displays a live-view image taken by at least one imaging part of the imaging unit;

a pan/tilt operating unit that outputs pan/tilt commands to the pan/tilt mechanism by a manual operation;

a determination unit that determines whether or not the angle calculated by the angle calculation unit exceeds the allowable angle range; and

an information unit that issues a warning in a case in which the determination unit determines that the angle calculated by the angle calculation unit exceeds the allowable angle range,

wherein the pan/tilt control unit controls the pan/tilt mechanism on the basis of the pan/tilt commands output from the pan/tilt operating unit.

7. The imaging device according to claim 1,

wherein the pan/tilt control unit automatically controls the pan/tilt mechanism on the basis of the angle calculated by the angle calculation unit to set the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected to 0°

.

8. The imaging device according to claim 1, further comprising:

a moving body on which the imaging device is mounted and which moves the imaging device along the flat surface of the object to be inspected;

a moving body operating unit that outputs a movement command to the moving body by a manual operation; and

a moving body control unit that moves the moving body on the basis of the movement command output from the moving body operating unit.

9. The imaging device according to claim 1, further comprising:

an image recording unit that records the images for inspection acquired by the imaging control unit in association with the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected at the time of taking the images for inspection.

10. The imaging device according to claim 1, further comprising:

a distance calculation unit that calculates a distance between the object to be inspected and the imaging unit in the imaging direction of the imaging unit on the basis of the plurality of images taken by the imaging unit and showing the object to be inspected; and

an image recording unit that records the images for inspection acquired by the imaging control unit in association with the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected at the time of taking the images for inspection and the distance calculated by the distance calculation unit.

11. The imaging device according to claim 10,

wherein the imaging unit includes an imaging optical system of which an imaging magnification is adjustable, and

the image recording unit records the images for inspection in association with the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected at the time of taking the images for inspection, the distance calculated by the distance calculation unit, and the imaging magnification of the imaging optical system.

12. The imaging device according to claim 9,

wherein the angle calculation unit calculates the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected on the basis of the plurality of images that are taken and acquired by the imaging unit at the time of taking the images for inspection and show the object to be inspected.

13. An imaging method of imaging an object to be inspected including a flat surface, the imaging method comprising:

a step of taking a plurality of images having parallax by an imaging unit including a plurality of imaging parts;

a step of calculating an angle between an imaging direction of the imaging unit and a normal direction of the flat surface of the object to be inspected on the basis of the plurality of images, which are taken by the imaging unit and show the object to be inspected, by an angle calculation unit;

a step of controlling a pan/tilt mechanism, which rotates the imaging unit in each of a pan direction and a tilt direction, and setting the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected to an angle within an allowable angle range; and

a step of making at least one imaging part of the imaging unit take images for inspection showing the object to be inspected to acquire the images for inspection in a case in which the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected is an angle within the allowable angle range.

14. The imaging method according to claim 13,

wherein the step of calculating the angle includes a step of detecting three or more corresponding points, which are not positioned on the same straight line, from the plurality of images, and a step of calculating spatial coordinates of the detected three or more corresponding points, and

the flat surface of the object to be inspected is estimated from the calculated spatial coordinates of the three or more corresponding points, and the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected is calculated on the basis of the normal direction of the estimated flat surface.

15. The imaging method according to claim 13,

wherein the allowable angle range is an angle range in which definition of the images for inspection at an end portion of an angle of view most distant from the imaging unit is the minimum definition required for the inspection accuracy of the object to be inspected, in a case in which the angle between the imaging direction of the imaging unit and the normal direction of the flat surface of the object to be inspected is the maximum angle within the allowable angle range.