COMPOUND-EYE IMAGING DEVICE, AND PARALLAX ADJUSTING METHOD AND PROGRAM THEREOF

• US 20120140043A1
• Filed: 09/13/2010
• Published: 06/07/2012
• Est. Priority Date: 03/30/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Mori, Koji
• Granted Patent: US 9,071,759 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators

Abstract

A composite eye camera has: imaging sections (21A, 21B) that generate plural viewpoint images per frame by imaging a same subject from plural viewpoints; a three-dimensional processing section (30) that acquires a parallax amount on the basis of the generated plural viewpoint images; and a CPU (35) that judges that there is an abnormality in the parallax amount in at least one case among a case in which fluctuation of a fixed interval in the acquired parallax amount is greater than a predetermined value, and a case in which the parallax amount reaches a predetermined allowed limit value, and a case in which an object of acquisition of the parallax amount can no longer be detected, and judges that there is no abnormality in the parallax amount in a case that is none of these, and, when it is judged that there is no abnormality in the parallax amount, carries out first parallax adjustment, and, when it is judged that there is an abnormality in the parallax amount, switches to control of a second parallax adjustment, that is different than control of the first parallax adjustment, and carries out parallax adjustment. Accordingly, parallax adjustment is carried out appropriately even when there are problems in the parallax amount itself that is determined from the plural viewpoint images, or when an object of parallax adjustment can no longer be detected.

Claims

1. A composite eye imaging device comprising:

imaging sections that generate a plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

a parallax amount acquiring section that acquires a parallax amount on the basis of the plurality of viewpoint images generated by the imaging sections;

an abnormality judging section that judges that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the parallax amount acquired by the parallax amount acquiring section is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected, and judges that there is no abnormality in the parallax amount in a case that is none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

a parallax adjusting section that, when it is judged by the abnormality judging section that there is no abnormality in the parallax amount, carries out first parallax adjustment, and that, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, switches to control of a second parallax adjustment that is different than control of the first parallax adjustment, and carries out parallax adjustment.

2. The composite eye imaging device of claim 1, wherein, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, the parallax adjusting section carries out parallax adjustment within a range of a parallax amount maximum change amount that is determined in advance.

3. The composite eye imaging device of claim 1 or claim 2, wherein, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, the parallax adjusting section carries out parallax adjustment by using a parallax amount of a previous frame.

4. The composite eye imaging device of claim 1, wherein, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, the parallax adjusting section lowers a parallax adjustment frequency.

5. The composite imaging device of claim 1, further comprising an associating section that associates at least one information of information that is obtained from the plurality of viewpoint images generated by the imaging sections and that is used for acquiring the parallax amount at the parallax amount acquiring section, information that expresses the parallax amount acquired by the parallax amount acquiring section by using the information obtained from the plurality of viewpoint images, or information expressing whether or not a predetermined processing has been carried out by using the parallax amount, with the plurality of viewpoint images that correspond to the at least one information and on which parallax adjustment has been carried out.

6. The composite eye imaging device of claim 5, wherein the associating section associates the at least one information with the plurality of viewpoint images that correspond to the at least one information and on which parallax adjustment has been carried out, and collects the at least one information in one electronic file.

16. The composite eye imaging device of claim 2, wherein, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, the parallax adjusting section lowers a parallax adjustment frequency.

17. The composite eye imaging device of claim 3, wherein, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, the parallax adjusting section lowers a parallax adjustment frequency.

18. The composite imaging device of claim 2, further comprising an associating section that associates at least one information of information that is obtained from the plurality of viewpoint images generated by the imaging sections and that is used for acquiring the parallax amount at the parallax amount acquiring section, information that expresses the parallax amount acquired by the parallax amount acquiring section by using the information obtained from the plurality of viewpoint images, or information expressing whether or not a predetermined processing has been carried out by using the parallax amount, with the plurality of viewpoint images that correspond to the at least one information and on which parallax adjustment has been carried out.

7. A composite eye imaging device comprising:

imaging sections that generate a plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

a parallax amount acquiring section that acquires a parallax amount of a predetermined object on the basis of the plurality of viewpoint images generated by the imaging sections;

an abnormality judging section that judges that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the parallax amount acquired by the parallax amount acquiring section is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected, and judges that there is no abnormality in the parallax amount in a case that is none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

a parallax adjusting section that, when it is judged by the abnormality judging section that there is no abnormality in the parallax amount, carries out parallax adjustment of the predetermined object, and that, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, carries out parallax adjustment on another object that is different than the predetermined object.

8. The composite eye imaging device of claim 7, wherein the parallax adjusting section makes an object, which has a closest distance to the predetermined object in a direction orthogonal to an image plane of the viewpoint images, be the other object.

9. The composite eye imaging device of claim 7, wherein the parallax adjusting section makes an object, which has a closest distance to the predetermined object in an image plane of the viewpoint images, be the other object.

10. The composite imaging device of claim 7, further comprising an associating section that associates at least one information of information that is obtained from the plurality of viewpoint images generated by the imaging sections and that is used for acquiring the parallax amount at the parallax amount acquiring section, information that expresses the parallax amount acquired by the parallax amount acquiring section by using the information obtained from the plurality of viewpoint images, or information expressing whether or not a predetermined processing has been carried out by using the parallax amount, with the predetermined object or the other object that corresponds to the at least one information and on which parallax adjustment has been carried out.

11. The composite eye imaging device of claim 10, wherein the associating section associates the at least one information with the predetermined object or the other object that corresponds to the at least one information and on which parallax adjustment has been carried out, and collects the at least one information in one electronic file.

12. A parallax adjusting method of a composite eye imaging device, the method comprising:

acquiring a parallax amount on the basis of a plurality of viewpoint images generated by imaging sections that generate the plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

judging that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the acquired parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected, and judging that there is no abnormality in the parallax amount in a case that is none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

when it is judged that there is no abnormality in the parallax amount, carrying out first parallax adjustment, and, when it is judged that there is an abnormality in the parallax amount, switching to control of a second parallax adjustment that is different than control of the first parallax adjustment, and carrying out parallax adjustment.

13. A parallax adjusting method of a composite eye imaging device, the method comprising:

acquiring a parallax amount of a predetermined object on the basis of a plurality of viewpoint images generated by imaging sections that generate the plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

judging that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the acquired parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, of a case in which an object of acquisition of the parallax amount can no longer be detected, and judging that there is no abnormality in the parallax amount in a case that is none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

when it is judged that there is no abnormality in the parallax amount, carrying out parallax adjustment of the predetermined object, and, when it is judged that there is an abnormality in the parallax amount, carrying out parallax adjustment on another object that is different than the predetermined object.

14. A parallax adjusting program of a composite eye imaging device including computer-readable instructions, for causing a computer to function as:

a parallax amount acquiring section that acquires a parallax amount on the basis of a plurality of viewpoint images generated by imaging sections that generate the plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

an abnormality judging section that judges that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the parallax amount acquired by the parallax amount acquiring section is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected, and judges that there is no abnormality in the parallax amount in a case that is none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

a parallax adjusting section that, when it is judged by the abnormality judging section that there is no abnormality in the parallax amount, carries out first parallax adjustment, and that, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, switches to control of a second parallax adjustment that is different than control of the first parallax adjustment, and carries out parallax adjustment.

15. A parallax adjusting program of a composite eye imaging device including computer-readable instructions, for causing a computer to function as:

a parallax amount acquiring section that acquires a parallax amount of a predetermined object on the basis of a plurality of viewpoint images generated by imaging sections that generate the plurality of viewpoint images per frame by imaging a same subject from a plurality of viewpoints;

an abnormality judging section that judges that there is an abnormality in the parallax amount in at least one case of a case in which fluctuation of a fixed interval in the parallax amount acquired by the parallax amount acquiring section is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected, and judges that there is no abnormality in the parallax amount in a case that is

none of a case in which fluctuation of a fixed interval in the parallax amount is greater than a predetermined value, a case in which the parallax amount reaches an allowed limit value that is determined in advance, or a case in which an object of acquisition of the parallax amount can no longer be detected; and

a parallax adjusting section that, when it is judged by the abnormality judging section that there is no abnormality in the parallax amount, carries out parallax adjustment of the predetermined object, and that, when it is judged by the abnormality judging section that there is an abnormality in the parallax amount, carries out parallax adjustment on another object that is different than the predetermined object.