Parallax image capturing apparatus and parallax image processing apparatus

• US 20020028014A1
• Filed: 08/23/2001
• Published: 03/07/2002
• Est. Priority Date: 08/25/2000
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Ono, Shuji
• Granted Patent: US 7,200,261 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

An image capturing apparatus for obtaining information regarding a depth of a subject, includes: a capturing section for performing image capturing for the subject at a plurality of viewpoints; and a controller for controlling the capturing section to successively perform the image capturing at the plurality of viewpoints, wherein the controller controls the capturing section to perform the image capturing for the subject two or more times at least one of the viewpoints.

Claims

1. An image capturing apparatus for obtaining information regarding a depth of a subject, comprising:

a capturing section operable to perform image capturing for said subject at a plurality of viewpoints; and

a controller operable to control said capturing section to perform said image capturing at different timings at said plurality of viewpoints, wherein

said controller controls said capturing section to perform said image capturing for said subject two or more times at least one of said plurality of viewpoints.

2. An image capturing apparatus as claimed in claim 1, wherein said capturing section includes a plurality of capturing units respectively provided at a plurality of different positions, and

said controller controls said plurality of capturing units to successively perform said image capturing for said subject in such a manner that at least one of said capturing units performs said image capturing two or more times.

3. An image capturing apparatus as claimed in claim 1, further comprising a viewpoint moving unit operable to move a position at which said capturing section performs said image capturing for said subject to said plurality of viewpoints, wherein

said controller controls said capturing section to perform said image capturing for said subject at said plurality of viewpoints successively by moving said position to said plurality of viewpoints, thereby allowing said image capturing to be performed at said at least one viewpoint two or more times.

4. An image capturing apparatus as claimed in claim 1, further comprising a depth calculating unit operable to calculate a depth of a particular region of said subject based on two or more images obtained by said image capturing performed for said subject two or more times at said one viewpoint and another image obtained by said image capturing performed at another viewpoint different from said one viewpoint.

5. An image capturing apparatus as claimed in claim 4, further comprising a positional difference detecting unit operable to detect a positional difference of an image of said particular region of said subject based on said two or more images obtained at said one viewpoint and said another image obtained at said another viewpoint, wherein

said depth calculating unit calculates said depth of said particular region of said subject based on said positional difference.

6. An image capturing apparatus as claimed in claim 5, further comprising an estimated image generating unit operable to generate an estimated image of said particular region of said subject that is assumed to be captured at said one viewpoint at the same time as a time at which said another image was captured at said another viewpoint, by estimation based on said two or more image obtained at said one viewpoint, wherein

said positional difference detecting unit detects a difference of a position of an image of said particular region of said subject between said estimated image and said another image obtained at said another viewpoint.

7. An image capturing apparatus as claimed in claim 6, further comprising a time setting unit operable to set a time of said image capturing by said capturing section, wherein

said estimated image generating unit estimates said position of said image of said particular region of said subject at a predetermined time at said one viewpoint, based on respective times at which said image capturing was performed two or more times at said one viewpoint and said position of said image of said particular region in said two ore more images obtained at said one viewpoint.

8. An image capturing apparatus as claimed in claim 3, wherein said capturing section includes a light converging unit operable to converge light incident thereon and a light-limiting unit having at least one aperture for limiting a range where said light is allowed to pass, and

said viewpoint moving unit moves said aperture by moving said light-limiting unit, to realize said plurality of viewpoints.

9. An image capturing apparatus as claimed in claim 3, wherein said capturing section includes a light converging unit operable to converge light incident thereon and a light-limiting unit having a plurality of apertures for limiting a range where said light is allowed to pass, and

said viewpoint moving unit closes at least one of said plurality of apertures to realize said plurality of viewpoints.

10. An image capturing apparatus as claimed in claim 3, wherein said viewpoint moving unit is capable of moving said capturing section at three or more viewpoints which are not aligned on one line.

11. An image capturing apparatus as claimed in claim 10, wherein said viewpoint moving unit is capable of moving said capturing section to five positions including apices and a center of a diamond.

12. An image capturing apparatus as claimed in claim 2, wherein said capturing section includes said plurality of capturing units at three or more viewpoints which are not aligned on one line.

13. An image capturing apparatus as claimed in claim 12, wherein said capturing section includes said plurality of capturing units at least five positions including apices and a center of a diamond.

14. An image capturing apparatus as claimed in claim 1, wherein said capturing section performs said image capturing for said subject at two of said plurality of viewpoints, and

said controller controls said capturing section to alternately perform said image capturing at said two viewpoints three or more times.

16. An image processing apparatus as claimed in claim 15, wherein said estimation image generating unit estimates said position of said image of said particular region of said subject when said subject is seen from said one viewpoint at the same time as said time at which said another image was captured at said another viewpoint, based on respective times at which said two or more images were shot at said one viewpoint and said position of said image of said particular region in said two or more images shot at said one viewpoint.

17. An image processing apparatus as claimed in claim 15, wherein said estimated image generating unit estimates said position of said image of said particular region of said subject when said subject is seen from said another viewpoint at the same time as said time at which said another image was captured at said another viewpoint, based on said position and a size of said image of said particular region in each of said two or more images shot at said one viewpoints.

18. An image processing apparatus as claimed in claim 15, wherein said estimated image generating unit separates a first region including a change of a position or size of an image of said subject between said two or more images shot at said one viewpoint from a second region including no change of said position or size of said image of said subject, and uses one of said two or more images shot at said one viewpoint as said estimated image in a case of generating said estimated image for said second region.

19. An image processing apparatus as claimed in claim 15, wherein said inputting unit alternately inputs a plurality of images shot at two of said plurality of viewpoints, said two viewpoints being different from each other,

said estimated image generating unit performs a first estimation using a first image shot at one of said two viewpoints and a second image shot at said one of said two viewpoints prior to said first image, and then performs a second estimation using a third image shot at said one of said two viewpoints after said first image and said first image, and

said positional difference detecting unit detects said positional difference between said first and second estimations by using said first image and a further image shot at the other of said two viewpoints.

15. An image processing apparatus for obtaining information regarding a depth of a subject, comprising:

an inputting unit operable to input a plurality of images of said subject successively shot at a plurality of different viewpoints, said plurality of images including two or more images shot at one viewpoints;

an estimated image generating unit operable to generate an estimated image that is assumed to be shot at said one viewpoint at a time the same as a time at which another one of said plurality of images was shot at another viewpoint different from said one viewpoint, by estimation based on said two or more images shot at said one viewpoint;

a positional difference detecting unit operable to detect a difference of a position of an image of a particular region of said subject between said estimated image and said another image shot at said another viewpoint; and

a depth calculating unit operable to calculate a depth of said particular region of said subject based on said difference of said position.

20. An image capturing method for obtaining information regarding a depth of a subject, comprising:

capturing a first image of said subject at a first time at a first viewpoint;

capturing a second image of said subject at a second time at a second viewpoint;

capturing a third image of said subject at a third time at said first viewpoint;

estimating an image of a particular region of said subject at said second time at said first viewpoint based on said first and third images;

detecting a positional difference between said estimated image of said particular region of said subject and an image of said particular region of said subject in said second image; and

calculating a depth of said particular region of said subject based on said detected positional difference.

21. A recording medium storing a computer program for obtaining information regarding a depth of a subject, said program comprising:

an inputting module operable to input a plurality of images of said subject successively shot at a plurality of different viewpoints, said plurality of images including two or more images shot at one viewpoints;

an estimated image generating module operable to generate an estimated image that is assumed to be shot at said one viewpoint at the same time as a time at which another one of said plurality of images was shot at another viewpoint different from said one viewpoint, by estimation based on said two or more images shot at said one viewpoint;

a positional difference detecting module operable to detect a difference of a position of an image of a particular region of said subject between said estimated image and said another image shot at said another viewpoint; and

a depth calculating module operable to calculate a depth of said particular region of said subject based on said difference of said position.

22. A computer program for obtaining information regarding a depth of a subject, comprising:

an inputting module operable to input a plurality of images of said subject successively shot at a plurality of different viewpoints, said plurality of images including two or more images shot at one viewpoints;

an estimated image generating module operable to generate an estimated image that is assumed to be shot at said one viewpoint at the same time as a time at which another one of said plurality of images was shot at another viewpoint different from said one viewpoint, by estimation based on said two or more images shot at said one viewpoint;

a positional difference detecting module operable to detect a difference of a position of an image of a particular region of said subject between said estimated image and said another image shot at said another viewpoint; and

a depth calculating module operable to calculate a depth of said particular region of said subject based on said difference of said position.