Image processing apparatus, image processing method, image pickup apparatus, and display device

• US 9,426,349 B2
• Filed: 05/27/2014
• Issued: 08/23/2016
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Fukuda, Koichi
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US14/287,327
• Publication Number:
  US 20140253753A1
• Time in Patent Office: 819 Days
• Field of Search: 348/42, 348/49, 348/218.1, 348/349, 348/222.1, 348/280, 348/239, 382/300
• US Class Current: 1/1
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An image processing apparatus generates a plurality of parallactic images having a relationship that the magnitude of an amount of image shift between parallactic images increases with an increase of a defocus amount, generates, through plural selections of the generated parallactic images permitting a repeated same selection, a series of parallactic images in which the selected parallactic images are arranged in a selection order; repeatedly applies interpolation processing to each pair of adjacent parallactic images included in the series of parallactic images to generate a plurality of interpolation images; and arranges the interpolation images between the adjacent parallactic images to output the interpolated parallactic images as an output image of a moving image.

Claims

1. An image processing apparatus comprising:

a generation unit which generates at least a first parallactic image, a second parallactic image, and a third parallactic image, wherein the first parallactic image, the second parallactic image, and the third parallactic image each corresponds to each image formed by light passing through different pupil areas; and

an output unit which repeatedly outputs the first parallactic image, the second parallactic image, and the third parallactic image in order, wherein the parallax between the first parallactic image and the second parallactic image is smaller than the parallax between the first parallactic image and the third parallactic image.

2. The image processing apparatus according to claim 1, wherein the generation unit selects the generated parallactic images such that the first parallactic image of a parallactic image series and a last parallactic image are the same.

3. The image processing apparatus according to claim 1, wherein the generation unit selects the generated parallactic images such that the pupil areas corresponding to adjacent parallactic images are adjacent to each other.

4. The image processing apparatus according to claim 1, further comprising an interpolation processing unit which repeatedly applies interpolation processing to each pair of adjacent parallactic images while changing weighting in a stepwise manner.

5. The image processing apparatus according to claim 1, wherein the output unit outputs the parallactic images as movie image.

6. An image pickup apparatus comprising:

a focusing optical system;

an image pickup unit in which pixels that receive light passing through different pupil areas of the focusing optical system are arranged; and

the image processing apparatus according to claim 1.

7. The image pickup apparatus according to claim 6, wherein the pixel has a plurality of sub-pixels receiving light passing through different pupil areas of the focusing optical system, and the generation unit generates the parallactic images from an output of the sub-pixel corresponding to each pupil area.

8. The image pickup apparatus according to claim 7, wherein the sub-pixels are two-dimensionally arranged.

9. A display device comprising:

a display unit; and

the image processing apparatus according to claim 1.

10. A control method of an image processing apparatus, comprising the steps of:

generating at least a first parallactic image, a second parallactic image, and a third parallactic image, wherein the first parallactic image, the second parallactic image, and the third parallactic image each corresponds to each image formed by light passing through different pupil areas; and

repeatedly outputting the first parallactic image, the second parallactic image, and the third parallactic image in order, wherein the parallax between the first parallactic image and the second parallactic image is smaller than the parallax between the first parallactic image and the third parallactic image.

11. An image processing apparatus comprising:

a first generation unit which generates at least a first parallactic image, a second parallactic image, and a third parallactic image, wherein the first parallactic image, the second parallactic image, and the third parallactic image each corresponds to each image formed by light passing through different pupil areas;

a second generation unit which generates a first interpolation image which is based on the first parallactic image and the second parallactic image, and generates a second interpolation image which is based on the second parallactic image and the third parallactic image; and

an output unit which outputs the first parallactic image, the first interpolation image, the second parallactic image, the second interpolation image and the third parallactic image in order, wherein the parallax between the first parallactic image and second parallactic image is smaller than the parallax between the first parallactic image and the third parallactic image.

12. The image processing apparatus according to claim 11, wherein the generation unit selects the generated parallactic images such that the first parallactic image of a parallactic image series and a last parallactic image are the same.

13. The image processing apparatus according to claim 11, wherein the generation unit selects the generated parallactic images such that the pupil areas corresponding to adjacent parallactic images are adjacent to each other.

14. The image processing apparatus according to claim 11, further comprising an interpolation processing unit which repeatedly applies interpolation processing to each pair of adjacent parallatic images while changing weighting in a stepwise manner.

15. The image processing apparatus according to claim 11, wherein the output unit outputs the parallactic images as movie image.

16. An image pickup apparatus comprising:

a focusing optical system;

an image pickup unit in which pixels that receive light passing through different pupil areas of the focusing optical system are arranged; and

the image processing apparatus according to claim 11.

17. The image pickup apparatus according to claim 16, wherein the pixel has a plurality of sub-pixels receiving light passing through different pupil areas of the focusing optical system, and the first generation unit generates the parallactic images from an output of the sub-pixel corresponding to each pupil area.

18. The image pickup apparatus according to claim 17, wherein the sub-pixels are two-dimensionally arranged.

19. A display device comprising:

a display unit; and

the image processing apparatus according to claim 11.

20. A control method of an image processing apparatus, comprising the steps of:

generating at least a first parallactic image, a second parallactic image, and a third parallactic image, wherein the first parallactic image, the second parallactic image, and the third parallactic image each corresponds to each image formed by light passing through different pupil areas;

generating a first interpolation image which is based on the first parallactic image and the second parallactic image, and generating a second interpolation image which is based on the second parallactic image and the third parallactic image; and

outputting the first parallactic image, the first interpolation image, the second parallactic image, the second interpolation image, and the third parallactic image in order, wherein the parallax between the first parallactic image and the second parallactic image is smaller than the parallax between the first parallactic image and the third parallactic image.