SIGNAL PROCESSING APPARATUS, IMAGING APPARATUS, SIGNAL PROCESSING METHOD AND PROGRAM

• US 20130027587A1
• Filed: 07/05/2012
• Published: 01/31/2013
• Est. Priority Date: 05/17/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: MATSUI, Akira, Mizukura, Takami
• Granted Patent: US 9,697,589 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

Provided is a signal processing apparatus, including an input unit into which first image data and second image data are input, the first image data and the second image data being obtained by capturing a predetermined subject with an imaging unit and having mutually different states of blurring; a distance information calculation unit that calculates first distance information in accordance with a position in a depth direction of the subject based on the first image data and the second image data; and a distance information correction unit that calculates second distance information by performing, on the first distance information, processing to correct a deviation of the blurring caused by a mismatch between an image-forming plane of the subject and an imaging plane of the imaging unit.

Claims

1. A signal processing apparatus, comprising:

an input unit into which first image data and second image data are input, the first image data and the second image data being obtained by capturing a predetermined subject with an imaging unit and having mutually different states of blurring;

a distance information calculation unit that calculates first distance information in accordance with a position in a depth direction of the subject based on the first image data and the second image data; and

a distance information correction unit that calculates second distance information by performing, on the first distance information, processing to correct a deviation of the blurring caused by a mismatch between an image-forming plane of the subject and an imaging plane of the imaging unit.

2. The signal processing apparatus according to claim 1,

wherein the distance information correction unit calculates a correction value to correct a deviation of the blurring by using known information and calculates the second distance information by performing, on the first distance information, processing using the correction value.

3. The signal processing apparatus according to claim 1,

wherein the distance information correction unit calculates a correction value to correct a deviation of the blurring by using the first distance information and calculates the second distance information by performing, on the first distance information, processing using the correction value.

4. The signal processing apparatus according to claim 3,

wherein the distance information correction unit acquires a plurality of bases modeling an error superimposed on the first distance information and calculates the correction value to correct the deviation of the blurring by using the first distance information and the plurality of bases.

5. The signal processing apparatus according to claim 4,

wherein the distance information correction unit calculates the correction value to correct the deviation of the blurring by representing the first distance information as a linear sum of the plurality of bases.

6. The signal processing apparatus according to claim 3,

wherein the subject includes a foreground subject and a background subject, and

the distance information correction unit replaces the first distance information in positions corresponding to the foreground subject by using a portion of the first distance information in positions corresponding to the background subject and calculates the correction value to correct the deviation of the blurring by using the first distance information in positions corresponding to the background subject and the first distance information after being replaced.

7. The signal processing apparatus according to claim 1,

wherein the first image data is obtained by capturing the subject in a first optical unit state of the imaging unit,

the second image data is obtained by capturing the subject in a second optical unit state of the imaging unit, and

the first optical unit state and the second optical unit state are mutually different states of a position of focus.

8. The signal processing apparatus according to claim 1,

wherein the first image data is obtained by capturing the subject in a first optical unit state of the imaging unit,

the second image data is obtained by capturing the subject in a second optical unit state of the imaging unit, and

the first optical unit state and the second optical unit state are mutually different states of an iris.

9. The signal processing apparatus according to claim 1,

wherein the distance information calculation unit performs blurring processing of adding the blurring to the first image data through a filter, determines a similarity between the first image data to which the blurring has been added and the second image data, detects the number of times of the blurring processing when the similarity is maximum, and calculates the first distance information from the detected number of times of the blurring processing.

10. The signal processing apparatus according to claim 9, wherein the number of times of the blurring processing that adds no blurring is set as 0.

11. An imaging apparatus, comprising:

an imaging unit;

an input unit into which first image data and second image data are input, the first image data and the second image data being obtained by capturing a predetermined subject with the imaging unit and having mutually different states of blurring;

a distance information calculation unit that calculates first distance information in accordance with a position in a depth direction of the subject based on the first image data and the second image data; and

a distance information correction unit that calculates second distance information by performing, on the first distance information, processing to correct a deviation of the blurring caused by a mismatch between an image-forming plane of the subject and an imaging plane of the imaging unit.

12. A signal processing method for a signal processing apparatus, the method comprising:

inputting first image data and second image data, the first image data and the second image data being obtained by capturing a predetermined subject with an imaging unit and having mutually different states of blurring;

calculating first distance information in accordance with a position in a depth direction of the subject based on the first image data and the second image data; and

calculating second distance information by performing, on the first distance information, processing to correct a deviation of the blurring caused by a mismatch between an image-forming plane of the subject and an imaging plane of the imaging unit.

13. A program causing a computer to execute a signal processing method for a signal processing apparatus, the method comprising:

inputting first image data and second image data, the first image data and the second image data being obtained by capturing a predetermined subject with an imaging unit and having mutually different states of blurring;

calculating first distance information in accordance with a position in a depth direction of the subject based on the first image data and the second image data; and

calculating second distance information by performing, on the first distance information, processing to correct a deviation of the blurring caused by a mismatch between an image-forming plane of the subject and an imaging plane of the imaging unit.