Three-dimensional image processing apparatus, three-dimensional image-pickup apparatus, three-dimensional image-pickup method, and program

• US 9,240,072 B2
• Filed: 03/06/2013
• Issued: 01/19/2016
• Est. Priority Date: 09/08/2010
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Yamashita, Haruo, Ito, Takeshi, Ono, Hiromichi, Kuwahara, Yasuhiro
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/787,656
• Publication Number:
  US 20130195349A1
• Time in Patent Office: 1,049 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance

Abstract

A sense of three-dimensionality and thickness is restored to a subject and a high-quality three-dimensional image with a low sense of a cardboard cutout effect is obtained, regardless of the cause of the cardboard cutout effect. In a three-dimensional image capturing apparatus (three-dimensional image processing apparatus) (1000), a depth generation unit (103) obtains L depth information and R depth information from a three-dimensional image, and an image correction unit (104) executes a shadow enhancement process that is stronger on the inside of an object than on the outside of the object, based on the L depth information and the R depth information. Through this, the occurrence of the cardboard cutout effect can be suppressed, and a high-quality three-dimensional image with a reduced cardboard cutout effect can be obtained.

Claims

1. A three-dimensional image processing apparatus that performs an image correction process on a left eye image and a right eye image contained in a three-dimensional image obtained through a dual-lens technique or a multiple-viewpoint technique, the apparatus comprising:

an image corrector configured to;

obtain a subject object from the left eye image and right eye image;

perform an enhancement process configured to emphasize a shadow or shade produced by protrusions and recesses in a surface of the subject object using a first strength on an inward region of the obtained subject object; and

perform an enhancement process configured to emphasize a shadow or shade produced by the protrusions and recesses in the surface of the subject object using a second strength that is weaker than the first strength on an outward region of the obtained subject object;

wherein

the image corrector includes an R image corrector and an L image corrector, and

the R image corrector includes;

an R image local tone converter configured to perform the enhancement process on a right eye image signal IS_R and output the right eye image signal on which the enhancement process has been performed as a corrected right eye image signal OS_R;

an R image strength generator configured to determine a strength of an image correction process based on the right eye image distance image; and

an R image synthesizer configured to synthesize the right eye image signal IS_R and the corrected right eye image signal OS_R based on the strength determined by the R image strength generator; and

the L image corrector includes;

an L image local tone converter configured to perform the enhancement process on a left eye image signal IS_L and output the left eye image signal on which the enhancement process has been performed as a corrected left eye image signal OS_L;

an L image strength generator configured to determine a strength of an image correction process based on the left eye image distance image; and

an L image synthesizer configured to synthesize the left eye image signal IS_L and the corrected left eye image signal OS_L based on the strength determined by the L image strength generator.

2. The three-dimensional image processing apparatus according to claim 1, wherein the second strength is zero.

3. The three-dimensional image processing apparatus according to claim 1, wherein:

the R image local tone converter includes;

an R image surrounding brightness detector configured to;

detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R and a first periphery pixel in the periphery of the first pixel of interest; and

output an R image surrounding brightness signal US_R that takes the first detected representative brightness value as a signal value; and

an R image second dynamic tone corrector configured to obtain the corrected right eye image signal OS_R by performing a first dynamic tone correction process based on the right eye image signal IS_R and the R image surrounding brightness signal US_R, the first dynamic tone correction process including;

obtaining the corrected right eye image signal OS_R by performing a first tone conversion process based on a first tone conversion property in which when the value of the right eye image signal IS_R is fixed at a first predetermined value within a predetermined input range of the right eye image signal IS_R, and the value of the corrected right eye image signal OS_R decreases as the value of the R image surrounding brightness signal US_R increases when the value of the right eye image signal IS_R is less than or equal to the value of the R image surrounding brightness signal US_R; and

obtaining the corrected right eye image signal OS_R by taking the right eye image signal IS_R as the corrected right eye image signal OS_R when the value of the right eye image signal IS_R is less than the value of the R image surrounding brightness signal US_R; and

wherein the L image local tone converter includes;

an L image surrounding brightness detector configured to;

detect a second brightness value of a second pixel of interest corresponding to the left eye image signal IS_L and a second periphery pixel in the periphery of the second pixel of interest; and

output an L image surrounding brightness signal US_L that takes the detected second brightness value as a signal value; and

an L image second dynamic tone corrector configured to;

obtain the corrected left eye image signal OS_L by performing a second dynamic tone correction process based on the left eye image signal IS_L and the L image surrounding brightness signal US_L, the second dynamic tone correction process including;

obtaining the corrected left eye image signal OS_L by performing a second tone conversion process based on a second tone conversion property in which, when the value of the left eye image signal IS_L is fixed at a second predetermined value within a predetermined input range of the left eye image signal IS_L, and the value of the corrected left eye image signal OS_L decreases as the value of the L image surrounding brightness signal US_L increases when the value of the left eye image signal IS_L is less than or equal to the value of the L image surrounding brightness signal US_L; and

obtaining the corrected left eye image signal OS_L by taking the left eye image signal IS_L as the corrected left eye image signal OS_L when the value of the left eye image signal IS_L is greater than the value of the L image surrounding brightness signal US_L.

4. The three-dimensional image processing apparatus according to claim 1,

wherein the R image local tone converter includes:

an R image second surrounding brightness detector configured to;

detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the first pixel of interest;

obtain an R image surrounding brightness signal US_R that takes the first detected representative brightness value as a signal value;

obtain a right eye offset value Δ

US_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region; and

obtain an R image corrected surrounding brightness signal US_R′

by adding the right eye offset value Δ

US_R to the R image surrounding brightness signal US_R; and

an R image dynamic tone corrector configured to obtain the corrected right eye image signal OS_R by performing a dynamic tone correction process based on the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′

, the dynamic tone correction process obtaining the corrected right eye image signal OS_R by performing a tone conversion process based on a tone conversion property in which the value of the corrected right eye image signal OS_R decreases as the value of the R image corrected surrounding brightness signal US_R′

increases when the value of the right eye image signal IS_R is fixed at a predetermined value in a predetermined input range of the right eye image signal IS_R; and

wherein the L image local tone converter includes;

an L image second surrounding brightness detector configured to;

detect a second brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and a second periphery pixel in the periphery of the second pixel of interest;

obtain an L image surrounding brightness signal US_L that takes the second detected brightness value as a signal value;

obtain a left eye offset value Δ

US_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and

obtain an L image corrected surrounding brightness signal US_L′

by adding the left eye offset value Δ

US_L to the L image surrounding brightness signal US_L; and

an L image dynamic tone corrector configured to obtain the corrected left eye image signal OS_L by performing a dynamic tone correction process based on the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′

, the dynamic tone correction process obtaining the corrected left eye image signal OS_L by performing a tone conversion process based on a tone conversion property in which the value of the corrected left eye image signal OS_L decreases as the value of the L image corrected surrounding brightness signal US_L′

increases when the value of the left eye image signal IS_L is fixed at a predetermined value in a predetermined input range of the left eye image signal IS_L.

5. The three-dimensional image processing apparatus according to claim 1, wherein:

the R image local tone converter includes;

an R image second surrounding brightness detector configured to;

detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the first pixel of interest;

obtain an R image surrounding brightness signal US_R that takes the first detected brightness value as a signal value;

obtain a right eye offset value Δ

US_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region; and

obtain an R image corrected surrounding brightness signal US_R′

by adding the right eye offset value Δ

US_R to the R image surrounding brightness signal US_R; and

an R image coefficient computation processor configured to;

determine a coefficient k that takes on a lower value the higher the value of a difference between the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′

according to the formula ((IS_R)−

(US_R′

)); and

obtain the corrected right eye image signal OS_R according to the formula OS_R=IS_R+k×

((IS_R)−

(US_R′

)) using the determined coefficient k; and

wherein the L image local tone converter includes;

an L image second surrounding brightness detector configured to;

detect a second representative brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and a second periphery pixel in the periphery of the first pixel of interest;

obtain an L image surrounding brightness signal US_L that takes the second detected representative brightness value as a signal value;

obtains a left eye offset value Δ

US_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and

obtain an L image corrected surrounding brightness signal US_L′

by adding the left eye offset value Δ

US_L to the L image surrounding brightness signal US_L; and

an L image coefficient computation processor configured to;

determine a coefficient k′

that takes on a lower value the higher the value of a difference between the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′

according to the formula ((IS_L)−

(US_L′

)); and

obtain the corrected left eye image signal OS_L through OS_L=IS_L+k′

×

((IS_L)−

(US_L′

)) using the determined coefficient k′

.

6. The three-dimensional image processing apparatus according to claim 1, wherein:

the R image local tone converter includes;

an R image second surrounding brightness detector configured to;

detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the pixel of interest;

obtain an R image surrounding brightness signal US_R that takes the detected first representative brightness value as a signal value;

obtain a right eye offset value Δ

US_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region;

and obtain an R image corrected surrounding brightness signal US_R′

by adding the right eye offset value Δ

US_R to the R image surrounding brightness signal US_R; and

an R image coefficient computation processor configured to;

determine a coefficient k that takes on a lower value the higher the value of a difference between the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′

according to the formula ((IS_R)−

(US_R′

));

obtain a signal LPF ((k+p)×

(IS_R−

US_R′

)) by setting a coefficient p, such that 0 is less than or equal to p which is less than or equal to 1, and performing a bandwidth limiting process on a signal obtained according to the formula (k+p)×

(IS_R−

US_R′

); and

obtain the corrected right eye image signal OS_R according to the formula OS_R=IS_R−

p×

(IS_R−

US_R′

)+LPF((k+p)×

(IS_R−

US_R′

)) using the obtained signal LPF((k+p)×

(IS_R−

US_R′

)), and

wherein the L image local tone converter includes;

an L image second surrounding brightness detector configured to;

detect a second representative brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and second periphery a pixel in the periphery of the second pixel of interest;

obtain an L image surrounding brightness signal US_L that takes the second detected brightness value as a signal value;

obtain a left eye offset value Δ

US_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and

obtain an L image corrected surrounding brightness signal US_L′

by adding the left eye offset value Δ

US_L to the L image surrounding brightness signal US_L; and

an L image coefficient computation processor configured to;

determine a coefficient k′

that takes on a lower value the higher the value of a difference between the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′

according to the formula ((IS_L)−

(US_L′

));

obtain a signal LPF ((k′

+p)×

(IS_L−

US_L′

)) by setting a coefficient p, such that 0 is less than or equal to p which is less than or equal to 1, and performing a bandwidth limiting process on a signal obtained according to the formula (k+p)×

(IS_L−

US_L′

); and

obtain the corrected left eye image signal OS_L according to the formula OS_L=IS_L−

p×

(IS_L−

US_L′

)+LPF((k′

+p)×

(IS_L−

US_L′

)) using the obtained signal LPF((k′

+p)×

(IS_L−

US_L′

)).