Image processing apparatus, image division program and image synthesising method

• US 8,184,194 B2
• Filed: 05/15/2009
• Issued: 05/22/2012
• Est. Priority Date: 06/26/2008
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Sato, Satoshi, Kanamori, Katsuhiro
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US12/674,538
• Publication Number:
  US 20100283883A1
• Time in Patent Office: 1,103 Days
• Field of Search: 353/20, 353/25, 353/121, 353/28, 382/162, 382/167, 356/364, 348/335
• US Class Current: 348/335
• CPC Class Codes: G02B 27/283 used for beam splitting or combining
• CPC Class Codes: G02B 27/288 Filters employing polarising elements, e.g. Lyot or Solc filters G02B5/3016 takes precedence
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A polarization image capturing section 103 obtains a first polarization image containing polarization information. A polarization information obtaining section 104 obtains first polarization information from the first polarization image. An illumination control section 102 changes an illumination section 101 and again obtains a second polarization image by means of the polarization image capturing section 103. The polarization information obtaining section 104 obtains second polarization information from the second polarization image. A light source dependency estimation section 105 obtains light source dependency from the first and second polarization information obtained from the first and second polarization images. An area dividing section 106 performs an area division by using the polarization information and light source dependency information.

Claims

1. An image processing apparatus for capturing an image of an object and thereby performing an optical area division of a surface of the object, comprising:

a polarization image capturing section for obtaining a polarization image of the object by receiving light having passed through a three-way or more polarizer having different polarization main axis angles,

wherein the area division is performed based on a polarization state on the surface of the object and a light source dependency, which represents a degree of change in the polarization state on the surface of the object occurring when changing at least one of a polarization state of light emitted from a light source and a light source position.

2. The image processing apparatus according to claim 1, comprising:

an illumination unit including the light source and being capable of changing at least one of the polarization state of light emitted from the light source and the light source position; and

a control section for synchronizing the changing by the illumination unit of at least one of the polarization state of light emitted from the light source and the light source position with the image-capturing by the polarization image capturing section.

3. The image processing apparatus according to claim 2, comprising:

an illumination control section for changing the polarization state of light emitted from the light source;

a polarization information obtaining section for generating, from the obtained polarization image, polarization information for received polarized light by using a correspondence with a brightness of the light having passed through the three-way or more polarizer, for each of unit pixels of the polarization image;

a light source dependency estimation section for estimating light source dependency information by making a comparison between the polarization information generated by the polarization information obtaining section when the polarization state of light emitted from the light source is changed by the illumination control section; and

an area dividing section for performing the area division based on the polarization information generated by the polarization information obtaining section and light source variation information estimated by the light source dependency estimation section.

4. The image processing apparatus according to claim 3, wherein the polarization information obtaining section generates, as the polarization information, at least one of a degree of polarization, a polarization phase, a polarization estimation error, a polarization maximum luminance value, and a polarization minimum luminance value.

5. The image processing device according to claim 4, wherein the area dividing section performs the area division so that the surface of the object is divided into areas each being a diffuse reflection area or a specular reflection area.

6. The image processing apparatus according to claim 4, wherein the area dividing section performs the area division so that the surface of the object is divided into areas each being a diffuse reflection non-polarized area, a diffuse reflection polarized area, or a specular reflection polarized area.

7. The image processing apparatus according to claim 4, wherein the area dividing section performs the area division so that the surface of the object is divided into areas each being a light source-dependent polarized area, a light source-dependent non-polarized area, a light source-independent polarized area, or a light source-independent non-polarized area.

8. The image processing apparatus according to claim 2, comprising:

an illumination position changing section for changing the position of the light source;

a polarization information obtaining section for generating, from the obtained polarization image, polarization information for received polarized light by using a correspondence with a brightness of the light having passed through the three-way or more polarizer, for each of unit pixels of the polarization image;

a light source dependency estimation section for estimating light source dependency information by making a comparison between the polarization information generated by the polarization information obtaining section when the position of the light source is changed by the illumination control section; and

an area dividing section for performing the area division based on the polarization information generated by the polarization information obtaining section and light source variation information estimated by the light source dependency estimation section.

9. The image processing apparatus according to claim 8, comprising:

a plurality of illumination sections of different light source positions,

wherein the illumination position changing section changes a position of illumination section by switching between the plurality of illumination sections.

10. The image processing apparatus according to claim 8, wherein the illumination position changing section changes the light source position by moving an illumination section.

11. The image processing apparatus according to claim 8, wherein the area dividing section performs the area division so that the surface of the object is divided into areas each being a diffuse reflection area or a specular reflection area.

12. The image processing apparatus according to claim 8, wherein the area dividing section performs the area division so that the surface of the object is divided into areas each being a specular reflection area, a diffuse reflection area, or a shaded area.

13. An image processing system comprising an illumination device and an area dividing device for capturing an image of an object and thereby performing an optical area division of a surface of the object,

the illumination device comprising:

an illumination section for emitting linearly-polarized light onto the object; and

an illumination control section for changing a polarization state of the linearly-polarized light of the illumination section,

the area dividing device comprising;

a polarization image capturing section for obtaining a polarization image of the object by receiving light having passed through a three-way or more polarizer having different polarization main axis angles;

a captured image determination section for comparing a polarization image captured in a past by the polarization image capturing section with a currently-captured polarization image to thereby determine whether the captured polarization states have changed sufficiently;

a polarization information obtaining section for generating, from the obtained polarization image, polarization information for received polarized light by using a correspondence with a brightness of the light having passed through the three-way or more polarizer, for each of unit pixels of the polarization image;

a light source dependency estimation section for estimating a light source dependency information by making a comparison between the polarization information generated by the polarization information obtaining section when it is determined by the captured image determination section that the polarization state of the illumination section has been changed by the illumination control section; and

an area dividing section for performing the area division by using the polarization information generated by the polarization information obtaining section and light source variation information estimated by the light source dependency estimation section.

14. An image processing system comprising an illumination device and an area dividing device for capturing an image of an object and thereby performing an optical area division of a surface of the object,

the illumination device comprising:

an illumination section for emitting linearly-polarized light onto the object;

an illumination control section for changing a polarization state of the linearly-polarized light of the illumination section; and

a communication section for transmitting a signal indicating a light emission to the area dividing device and receiving a signal indicating a completion of image-capturing from the area dividing device,

the area dividing device comprising;

a communication section for receiving a signal indicating a light emission from the illumination device and transmitting a signal indicating a completion of image-capturing to the illumination device;

a polarization image capturing section for obtaining a polarization image of the object by receiving light having passed through a three-way or more polarizer having different polarization main axis angles;

a captured image determination section for comparing a polarization image captured in a past by the polarization image capturing section with a currently-captured polarization image to thereby determine whether the captured polarization states have changed sufficiently;

a polarization information obtaining section for generating, from the obtained polarization image, polarization information for received polarized light by using a correspondence with a brightness of the light having passed through the three-way or more polarizer, for each of unit pixels of the polarization image;

a light source dependency estimation section for estimating light source dependency information by making a comparison between the polarization information generated by the polarization information obtaining section when it is determined by the captured image determination section that the polarization state of the illumination section has been changed by the illumination control section; and

an area dividing section for performing the area division by using the polarization information generated by the polarization information obtaining section and light source variation information estimated by the light source dependency estimation section.

15. An area dividing method for capturing an image of an object and thereby performing an optical area division of a surface of the object, comprising:

an illumination step of emitting linearly-polarized light onto the object;

an illumination control step of changing a polarization state of the linearly-polarized light of the illumination section;

a polarization image capturing step of obtaining a polarization image of the object by receiving light having passed through a three-way or more polarizer having different polarization main axis angles;

a polarization information obtaining step of generating, from the obtained polarization image, polarization information for received polarized light by using a correspondence with a brightness of the light having passed through the three-way or more polarizer, for each of unit pixels of the polarization image;

a control step of synchronizing the illumination control step with the polarization image capturing step;

a light source dependency estimation step of estimating light source dependency information by making a comparison between the polarization information generated by the polarization information obtaining step when the polarization state of the illumination step is changed by the illumination control step; and

an area dividing step of performing the area division by using the polarization information generated by the polarization information obtaining step and light source variation information estimated by the light source dependency estimation step.

16. A non-transitory computer readable medium having stored thereon a program for an area dividing device for capturing an image of an object and thereby performing an optical area division of a surface of the object,

wherein the program instructs a computer to perform steps included in the area dividing method according to claim 15.

17. A model-based image synthesis device comprising a parameter estimation device and an image synthesis apparatus for synthesizing an image using a reflection model,

the parameter estimation device comprising:

an image capturing section for capturing an image of an object;

an area dividing section for performing an optical area division of an image captured by the image capturing section according to the area dividing method according to claim 15;

a light source information estimating section for estimating light source information regarding a light source illuminating the object;

a shape information obtaining section for obtaining, as shape information, surface normal information or 3-dimensional position information of the object;

a parameter estimation section for estimating a reflection model parameter by modeling the light source information estimated by the light source information estimating section for each area divided by the area dividing section and the shape information obtained by the shape information obtaining section, from the captured image of the object; and

a parameter database for storing the reflection parameter estimated by the parameter estimation section,

the image synthesis apparatus comprising;

a viewpoint/light source information obtaining section for obtaining a viewpoint or light source information of an image to be synthesized; and

a rendering section for synthesizing an image according to the viewpoint or light source information obtained by the viewpoint/light source information obtaining section by using the reflection parameter information stored in the parameter database.

18. A model-based image synthesis method comprising a parameter estimation step and an image synthesis method step for synthesizing an image using a reflection model,

the parameter estimation step comprising:

an image capturing step of capturing an image of an object;

an area dividing step of performing an optical area division of an image captured by the image capturing section according to the area dividing method according to claim 15;

a light source information estimating step of estimating light source information regarding a light source illuminating the object;

a shape information obtaining step of obtaining, as shape information, surface normal information or 3-dimensional position information of the object; and

a parameter estimation step of estimating a reflection model parameter by modeling the light source information estimated by the light source information estimating section for each area divided by the area dividing section and the shape information obtained by the shape information obtaining section, from the captured image of the object,

the image synthesis step comprising;

a viewpoint/light source information obtaining step of obtaining a viewpoint or light source information of an image to be synthesized; and

a rendering step of synthesizing an image according to the viewpoint or light source information obtained by the viewpoint/light source information obtaining section by using reflection parameter information estimated by the shape information obtaining step.