Image capturing apparatus capable of extracting subject region from captured image

• US 8,441,554 B2
• Filed: 03/04/2010
• Issued: 05/14/2013
• Est. Priority Date: 03/06/2009
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Makino, Tetsuji
• Primary Examiner(s): Henn, Timothy J
• Application Number:
  US12/717,342
• Publication Number:
  US 20100225787A1
• Time in Patent Office: 1,167 Days
• Field of Search: 348/222.1, 348/239, 385/173, 385/174, 385/190, 385/195
• US Class Current: 348/239
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 1/403 Discrimination between the two tones in the picture signal of a two-tone original
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An image capturing apparatus includes: a first image capturing unit that captures a subject existing image; a first nonflatness calculating unit that calculates nonflatness of a peripheral portion of the subject existing image; a first determination unit that determines whether the nonflatness of the peripheral portion of the subject existing image is higher than a threshold value; a background generating unit that generates an extracting background image from the peripheral portion of the subject existing image when the first determination unit determines that the nonflatness is not higher than the threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image; and a subject extracting unit that extracts the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image.

Claims

1. An image capturing apparatus comprising:

an image capturing unit that captures a subject existing image in which a subject is present in a background;

a dividing unit that divides a peripheral portion of the subject existing image into a plurality of image regions;

a nonflatness calculating unit that calculates nonflatness by extracting a feature point from each of the image regions divided by the dividing unit;

a determination unit that determines whether each of the feature points of the image regions extracted by the nonflatness calculating unit is higher than a first threshold value;

a background generating unit that generates an extracting background image from the peripheral portion of the subject existing image when the determination unit determines that each of the feature points is not higher than the first threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image; and

a subject extracting unit that extracts the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image.

2. The image capturing apparatus according to claim 1,

wherein the nonflatness calculating unit calculates each of dispersion values of pixel values as the nonflatness in the image regions divided by the dividing unit,

wherein the determination unit further determines whether each of the dispersion values of the pixel values in the image regions calculated by the nonflatness calculating unit is higher than a second threshold value, and

wherein the background generating unit generates the extracting background image from the peripheral portion of the subject existing image when the determination unit determines that each of the dispersion values of the pixel values is not higher than the second threshold value.

3. The image capturing apparatus according to claim 1, wherein the background generating unit generates the extracting background image having a same color as the peripheral portion of the subject existing image.

4. The image capturing apparatus according to claim 1, further comprising:

a mask generating unit that generates a mask image for extracting the subject region from the subject existing image based on the difference information of each corresponding pixel between the extracting background image and the subject existing image,

wherein the subject extracting unit extracts the subject region from the subject existing image using the mask image generated by the mask generating unit.

5. An image capturing apparatus comprising:

a first image capturing unit that captures a subject existing image in which a subject is present in a background;

a first nonflatness calculating unit that calculates nonflatness of a peripheral portion of the subject existing image captured by the first image capturing unit;

a first determination unit that determines whether the nonflatness of the peripheral portion of the subject existing image calculated by the first nonflatness calculating unit is higher than a first threshold value;

a background generating unit that generates an extracting background image from the peripheral portion of the subject existing image when the first determination unit determines that the nonflatness is not higher than the first threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image;

a subject extracting unit that extracts the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image;

a background image capturing unit that captures a background image in which the subject is absent and generates image frames of the background image sequentially;

a second nonflatness calculating unit that calculates nonflatness of each of the image frames of the background image; and

a second determination unit that determines whether the nonflatness of each of the image frames of the background image calculated by the second nonflatness calculating unit is higher than a second threshold value,

wherein the first image capturing unit captures a subject existing image in which the subject is present in a substantially same background as the background image when the second determination unit determines that the nonflatness is not higher than the second threshold value.

6. The image capturing apparatus according to claim 5,

wherein the second nonflatness calculating unit extracts a feature point as the nonflatness from the background image,

wherein the second determination unit further determines whether each of feature points extracted from a plurality of image regions of the background image by the second nonflatness calculating unit is higher than a third threshold value, and

wherein the first image capturing unit captures the subject existing image when the second determination unit determines that each of the feature points is not higher than the third threshold value.

7. The image capturing apparatus according to claim 5,

wherein the second nonflatness calculating unit calculates dispersion values of pixel values in a plurality of image regions of the background image,

wherein the second determination unit further determines whether each of the dispersion values of the pixel values in the image regions of the background image calculated by the second nonflatness calculating unit is higher than a third threshold value, and

wherein the first image capturing unit further captures the subject existing image when the second determination unit determines that each of the dispersion values of the pixel values in the image regions is not higher than the third threshold value.

8. The image capturing apparatus according to claim 5, further comprising:

a second image capturing unit that captures a subject nonexisting image where the subject is absent in a same background as the background of the subject existing image after the subject existing image is captured when the second determination unit determines that the nonflatness is higher than the second threshold value; and

an alignment unit that performs alignment between the subject existing image and the subject nonexisting image,

wherein the subject extracting unit extracts the subject region from the subject existing image based on difference information of each corresponding pixel between the subject existing image and the subject nonexisting image aligned with each other by the alignment unit.

9. A method for processing an image by an image capturing apparatus comprising an image capturing unit that captures a subject existing image in which a subject is present in a background, the method comprising:

dividing a peripheral portion of the subject existing image into a plurality of image regions;

calculating nonflatness by extracting a feature point from each of the image regions;

determining whether each of the extracted feature points of the image regions is higher than a threshold value;

generating an extracting background image from the peripheral portion of the subject existing image when it is determined that each of the feature points is not higher than the threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image; and

extracting the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image.

10. A non-transitory computer readable storage medium having a program stored thereon for controlling a computer of an image capturing apparatus, wherein the image capturing apparatus comprises an image capturing unit that captures a subject existing image in which a subject is present in a background, and wherein the program controls the computer to perform image processing functions comprising:

dividing a peripheral portion of the subject existing image into a plurality of image regions;

calculating nonflatness by extracting a feature point from each of the image regions;

determining whether each of the extracted feature points of the image regions is higher than a threshold value;

generating an extracting background image from the peripheral portion of the subject existing image when it is determined that each of the feature points is not higher than the threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image; and

extracting the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image.

11. A method for processing an image by an image capturing apparatus comprising an image capturing unit that captures a subject existing image in which a subject is present in a background, the method comprising:

calculating nonflatness of a peripheral portion of the subject existing image captured by the first image capturing unit;

determining whether the nonflatness of the peripheral portion of the subject existing image is higher than a first threshold value;

generating an extracting background image from the peripheral portion of the subject existing image when it is determined that the nonflatness is not higher than the first threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image;

extracting the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image;

capturing a background image in which the subject is absent and generating image frames of the background image sequentially;

calculating nonflatness of each of the image frames of the background image;

determining whether the nonflatness of each of the image frames of the background image is higher than a second threshold value; and

capturing a subject existing image in which the subject is present in a substantially same background as the background image when it is determined that the nonflatness of each of the image frames of the background image is not higher than the second threshold value.

12. A non-transitory computer readable storage medium having a program stored thereon for controlling a computer of an image capturing apparatus, wherein the image capturing apparatus comprises an image capturing unit that captures a subject existing image in which a subject is present in a background, and wherein the program controls the computer to perform image processing functions comprising:

calculating nonflatness of a peripheral portion of the subject existing image captured by the first image capturing unit;

determining whether the nonflatness of the peripheral portion of the subject existing image is higher than a first threshold value;

generating an extracting background image from the peripheral portion of the subject existing image when it is determined that the nonflatness is not higher than the first threshold value, the extracting background image serving to extract a subject region containing the subject in the subject existing image;

extracting the subject region from the subject existing image based on difference information of each corresponding pixel between the extracting background image and the subject existing image;

capturing a background image in which the subject is absent and generating image frames of the background image sequentially;

calculating nonflatness of each of the image frames of the background image;

determining whether the nonflatness of each of the image frames of the background image is higher than a second threshold value; and

capturing a subject existing image in which the subject is present in a substantially same background as the background image when it is determined that the nonflatness of each of the image frames of the background image is not higher than the second threshold value.