Image processing apparatus, image capturing apparatus, image processing method, and storage medium

• US 10,115,178 B2
• Filed: 12/23/2013
• Issued: 10/30/2018
• Est. Priority Date: 12/26/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tamura, Yusuke, Kodama, Yasunobu
• Primary Examiner(s): Patel, Jitesh
• Application Number:
  US14/138,813
• Publication Number:
  US 20140176612A1
• Time in Patent Office: 1,772 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2210/22 Cropping
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/7921 for more than one processing mode

Abstract

Provided is an image processing apparatus that determines crop positions for an image including a plurality of objects in a preferred manner. This image processing apparatus specifies object regions from the image, and sets a plurality of crop region candidates for each of the specified object regions. The image processing apparatus selects a predetermined number of crop regions from among the plurality of crop region candidates based on evaluation values obtained for the plurality of crop region candidates and on similarities among the plurality of crop region candidates.

Claims

1. An image processing apparatus comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to;

specify object regions from an image;

set a plurality of crop region candidates for each of the specified object regions;

obtain evaluation values of the set plurality of crop region candidates;

select a predetermined number of crop regions from among the plurality of crop region candidates, wherein a crop region candidate having the highest evaluation value among the plurality of crop region candidates is selected as one crop region, and remaining crop regions are selected after selecting the one crop region; and

generate, from the image, crop images corresponding to each of the predetermined number of crop regions,

wherein in the selecting of the remaining crop regions, the instructions, when executed by the processor, cause the processor to repeat the following processing until the predetermined number of crop regions are selected;

choose a crop region candidate having the next highest evaluation value from among the plurality of crop region candidates;

perform similarity determination processing which determines whether the chosen crop region candidate including at least a portion of a specific object region has a similarity with any of the already selected crop regions including at least a portion of the specific object region;

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region has a similarity with any of the already selected crop regions including at least a portion of the specific object region, omit the chosen crop region candidate from the crop region candidates; and

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region does not have a similarity with the already selected crop regions including at least a portion of the specific object region, select the chosen crop region candidate as a crop region,

wherein the crop images corresponding to each of the selected crop regions are displayed on a display device.

2. The image processing apparatus according to claim 1, wherein the similarity determination processing is performed based on a difference between an aspect ratio of the chosen crop region candidate including at least a portion of the specific object region and an aspect ratio of each of the already selected crop regions including at least a portion of the specific object region.

3. The image processing apparatus according to claim 1, wherein the similarity determination processing is performed based on a difference between a size of the chosen crop region candidate including at least a portion of the specific object region and a size of each of the already selected crop regions including at least a portion of the specific object region.

4. The image processing apparatus according to claim 1, wherein the similarity determination processing is performed based on a difference between region start coordinates of the chosen crop region candidate including at least a portion of the specific object region and region start coordinates of each of the already selected crop regions including at least a portion of the specific object region.

5. The image processing apparatus according to claim 1, wherein the instructions, when executed by the processor, further cause the processor to:

obtain the evaluation values of the set plurality of crop region candidates based on areas and distances from an image center.

6. The image processing apparatus according to claim 1, wherein the instructions, when executed by the processor, further cause the processor to:

extract, from the image, images of regions defined by the selected predetermined number of crop regions, and output the extracted images.

7. An image capturing apparatus comprising:

an image sensor which captures an image; and

an image processing apparatus comprising;

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to;

specify object regions from the image,

set a plurality of crop region candidates for each of the specified object regions,

obtain evaluation values of the set plurality of crop region candidates,

select a predetermined number of crop regions from among the plurality of crop region candidates, wherein a crop region candidate having the highest evaluation value among the plurality of crop region candidates is selected as one crop region, and remaining crop regions are selected after selecting the one crop region, and

generate, from the image, crop images corresponding to each of the predetermined number of crop regions,

wherein in the selecting of the remaining crop regions, the instructions, when executed by the processor, cause the processor to repeat the following processing until the predetermined number of crop regions are selected;

choose a crop region candidate having the next highest evaluation value from among the plurality of crop region candidates;

perform similarity determination processing which determines whether the chosen crop region candidate including at least a portion of a specific object region has a similarity with any of the already selected crop regions including at least a portion of the specific object region;

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region has a similarity with any of the already selected crop region including at least a portion of the specific object region, omit the chosen crop region candidate from the crop region candidates; and

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region does not have a similarity with the already selected crop regions including at least a portion of the specific object region, select the chosen crop region candidate as a crop region,

wherein the crop images corresponding to each of the selected crop regions are displayed on a display device.

8. An image processing method comprising:

specifying object regions from an image;

setting a plurality of crop region candidates for each of the specified object regions;

obtaining evaluation values of the set plurality of crop region candidates;

selecting a predetermined number of crop regions from among the plurality of crop region candidates, wherein a crop region candidate having the highest evaluation value among the plurality of crop region candidates is selected as one crop region, and the remaining crop regions are selected after selecting the one crop region; and

generating, from the image, crop images corresponding to each of the predetermined number of crop regions,

wherein in the selecting of the remaining crop regions, the method comprises repeating the following processing until the predetermined number of crop regions are selected;

choosing a crop region candidate having the next highest evaluation value from among the plurality of crop region candidates;

performing similarity determination processing which determines whether the chosen crop region candidate including at least a portion of a specific object region has a similarity with any of the already selected crop regions including at least a portion of the specific object region;

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region has a similarity with any of the already selected crop region, omitting the chosen crop region candidate from the crop region candidates including at least a portion of the specific object region; and

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region does not have a similarity with the already selected crop regions, selecting the chosen crop region candidate as a crop region including at least a portion of the specific object region,

wherein the crop images corresponding to each of the selected crop regions are displayed on a display device.

9. A non-transitory computer-readable storage medium having stored thereon a program that, when executed by a computer, causes the computer to:

specify object regions from an image;

set a plurality of crop region candidates for each of the specified object regions;

obtain evaluation values of the set plurality of crop region candidates;

select a predetermined number of crop regions from among the plurality of crop region candidates, wherein a crop region candidate having the highest evaluation value among the plurality of crop region candidates is selected as one crop region, and the remaining crop regions are selected after selecting the one crop region; and

generate, from the image, crop images corresponding to each of the predetermined number of crop regions,

wherein in the selecting of the remaining crop regions, the program, when executed by the computer, causes the computer to repeat the following processing until the predetermined number of crop regions are selected;

choose a crop region candidate having the next highest evaluation value from among the plurality of crop region candidates;

perform similarity determination processing which determines whether the chosen crop region candidate including at least a portion of a specific object region has a similarity with any of the already selected crop regions including at least a portion of the specific object region;

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region has a similarity with any of the already selected crop region including at least a portion of the specific object region, omit the chosen crop region candidate from the crop region candidates; and

in a case where it is determined, in the similarity determination processing, that the chosen crop region candidate including at least a portion of the specific object region does not have a similarity with the already selected crop regions, select the chosen crop region candidate as a crop region including at least a portion of the specific object region,

wherein the crop images corresponding to each of the selected crop regions are displayed on a display device.