Image processing apparatus, image processing method, and computer program

• US 20070133900A1
• Filed: 12/04/2006
• Published: 06/14/2007
• Est. Priority Date: 12/08/2005
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Nielsen, Frank, Hasegawa, Yuichi
• Granted Patent: US 8,320,672 B2
• Time in Patent Office: Days
• US Class Current: 382/282
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 1/3872 Repositioning or masking

Abstract

An image processing apparatus automatically determining a composition of an image includes area dividing means for dividing an input image into areas; target area determining means for determining a target area having higher visibility from the divided image; unnecessary area determining means for determining an unnecessary area having lower visibility from the divided image; and composition determining means for determining from the input image a composition that includes the target area and does not include the unnecessary area.

Claims

1. An image processing apparatus automatically determining a composition of an image, the image processing apparatus comprising:

area dividing means for dividing an input image into areas;

target area determining means for determining a target area having higher visibility from the divided image;

unnecessary area determining means for determining an unnecessary area having lower visibility from the divided image; and

composition determining means for determining from the input image a composition that includes the target area and does not include the unnecessary area.

2. The image processing apparatus according to claim 1,

wherein the target area determining means performs face detection to the input image to determine the target area on the basis of an area where a face is detected.

3. The image processing apparatus according to claim 1, further comprising target object information holding means for holding information concerning a target object,

wherein the target area determining means performs matching between the input image and the information concerning the target object to determine an area having a higher degree of matching to be the target area.

4. The image processing apparatus according to claim 1,

wherein the target area determining means determines an area having higher attractiveness to be the target area, the attractiveness being evaluated from a physical characteristic, such as a color, shape, area (the ratio in area of each object to the input image), or texture, of each object included in the input image.

5. The image processing apparatus according to claim 1,

wherein the target area determining means and the unnecessary area determining means determine the target area and the unnecessary area, respectively, on the basis of a degree of focusing yielded for every area.

6. The image processing apparatus according to claim 1, further comprising:

edge image extracting means for extracting an edge of each divided image area in the input image; and

amount-of-edge calculating means for calculating a sum of pixel values in the edge image for every divided image area to yield an amount of edge resulting from normalization of the sum of the pixel values with a length or an area of the edge as a value indicating a degree of focusing.

7. The image processing apparatus according to claim 6,

wherein the unnecessary area determining means determines a peripheral area having a smaller amount of edge in the input image to be the unnecessary area.

8. The image processing apparatus according to claim 1, further comprising:

image display means for displaying the input image and the image divided into the areas by the area dividing means; and

user input means for receiving a manual input by a user on the displayed image,

wherein at least one of the target area and the unnecessary area is determined in accordance with the manual selection by the user with the user input means after the area division.

9. The image processing apparatus according to claim 1,

wherein the composition determining means determines a maximum rectangular area in the input image, which area does not include the unnecessary area and has a predetermined aspect ratio, to be the composition.

10. The image processing apparatus according to claim 1,

wherein the area dividing means gives weight values based on a result of comparison between attribute values of the image areas to edges corresponding to sides that touch the image areas, sorts the edges on the basis of the weight values, and determines whether pairs of the image areas sandwiching the edges are to be integrated on the basis of a statistical algorithm in the sorting order.

11. The image processing apparatus according to claim 10,

wherein the area dividing means gives the weight values based on differences in the attribute values between the image areas to the edges sandwiched between the image areas and sorts the edges in ascending order of the weight values.

12. The image processing apparatus according to claim 11,

wherein the area dividing means uses pixel attribute information, such as areas or colors of the image areas, as the attribute values of the image areas.

13. The image processing apparatus according to claim 10,

wherein the area dividing means determines whether the image areas connected via the edges are to be integrated on the basis of a predicate derived from concentration inequality in the areas of polygons included in the image areas.

14. An image processing method automatically determining a composition of an image, the image processing method comprising the steps of:

dividing an input image into areas;

determining a target area having higher visibility from the divided image;

determining an unnecessary area having lower visibility from the divided image; and

determining from the input image a composition that includes the target area and does not include the unnecessary area.

15. The image processing method according to claim 14,

wherein the target area determining step performs face detection to the input image to determine the target area on the basis of an area where a face is detected.

16. The image processing method according to claim 14, further comprising the step of acquiring information concerning a target object,

wherein the target area determining step performs matching between the input image and the information concerning the target object to determine an area having a higher degree of matching to be the target area.

17. The image processing method according to claim 14,

wherein the target area determining step determines an area having higher attractiveness to be the target area, the attractiveness being evaluated from a physical characteristic, such as a color, shape, area (the ratio in area of each object to the input image), or texture, of each object included in the input image.

18. The image processing method according to claim 14,

wherein the target area determining step and the unnecessary area determining step determine the target area and the unnecessary area, respectively, on the basis of a degree of focusing yielded for every area.

19. The image processing method according to claim 14, further comprising the steps of:

extracting an edge of each divided image area in the input image; and

calculating a sum of pixel values in the edge image for every divided image area to yield an amount of edge resulting from normalization of the sum of the pixel values with a length or an area of the edge as a value indicating a degree of focusing.

20. The image processing method according to claim 19,

wherein the unnecessary area determining step determines a peripheral area having a smaller amount of edge in the input image to be the unnecessary area.

21. The image processing method according to claim 14, further comprising the steps of:

displaying the input image and the image divided into the areas in the area dividing step; and

receiving a manual input by a user on the displayed image,

wherein at least one of the target area and the unnecessary area is determined in accordance with the manual selection by the user in the user input step after the area division.

22. The image processing method according to claim 14,

wherein the composition determining step determines a maximum rectangular area in the input image, which area does not include the unnecessary area and has a predetermined aspect ratio, to be the composition.

23. The image processing method according to claim 14,

wherein the area dividing step gives weight values based on a result of comparison between attribute values of the image areas to edges corresponding to sides that touch the image areas, sorts the edges on the basis of the weight values, and determines whether pairs of the image areas sandwiching the edges are to be integrated on the basis of a statistical algorithm in the sorting order.

24. The image processing method according to claim 23,

wherein the area dividing step gives the weight values based on differences in the attribute values between the image areas to the edges sandwiched between the image areas and sorts the edges in ascending order of the weight values.

25. The image processing method according to claim 24,

wherein the area dividing step uses pixel attribute information, such as areas or colors of the image areas, as the attribute values of the image areas.

26. The image processing method according to claim 23,

wherein the area dividing step determines whether the image areas connected via the edges are to be integrated on the basis of a predicate derived from concentration inequality in the areas of polygons included in the image areas.

27. A computer-readable program causing a computer system to execute a process of automatically determining a composition of an image, the program comprising the steps of:

dividing an input image into areas;

determining a target area having higher visibility from the divided image;

determining an unnecessary area having lower visibility from the divided image; and

determining from the input image a composition that includes the target area and does not include the unnecessary area.

28. An image processing apparatus automatically determining a composition of an image, the image processing apparatus comprising:

an area dividing unit dividing an input image into areas;

a target area determining unit determining a target area having higher visibility from the divided image;

an unnecessary area determining unit determining an unnecessary area having lower visibility from the divided image; and

a composition determining unit determining from the input image a composition that includes the target area and does not include the unnecessary area.