IMAGE PROCESSING METHOD AND APPARATUS, DIGITAL CAMERA, AND RECORDING MEDIUM RECORDING IMAGE PROCESSING PROGRAM

• US 20090225180A1
• Filed: 08/09/2006
• Published: 09/10/2009
• Est. Priority Date: 08/25/2005
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Aoki, Shin, Maruyama, Go
• Granted Patent: US 8,120,665 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A technology for recognizing one or more quadrangles from an input image is disclosed. Edge areas are detected from the input image, lines corresponding to the edge areas are extracted, a line pair selected from the extracted lines is categorized according to a positional relationship between two lines included in the line pair, a line pair evaluation value is calculated for the line pair, a combination of two line pairs is selected, a quadrangle is generated from four lines included in the two line pairs selected, a quadrangle evaluation value is calculated for the quadrangle based on the categories and the line pair evaluation values of the two line pairs forming the quadrangle, and a quadrangle is selected based on the calculated quadrangle evaluation value.

Claims

1. An image processing method for recognizing one or more quadrangles from an input image, the method comprising the steps of:

(a) detecting a plurality of edge areas from the input image;

(b) extracting a plurality of lines;

the lines corresponding to the edge areas detected;

(c) categorizing a line pair selected from the extracted lines according to a positional relationship between two lines included in the line pair, and calculating a line pair evaluation value for the line pair;

(d) selecting a combination of two line pairs from a plurality of line pairs, generating a quadrangle from four lines included in the two line pairs selected, and calculating a quadrangle evaluation value for the quadrangle based on the categories and the line pair evaluation values of the two line pairs forming the quadrangle; and

(e) selecting a quadrangle based on the calculated quadrangle evaluation value.

2. The image processing method according to claim 1, further comprising a step of:

(f) selecting a plurality of line pairs from the lines extracted at the step (b), generating a new line according to a positional relationship between two lines included in a line pair among the selected line pairs, and adding the new line to the lines extracted at the step (b);

wherein

the step (c) includes selecting all line pairs from the lines extracted at the step (b) and the new line generated at the step (f).

3. The image processing method according to claim 1, wherein the step (a) includes

calculating a vertical luminance variation amount and a horizontal luminance variation amount for each pixel of the input image,

detecting the edge areas from the input image based on the calculated vertical luminance variation amounts and horizontal luminance variation amounts,

dividing a two-dimensional space formed by the vertical luminance variation amounts and the horizontal luminance variation amounts by direction, and

assigning the edge areas to a plurality of direction groups corresponding to the directions.

4. The image processing method according to claim 3, wherein the step (b) includes extracting lines by performing principal component analysis on pixel information of each edge area for each of the direction groups.

5. The image processing method according to claim 4, wherein the step (b) includes searching each edge area in a principal component direction, integrating a plurality of edge areas which edge areas are adjacent to each other by a predetermined number of pixels to form an integrated edge area, removing the original edge areas used for forming the integrated edge area, and extracting a line corresponding to the integrated edge area.

6. The image processing method according to claim 1, wherein the step (c) includes categorizing a line pair as opposite, adjacent, or unrelated according to a positional relationship between two lines included in the line pair, and

the step (d) includes generating a quadrangle from line pairs categorized as opposite and adjacent.

7. The image processing method according to claim 1, wherein the step (d) includes obtaining an area of a quadrangle, and weighting the quadrangle evaluation value based on the area.

8. The image processing method according to claim 1, wherein the step (d) includes obtaining a projection transformation matrix to transform a quadrangle into a parallelogram, and weighting the quadrangle evaluation value based on a shape of the parallelogram obtained by performing projection transformation on the quadrangle.

9. The image processing method according to claim 1, further comprising a step of:

(g) calculating a projection transformation matrix from a quadrangle selected at the step (e), and performing projection transformation on the input image.

20. A program product that causes a computer to execute the steps of the image processing method according to claim 1.

10. An image processing apparatus for recognizing one or more quadrangles from an input image, the apparatus comprising:

an edge area detecting unit configured to detect a plurality of edge areas from the input image;

a line extracting unit configured to extract a plurality of lines;

the lines corresponding to the edge areas detected;

a line pair categorization/evaluation unit configured to select a line pair from the extracted lines, categorize the line pair according to a positional relationship between two lines included in the line pair, and calculate a line pair evaluation value for the line pair;

a quadrangle evaluation unit configured to select a combination of two line pairs from a plurality of line pairs, generate a quadrangle from four lines included in the two line pairs selected, and calculate a quadrangle evaluation value for the quadrangle based on the categories and the line pair evaluation values of the two line pairs forming the quadrangle; and

a quadrangle selecting unit configured to select a quadrangle based on the calculated quadrangle evaluation value.

11. The image processing apparatus according to claim 10, further comprising:

a line generating unit configured to select a plurality of line pairs from the lines extracted by the line extracting unit, generate a new line according to a positional relationship between two lines included in a line pair among the selected line pairs, and add the new line to the lines extracted by the line extracting unit;

wherein

the line pair categorization/evaluation unit selects all line pairs from the lines extracted by the line extracting unit and the new line generated by the line generating unit.

12. The image processing apparatus according to claim 10, wherein the edge area detecting unit

calculates a vertical luminance variation amount and a horizontal luminance variation amount for each pixel of the input image,

detects the edge areas from the input image based on the calculated vertical luminance variation amounts and horizontal luminance variation amounts,

divides a two-dimensional space formed by the vertical luminance variation amounts and the horizontal luminance variation amounts by direction, and

assigns the edge areas to a plurality of direction groups corresponding to the directions.

19. A digital camera including functions of the image processing apparatus according to claim 10.

13-18. -18. (canceled)

21. An image processing method comprising the steps of:

extracting a plurality of shapes from an input image;

creating a plurality of highlighted shape area images corresponding to the shapes extracted at the extracting step by superposing the shapes in the input image and highlighting an image area of each shape; and

controlling display of the highlighted shape area images.

22. The image processing method according to claim 21, wherein

the creating step includes creating the highlighted shape area images by performing different processes on one or more different areas of the input image, the areas being an inside part of the shape, an outside part of the shape, and a perimeter part of the shape.

23. The image processing method according to claim 21, wherein

the creating step includes determining one of the extracted shapes as a target shape, and creating a target highlighted shape area image corresponding the target shape so as to be distinguishable from highlighted shape area images corresponding to shapes other than the target shape.

29. The image processing method according to claim 21, further comprising a step of:

calculating a projection transformation matrix based on shape data corresponding to a highlighted shape area image, and performing projection transformation on the input image based on the projection transformation matrix.

40. A program product that causes a computer to execute the steps of the image processing method according to claim 21.

24-28. -28. (canceled)

30-39. -39. (canceled)