Locating and aligning embedded images

• US 6,298,157 B1
• Filed: 02/27/1998
• Issued: 10/02/2001
• Est. Priority Date: 02/27/1998
• Status: Expired due to Term

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Wilensky, Gregg D.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): AZARIAN, SEYED H
• Application Number:
  US09/032,718
• Time in Patent Office: 1,313 Days
• Field of Search: 382/173, 382/181, 382/182, 382/183, 382/176, 382/184, 382/185, 382/190, 382/193, 382/194, 382/195, 382/196, 382/107, 382/198, 382/202, 382/199, 382/200, 382/209, 382/210, 382/203, 382/211, 395/163, 395/164, 235/454, 235/462, 235/472, 235/439
• US Class Current: 382/199
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 1/3872 Repositioning or masking
• CPC Class Codes: H04N 1/3878 Skew detection or correction

Abstract

Apparatus and methods useful to automate the task of locating a photograph or other image that is embedded within a larger image. An edge curve is found that approximates the perimeter of the embedded image and from the edge curve the rectangle of the embedded image, or a rectangle covering the embedded image, is calculated by processing density profiles of the edge curve taken along two axes. Both location and orientation of an embedded image can be calculated. The location of the four corners of a rectangular embedded image can be calculated, which enables automatic cropping and rotation of the embedded image, even if fewer than all four corners are visible. A rectangle covering the image can be calculated, including a rectangle aligned with the axes of a larger embedding image such as is created when scanning a small photograph on a flatbed scanner. The invention operates well on rectangular images, as well as on images having polygonal and other border shapes, such as circles and ellipses, including images with irregular shapes.

Claims

1. An apparatus for finding an image embedded in a larger image, the apparatus comprising:

edge detecting means for detecting, within the larger image, an edge of the embedded image, the edge including one or more edge points;

edge curve finding means for finding an edge curve from the detected edge of the embedded image, wherein the edge curve finding means compares an intensity value of an edge point of the detected edge to a threshold intensity level and designates edge points having an intensity value exceeding the threshold intensity level as being included in the edge curve, the edge curve defining visible perimeter portions of the embedded image and excluding substantially all interior portions of the embedded image;

rotation angle calculating means for calculating a rotation angle of the embedded image from the edge curve; and

line segment calculating means for calculating from the edge curve a location and orientation of one or more line segments of a side of a rectangle bounding the embedded image in the larger image.

2. The apparatus of claim 1, wherein the embedded image is an image of a rectangular photograph, the apparatus further comprising:

edge segment locating means for locating edge segments in the edge curve corresponding to sides of the embedded image in the larger image.

3. The apparatus of claim 1, further comprising:

covering rectangle locating means for locating corners of a covering rectangle of the embedded image.

4. The apparatus of claim 3, further comprising:

cropping means for cropping the embedded image in a frame of reference of the larger image.

5. The apparatus of claim 3, further comprising:

edge point detecting means for detecting edge points.

6. The apparatus of claim 5, further comprising:

noise removing means for removing pixel noise from the larger image.

13. The apparatus of claim 3, further comprising:

density profile calculating means for calculating density profiles on two axes of the larger image.

14. The apparatus of claim 1, wherein the embedded image has a non-rectangular border shape.

7. A method of processing a digital raster image embedded in a larger digital raster image, comprising:

detecting, within the larger image, an edge of the embedded image, the edge including one or more edge points;

finding an edge curve from the detected edge of the embedded image by comparing an intensity value of an edge point of the detected edge to a threshold intensity level and designating edge points having an intensity value exceeding the threshold intensity level as being included in the edge curve, the edge curve defining visible perimeter portions of the embedded image and excluding substantially all interior portions of the embedded image;

calculating a rotation angle of the embedded image from the edge curve; and

calculating from the edge curve a location and orientation of a line segment of a side of a rectangle bounding the embedded image in the larger image.

8. The method of claim 7, further comprising:

finding the edge curve by;

calculating the positions of edge points in the larger image; and

scanning from the perimeter into the larger image from starting points on all sides of the larger image and selecting as a point in the edge curve the first edge point encounter from each starting point;

calculating a rotation angle by calculating an average angle of the edge curve points over a windowed set of neighboring edge curve points;

rotating the edge curve by the complement of the calculated rotation angle; and

calculating the location of corners of a rectangle covering the embedded image.

9. The method of claim 8, wherein:

the two axes are perpendicular; and

calculating the positions of edge points comprises applying a Sobel filter to the larger image.

10. The method of claim 8, wherein:

the two axes are perpendicular; and

calculating the positions of edge points comprises applying a Hough transform to the larger image.

11. The method of claim 8, further comprising:

cropping the embedded image according the covering rectangle.

12. The method of claim 7, further comprising:

removing pixel noise from the larger image before performing the step of finding an edge curve;

finding the edge curve by

calculating density profiles of the larger image on two axes;

calculating the positions of edge points in the larger image; and

scanning from the perimeter into the larger image from starting points on all sides of the larger image and selecting as a point in the edge curve the first edge point encounter from each starting point;

calculating a rotation angle by calculating an average angle of the edge curve points over a windowed set of neighboring edge curve points;

rotating the edge curve by the complement of the calculated rotation angle;

removing noise points from the edge curve; and

calculating the location of corners of a rectangle covering the embedded image.

15. The method of claim 7, wherein:

the larger image is a digital raster image generated by scanning an analog input image.

16. The method of claim 8, further comprising:

performing a scan at a first resolution of a first area to obtain the larger digital raster image; and

using the bounding rectangle to define a second area smaller than, and within, the first area, and performing a scan of the second area at a second resolution higher than the first resolution to obtain a raster image including the embedded digital raster image at the second resolution.

17. The method of claim 12, further comprising:

performing a scan at a first resolution of a first area to obtain the larger digital raster image; and

using the bounding rectangle to define a second area smaller than, and within, the first area, and performing a scan of the second area at a second resolution higher than the first resolution to obtain a raster image including the embedded digital raster image at the second resolution.

18. The method of claim 17, wherein the scan of the second area is limited to the second area.

19. The method of claim 7, further comprising:

performing a scan at a first resolution of a first area to obtain the larger digital raster image; and

using the edge curve to define a second area smaller than, and within, the first area, and performing a scan of the second area at a second resolution higher than the first resolution to obtain a raster image including the embedded digital raster image at the second resolution.