Estimating and removing distortion from an image

• US 8,116,586 B2
• Filed: 01/31/2011
• Issued: 02/14/2012
• Est. Priority Date: 10/29/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Souchard, Christophe
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US13/018,418
• Publication Number:
  US 20110122149A1
• Time in Patent Office: 379 Days
• Field of Search: 382/100, 382/254, 382/255, 382260-266, 382/274, 382/275, 345/427, 345/428, 345/581, 345/611, 345/619, 345/647, 345/650, 345/651, 345/652, 345/655, 345/661, 348/207.99, 348/208.4, 348/208.13, 348/241, 348/252, 348/333.12
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20096 Interactive definition of curve of interest
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

Some embodiments provide a method of editing images with distortion caused by a camera lens. The method identifies a set of geometries on a set of images taken by the camera lens. The method finds transformations for applying on the set of identified geometries to remove distortion on the geometries. The method applies the transformations to images taken by the camera lens to remove distortions. In some embodiments the identified geometries are curvilinear lines that correspond to straight lines in the scene being captured by the lens. Some embodiments provide a method of matching distortion for adding graphical objects to images. The method first removes the distortion on the images as described above. The method then adds the graphical objects to these images. Finally, the method applies inverse transformations on the images to restore them to their original form.

Claims

1. A non-transitory computer readable medium storing a computer program that when executed by at least one processor implements a graphical user interface (GUI) for removing distortion from an image, the GUI comprising:

a display area for displaying the image; and

a distortion removal tool for (i) allowing a user of the GUI to identify, in the image, a first curvilinear line representing the distortion that should appear straight by drawing, on the image, a second curvilinear line over or near the first curvilinear line, (ii) determining a set of transformations based on the identification, and (iii) applying the set of transformations to the image in order to remove the distortion from the image.

2. The non-transitory computer readable medium of claim 1, wherein the distortion removal tool includes a spline tool for drawing the second curvilinear line to identify the first curvilinear line.

3. The non-transitory computer readable medium of claim 1, wherein the display area is further for displaying a version of the image with the distortion removed and the second curvilinear line straightened.

4. The non-transitory computer readable medium of claim 1, wherein the distortion removal tool allows the user to identify the first curvilinear line by specifying a plurality of points on the image.

5. The non-transitory computer readable medium of claim 4, wherein the distortion removal tool allows the user to specify the plurality of points by adding, deleting, or moving a set of points.

6. The non-transitory computer readable medium of claim 1, wherein the distortion removal tool (i) allows the user to identify at least one other curvilinear line that represents the distortion, and (ii) determines the set of transformations based on the identification of the curvilinear lines.

7. The non-transitory computer readable medium of claim 1, wherein the first curvilinear line represents a distortion caused by a lens of a camera.

8. A non-transitory computer readable medium storing a computer program that when executed by at least one processor implements a graphical user interface (GUI) for removing distortion from an image, the GUI comprising:

a display area for displaying the image; and

a distortion removal tool for (i) allowing a user of the GUI to identify, in the image, a curvilinear line representing the distortion that should appear straight, (ii) determining a set of transformations based on the identification, (iii) applying the set of transformations to the image in order to remove the distortion from the image, and (iv) distorting a set of graphical objects added to an undistorted version of the image.

9. The non-transitory computer readable medium of claim 8, wherein the distortion removal tool distorts the set of graphical objects by applying an inverse of at least one transformation in the set of transformations.

10. A method of removing distortion from an image, the method comprising:

displaying the image having the distortion;

identifying, in the image, a first curvilinear line representing the distortion that should appear straight by drawing, on the image, a second curvilinear line over or near the first curvilinear line;

determining a set of transformations based on the identification; and

applying the set of transformations to the image in order to remove the distortion from the image.

11. The method of claim 10, wherein drawing the second curvilinear line comprises specifying a plurality of points by adding, deleting, or moving a set of points on the image.

12. The method of claim 10, wherein drawing the second curvilinear line comprises using a spline tool to draw the second curvilinear line over or near the first curvilinear line.

13. A method of removing distortion from an image, the method comprising:

displaying the image having the distortion;

identifying, in the image, a curvilinear line representing the distortion that should appear straight;

determining a set of transformations based on the identification;

applying the set of transformations to the image in order to remove the distortion from the image;

adding a set of graphical objects to an undistorted version of said image; and

distorting the set of graphical objects added to the undistorted version of said image.

14. The method of claim 13, wherein distorting the set of graphical objects comprises applying an inverse of at least one transformation in the set of transformations.

15. A non-transitory computer readable medium storing a computer program for execution by at least one processor, said computer program comprising sets of instructions for:

receiving identification of a distortion in a first image;

identifying a transformation that represents the distortion;

adding a set of graphical objects to at least a second image; and

distorting, based on the transformation, the set of graphical objects added to the second image in accord with the distortion in the first image.

16. The non-transitory computer readable medium of claim 15, wherein the second image is an undistorted version of the first image.

17. The non-transitory computer readable medium of claim 15, wherein the computer program further comprises a set of instructions for adding the distorted set of graphical objects to the first image.

18. The non-transitory computer readable medium of claim 15, wherein the set of instructions for receiving identification of the distortion comprises a set of instructions for receiving a set of inputs using a spline tool to draw on the first image.

19. The non-transitory computer readable medium of claim 15, wherein the distortion is a lens distortion caused with a camera lens.

20. A non-transitory computer readable medium storing a computer program that when executed by at least one processor removes distortion from an image, the computer program comprising sets of instructions for:

displaying the image having the distortion;

identifying, in the image, a curvilinear line representing the distortion that should appear straight;

determining a set of transformations based on the identification;

applying the set of transformations to the image in order to remove the distortion from the image;

adding a set of graphical objects to an undistorted version of said image; and

distorting the set of graphical objects added to the undistorted version of the image.

21. The non-transitory computer readable medium of claim 20, wherein the set of instructions for distorting the set of graphical objects comprises a set of instructions for applying an inverse of at least one transformation in the set of transformations.

22. The non-transitory computer readable medium of claim 20, wherein the distortion of the image is a lens distortion caused with a camera lens.

23. A non-transitory computer readable medium storing a computer program that when executed by at least one processor removes distortion from an image, the computer program comprising sets of instructions for:

displaying the image having the distortion;

identifying, in the image, a first curvilinear line representing the distortion that should appear straight by drawing, on the image, a second curvilinear line over or near the first curvilinear line;

determining a set of transformations based on the identification; and

applying the set of transformations to the image in order to remove the distortion from the image.

24. The non-transitory computer readable medium of claim 23, wherein the set of instructions for drawing the second curvilinear line comprises a set of instructions for using a distortion removal tool that includes a spline tool to draw the second curvilinear line over or near the first curvilinear line.

25. The non-transitory computer readable medium of claim 23, wherein the set of instructions for drawing the second curvilinear line comprises a set of instructions for specifying a plurality of points by adding, deleting, or moving a set of points on the image.

26. A method of distorting objects added to an image, the method comprising:

receiving identification of a distortion in a first image;

identifying a transformation that represents the distortion;

adding a set of graphical objects to at least a second image; and

distorting, based on the transformation, the set of graphical objects added to the second image in accord with the distortion in the first image.

27. The method of claim 26, wherein distorting the set of graphical objects comprises applying an inverse of the transformation to the set of graphical objects.

28. The method of claim 26, wherein the second image is an undistorted version of the first image, the method further comprising adding the distorted set of graphical objects to the first image.

29. The method of claim 26, wherein the set of graphical objects comprises at least one of a text object, an image object, and an icon object.