Estimating and removing lens distortion from scenes

• US 20070098296A1
• Filed: 10/29/2005
• Published: 05/03/2007
• Est. Priority Date: 10/29/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Souchard, Christophe
• Granted Patent: US 7,912,317 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20096 Interactive definition of curve of interest
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

Some embodiments provide a method of editing images with distortion caused by a camera lens. The method identifies a set of geometries on a set of images taken by the camera lens. The method finds transformations for applying on the set of identified geometries to remove distortion on the geometries. The method applies the transformations to images taken by the camera lens to remove distortions. In some embodiments the identified geometries are curvilinear lines that correspond to straight lines in the scene being captured by the lens. Some embodiments provide a method of matching distortion for adding graphical objects to images. The method first removes the distortion on the images as described above. The method then adds the graphical objects to these images. Finally, the method applies inverse transformations on the images to restore them to their original form.

Claims

1. A method of editing images with distortion caused by a camera lens, the method comprising:

a) identifying a set of geometries on a first set of images taken by the camera lens;

b) finding transformations for applying on the set of geometries to remove distortion on the set of geometries;

c) applying the transformations to a second set of images to remove distortions caused by the camera lens.

2. The method of claim 1, wherein the first set of images and the second set of images are a sequence of video images taken by said camera lens.

3. The method of claim 1, wherein the first set of images comprises of at least two images.

4. The method of claim 1, wherein the second set of images comprises of at least two images.

5. The method of claim 1, wherein the first set of images and the second set of images include at least one common image.

6. The method of claim 1, wherein the first set of images and the second set of images are comprised of at least one image taken by a single shot camera.

7. The method of claim 1, wherein the identified geometries are curvilinear lines corresponding to substantially straight lines in scenes that the images are taken from.

8. The method of claim 1, wherein the geometries are identified using a graphical user interface in a video editing application.

9. The method of claim 8, wherein a spline tool in the video editing application is used to identify the geometries.

10. The method of claim 1, further comprising:

a) adding a set of graphical objects to at least one image in the second set of images; and

b) applying inverse of transformations to the at least one image.

11. A graphical user interface (GUI) for removing lens distortion from a video image, the GUI comprising:

a) a display section for displaying the video image;

b) a tool for defining a geometry in the video image that has distortion, wherein after the geometry is identified, a user can direct the GUI to perform a lens distortion removal operation based on the identified geometry.

12. The GUI of claim 11, wherein the tool allows the user to identify the geometry by a set of points.

13. The GUI of claim 12, wherein the tool allows the user to adjust the set of points.

14. The GUI of claim 12, wherein said adjusting comprises adding, deleting, and moving points.

15. The GUI of claim 11, wherein the tool allows for more than one geometry that has distortion to be identified on the image.

16. The GUI of claim 11, wherein the tool allows several geometries that have distortion to be identified on several images.

17. The GUI of claim 11, wherein the lens distortion operation is performed by a set of transformations performed on the identified geometry.

18. The GUI of claim 11, wherein after the lens distortion operation is perform on the image, a set of graphical objects can be superimposed on the image.

19. The GUI of claim 18, wherein after the set of graphical objects is superimposed, the user can apply an inverse of the lens distortion operation on the image.

20. A computer readable medium storing a compute program for of editing images with distortion caused by a camera lens, the computer program comprising sets of instructions for:

a) identifying a set of geometries on a first set of images taken by the camera lens;

b) finding transformations for applying on the set of geometries to remove distortion on the set of geometries;

c) applying the transformations to a second set of images to remove distortions caused by the camera lens.

21. The computer readable medium of claim 20, wherein the first set of images and the second set of images are a sequence of video images taken by said camera lens.

22. The computer readable medium of claim 20, wherein the first set of images comprises of at least two images.

23. The computer readable medium of claim 20, wherein the second set of images comprises of at least two images.

24. The computer readable medium of claim 20, further comprising sets of instruction for:

a) adding a set of graphical objects to at least one image in the second set of images; and

b) applying inverse of transformations to the at least one image.