Projecting images onto a surface

• US 20020181802A1
• Filed: 05/03/2001
• Published: 12/05/2002
• Est. Priority Date: 05/03/2001
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Peterson, John
• Granted Patent: US 7,006,707 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 3/153 using elastic snapping

Abstract

The orientation of a camera associated with a first image is determined based on a shape of a perimeter of a corrected version of the first image. The corrected version of the first image has less perspective distortion relative to a reference image than the first image. The shape of the perimeter of the corrected version of the first image is also different from the shape of the perimeter of the first image. The first image is then projected onto a surface based on the orientation of the camera.

Claims

1. A method comprising:

determining an orientation of a camera associated with a first image based on a shape of a perimeter of a corrected version of the first image, wherein the corrected version of the first image has less perspective distortion relative to a reference image than the first image and the shape of the perimeter of the corrected version of the first image is different from the shape of the perimeter of the first image; and

projecting the first image on a surface based on the orientation of the camera associated with the first image.

2. The method of claim 1 further comprising:

determining a focal length of a camera associated with the first image based on the shape of the perimeter of the corrected version of the first images, wherein the step of projecting the first image is further based on the focal length.

3. The method of claim 1, further comprising:

projecting the reference image on the surface.

4. The method of claim 3 further comprising:

merging the projected reference image and the projected first image to form a panoramic image.

5. The method of claim 3 further comprising:

projecting a three-dimensional object onto the surface;

merging the projected three-dimensional object, the reference image and the first image to form a panoramic image.

6. The method of claim 1 wherein the surface is cylindrical.

7. The method of claim 1 wherein the surface is spherical.

8. The method of claim 1 wherein the surface is planar.

9. The method of claim 1 further comprising:

determining an orientation of a camera associated with a second image based on a shape of a perimeter of a corrected version of the second image, wherein the corrected version of the second image has less perspective distortion relative to the reference image than the first image; and

projecting the second image on the surface based on the orientation of the camera associated with the second image.

10. The method of claim 2 wherein determining the focal length and rotation angle further comprises:

selecting initial values for the orientation and the focal length; and

improving the accuracy of the selected values of the orientation and the focal length by;

estimating the shape of the perimeter of the corrected version of the first image based on the selected values of the orientation and the focal length;

comparing the estimated shape and the actual shape of the perimeter of the corrected version of the first image;

adjusting the selected values of the orientation and the focal length based on a difference between the estimated shape and the actual shape of the perimeter of the corrected version of the first image.

11. The method of claim 10 wherein improving the accuracy of the selected values of the orientation and the focal length further comprises:

computing a difference between the selected values of the orientation and the focal length with the adjusted values of the orientation and the focal length;

if the computed difference is below a threshold value;

determining that the adjusted values of the orientation and the adjusted value of the focal length are the actual orientation and the actual focal length;

otherwise, if the computed difference is not below the threshold value;

selecting the adjusted values of the orientation and the focal length as the values of the orientation and the focal length; and

repeating the step of improving the accuracy of the selected values of the orientation and the focal length.

12. The method of claim 10 wherein the initial value of the orientation is selected to be an orientation of a camera associated with the reference image.

13. The method of claim 1, wherein the orientation of the camera associated with the first image is measured relative to an orientation of a camera associated with the reference image.

14. The method of claim 10 wherein the initial value of the focal length is selected based on a measurement of the first image.

15. The method of claim 14 wherein the selected initial value of the focal length is the sum of a length and a width of the image.

16. The method of claim 10 wherein a Newton'"'"'s iteration is used to adjust the initial values of the rotation angle and the focal length.

17. The method of claim 1 wherein the orientation comprises a rotation angle of the camera.

18. The method of claim 1 wherein the reference image is an image of a reference segment of a view and the first image is an image of a first segment of the view that overlaps the reference segment of the view, the method further comprising:

correcting for perspective distortion in the first image relative to the reference image to generate the corrected version of the first image.

19. The method of claim 18 further comprising:

determining a position offset of the first segment of the view relative to the reference segment of the view, wherein correcting for perspective distortion is based on the determined position offset

20. The method of claim 18 wherein the perimeter of the first image includes at least a first reference point and a second reference point and correcting for perspective distortion alters the shape of the perimeter of the first image by moving the first reference point relative to the second reference point.

21. The method of claim 20 wherein the first and second reference points are vertices defined by the shape of the perimeter of the first image.

22. The method of claim 21 wherein the shape of the perimeter of the first image is rectangular and correcting for perspective distortion alters the shape of the perimeter of the first image into a trapezoid.

23. The method of claim 1 wherein determining the orientation is further based on the shape of the perimeter of the first image.

24. The method of claim 1 wherein the perimeter of the first image has the same shape as the perimeter of the reference image.

26. The article of claim 25 wherein the instructions further cause the machine to:

determine a focal length of a camera associated with the first image based on the shape of the perimeter of the corrected version of the first images, wherein the step of projecting the first image is further based on the focal length.

27. The article of claim 25 wherein the instructions further cause the machine to:

project the reference image on the surface.

28. The article of claim 27 wherein the instructions further cause the machine to:

merge the projected reference image and the projected first image to form a panoramic image.

29. The article of claim 27 wherein the instructions further cause the machine to:

project a three-dimensional object onto the surface;

merge the projected three-dimensional object, the reference image and the first image to form a panoramic image.

30. The article of claim 25 wherein the surface is cylindrical.

31. The article of claim 25 wherein the surface is spherical.

32. The article of claim 25 wherein the surface is planar.

33. The article of claim 25 wherein the instructions further cause the machine to:

determine an orientation of a camera associated with a second image based on a shape of a perimeter of a corrected version of the second image, wherein the corrected version of the second image has less perspective distortion relative to the reference image than the first image; and

project the second image on the surface based on the orientation of the camera associated with the second image.

34. The article of claim 26 wherein determining the focal length and rotation angle further comprises:

selecting initial values for the orientation and the focal length; and

improving the accuracy of the selected values of the orientation and the focal length by;

estimating the shape of the perimeter of the corrected version of the first image based on the selected values of the orientation and the focal length;

comparing the estimated shape and the actual shape of the perimeter of the corrected version of the first image;

adjusting the selected values of the orientation and the focal length based on a difference between the estimated shape and the actual shape of the perimeter of the corrected version of the first image.

35. The article of claim 34 wherein improving the accuracy of the selected values of the orientation and the focal length further comprises:

computing a difference between the selected values of the orientation and the focal length with the adjusted values of the orientation and the focal length;

if the computed difference is below a threshold value;

determining that the adjusted values of the orientation and the adjusted value of the focal length are the actual orientation and the actual focal length;

otherwise, if the computed difference is not below the threshold value;

selecting the adjusted values of the orientation and the focal length as the values of the orientation and the focal length; and

repeating the step of improving the accuracy of the selected values of the orientation and the focal length.

36. The article of claim 34 wherein the initial value of the orientation is selected to be an orientation of a camera associated with the reference image.

37. The article of claim 25 wherein the orientation of the camera associated with the first image is measured relative to an orientation of a camera associated with the reference image.

38. The article of claim 34 wherein the initial value of the focal length is selected based on a measurement of the first image.

39. The article of claim 38 wherein the selected initial value of the focal length is the sum of a length and a width of the image.

40. The article of claim 34 wherein a Newton'"'"'s iteration is used to adjust the initial values of the rotation angle and the focal length.

41. The article of claim 25 wherein the orientation comprises a rotation-angle of the camera.

42. The article of claim 25 wherein the reference image is an image of a reference segment of a view and the first image is an image of a first segment of the view that overlaps the reference segment of the view, the instructions further causing the processor to:

correct for perspective distortion in the first image relative to the reference image to generate the corrected version of the first image.

43. The article of claim 42 wherein the instructions further cause the machine to:

determine a position offset of the first segment of the view relative to the reference segment of the view, wherein correcting for perspective distortion is based on the determined position offset

44. The article of claim 42 wherein the perimeter of the first image includes at least a first reference point and a second reference point and correcting for perspective distortion alters the shape of the perimeter of the first image by moving the first reference point relative to the second reference point.

45. The article of claim 44 wherein the first and second reference points are vertices defined by the shape of the perimeter of the first image.

46. The article of claim 45 wherein the shape of the perimeter of the first image is rectangular and correcting for perspective distortion alters the shape of the perimeter of the first image into a trapezoid.

47. The article of claim 25 wherein determining the orientation is further based on the shape of the perimeter of the first image.

48. The article of claim 25 wherein the perimeter of the first image has the same shape as the perimeter of the reference image.

25. An article comprising a machine-readable medium on which are tangibly stored machine-executable instructions the stored instructions being operable to cause a machine to:

determine an orientation of a camera associated with a first image based on a shape of a perimeter of a corrected version of the first image, wherein the corrected version of the first image has less perspective distortion relative to a reference image than the first image and the shape of the perimeter of the corrected version of the first image is different from the shape of the perimeter of the first image; and

project the first image on a surface based on the orientation of the camera associated with the first image.