System and method for panoramic imaging

• US 20030068098A1
• Filed: 09/26/2002
• Published: 04/10/2003
• Est. Priority Date: 09/27/2001
• Status: Active Grant

• Current Assignee: 360ai Solutions LLC
• Original Assignee: EyeSee360, Inc.
• Inventors: Herman, Herman, Singh, Sanjiv, Rondinelli, Michael
• Granted Patent: US 7,123,777 B2
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations

Abstract

The present invention provides a system for processing panoramic photographic images. The system includes a mirror for reflecting an image of a scene, a mounting assembly for mounting the mirror on an axis, a camera for capturing the image reflected by the mirror, a digital converter device for producing pixel data representative of the captured image, and means for radially linearly mapping the pixel data into a viewable image. The mirror includes a convex reflective surface defined by rotating around the axis: an equi-angular shape or a compensated equi-angular shape. Methods for processing images in accordance with the system are also provided.

Claims

1. A system for processing images, the system comprising:

a mirror for reflecting an image of a scene;

a mounting assembly for mounting the mirror on an axis, wherein the mirror includes a convex reflective surface defined by rotating around the axis;

an equi-angular shape or a compensated equi-angular shape;

a camera for capturing the image reflected by the mirror;

a digital converter device for producing pixel data representative of the captured image; and

means for radially linearly mapping the pixel data into a viewable image.

2. The system of claim 1, wherein the mirror has a compensated equi-angular shape described by the equation:

3. The system of claim 1, wherein the means for radially linearly mapping the pixel data into a viewable image comprises:

means for retrieving a source image file including the pixel data of the captured image;

a processor for creating a destination image file buffer, for mapping the pixel data of the captured image to the destination image file buffer, and for outputting pixel data from the destination image file buffer as a destination image file; and

means for displaying a viewable image defined by the destination file.

4. The system of claim 3, wherein the processor further serves as means for:

defining a first set of coordinates of pixels in the destination image file;

defining a second set of coordinates of pixels in the source image file;

identifying coordinates of the second set that correspond to coordinates of the first set; and

inserting pixel data for pixel locations corresponding to the second set of coordinates into pixel locations corresponding to the first set of coordinates.

5. The system of claim 3, wherein the processor further serves as means for interpolating the source image pixel data to produce pixel data for the destination image file buffer.

6. The system of claim 3, wherein the source image file comprises a panoramic projection image file.

7. The system of claim 3, wherein the destination image file comprises one of:

a cylindrical panoramic projection image file, a perspective panoramic projection image file, an equi-rectangular panoramic projection image file, and an equi-angular panoramic projection image file.

8. The system of claim 4, wherein the first set of coordinates are spherical coordinates and the second set of coordinates are rectangular coordinates.

9. The system of claim 6, wherein the panoramic projection image file comprises a partial equi-rectangular projection.

10. The system of claim 6, wherein the panoramic projection image file comprises a modified partial equi-rectangular projection.

11. The system of claim 1, further comprising a target apparatus attached to the mirror.

12. The system of claim 1, further comprising:

means for transmitting the pixel data of the captured image to a server computer; and

means for processing the pixel data of the captured image on the server computer to obtain the viewable image.

13. A system for processing images, the system comprising:

a mirror for reflecting an image of a scene;

means for mounting the mirror on an axis, wherein the mirror includes a convex reflective surface defined by rotating around the axis;

an equi-angular shape or a compensated equi-angular shape;

means for capturing the image reflected by the mirror;

means for producing pixel data representative of the captured image; and

means for radially linearly mapping the pixel data into a viewable image.

14. The system of claim 13, wherein the mirror has a compensated equi-angular shape described by the equation:

15. The system of claim 13, wherein the means for radially linearly mapping the pixel data into a viewable image comprises:

means for retrieving a source image file including the pixel data of the captured image;

a processor for creating a destination image file buffer, for mapping the pixel data of the captured image to the destination image file buffer, and for outputting pixel data from the destination image file buffer as a destination image file; and

means for displaying a viewable image defined by the destination file.

16. The system of claim 15, wherein the processor further serves as means for:

defining a first set of coordinates of pixels in the destination image file;

defining a second set of coordinates of pixels in the source image file;

identifying coordinates of the second set that correspond to coordinates of the first set; and

inserting pixel data for pixel locations corresponding to the second set of coordinates into pixel locations corresponding to the first set of coordinates.

17. The system of claim 15, wherein the processor further serves as means for interpolating the source image pixel data to produce pixel data for the destination image file buffer.

18. The system of claim 15, wherein the source image file comprises a panoramic projection image file.

19. The system of claim 15, wherein the destination image file comprises one of:

a cylindrical panoramic projection image file, a perspective panoramic projection image file, an equi-rectangular panoramic projection image file, and an equi-angular panoramic projection image file.

20. The system of claim 16, wherein the first set of coordinates are spherical coordinates and the second set of coordinates are rectangular coordinates.

21. The system of claim 18, wherein the panoramic projection image file comprises a partial equi-rectangular projection.

22. The system of claim 18, wherein the panoramic projection image file comprises a modified partial equi-rectangular projection.

23. The system of claim 13, further comprising a target apparatus attached to the mirror.

24. The system of claim 13, further comprising:

means for transmitting the pixel data of the captured image to a server computer; and

means for processing the pixel data of the captured image on the server computer to obtain the viewable image.

25. A method of processing images, the method comprising the steps of:

providing a mirror for reflecting an image of a scene;

mounting the mirror on an axis, wherein the mirror includes a convex reflective surface defined by rotating around the axis;

an equi-angular shape or a compensated equi-angular shape;

capturing the image reflected by the mirror;

producing pixel data representative of the captured image; and

radially linearly mapping the pixel data into a viewable image.

26. The method of claim 25, wherein the mirror has a compensated equi-angular shape described by the equation:

27. The method of claim 25, wherein the step of radially linearly mapping the pixel data into a viewable image further comprises:

retrieving a source image file including the pixel data of the captured image;

creating a destination image file buffer;

mapping the pixel data from the source image file to the destination image file buffer;

outputting pixel data from the destination image file buffer as a destination image file; and

displaying a viewable image defined by the destination file.

28. The method of claim 27, wherein the step of mapping pixel data from the source image file to the destination image file buffer comprises the steps of:

defining a first set of coordinates of pixels in the destination image file;

defining a second set of coordinates of pixels in the source image file;

identifying coordinates of the second set that correspond to coordinates of the first set;

inserting pixel data for pixel locations corresponding the second set of coordinates into pixel locations corresponding to the first set of coordinates.

29. The method of claim 27, wherein the step of mapping the pixel data from the source image file to the destination image file buffer includes the step of:

interpolating the source image pixel data to produce pixel data for the destination image file buffer.

30. The method of claim 27, wherein the source image file comprises a panoramic projection image file.

31. The method of claim 27, wherein the destination image file comprises one of:

a cylindrical panoramic projection image file, a perspective panoramic projection image file, an equi-rectangular panoramic projection image file, and an equi-angular panoramic projection image file.

32. The method of claim 28, wherein the first set of coordinates are spherical coordinates and the second set of coordinates are rectangular coordinates.

33. The method of claim 30, wherein the panoramic projection image file comprises a partial equi-rectangular projection.

34. The method of claim 30, wherein the panoramic projection image file comprises a modified partial equi-rectangular projection.

35. The method of claim 25, further comprising the steps of:

transmitting the pixel data of the captured image to a server computer; and

processing the pixel data of the captured image on the server computer to obtain the viewable image.

36. A method of processing images, the method comprising the steps of:

retrieving a source image including pixel data;

creating a first texture map memory buffer;

transferring the pixel data from the source image to the first texture map memory buffer;

producing a plurality of vertices for a first model of a viewable image, wherein the vertices are representative of one or more points corresponding to one or more space vectors of the source image;

computing one or more texture map coordinates for each of the vertices, wherein the one or more texture map coordinates are representative of one or more pieces of pixel data in the first texture map memory buffer corresponding to one or more pieces of pixel data in the source image;

transferring the first model, including the vertices and the one or more texture map coordinates, to a graphics hardware device;

instructing the graphics hardware device to use the pixel data to complete the first model and display the completed model as a viewable panoramic image.

37. The method of claim 36, wherein the steps may be performed sequentially.

38. The method of claim 36, wherein one or more of the steps may be performed simultaneously.

39. The method of claim 36, wherein one or more of the steps may be repeated to sequentially display a plurality of viewable images, and wherein the plurality of viewable images may be displayed at a video frequency rate.

40. The method of claim 36, wherein the step of producing the plurality of vertices for a first model of a viewable image is executed once;

the step of computing one or more texture map coordinates for each of the vertices is executed once; and

the step of transferring the first model is executed once, further comprising the steps of;

updating the pixel data in the first texture map memory buffer;

instructing the graphics hardware device to use the updated pixel data to complete the first model and to display the completed model as a viewable image; and

repeating the fourth and fifth steps so as to sequentially display a plurality of viewable images, wherein the plurality of viewable images may be displayed at a video frequency rate.

41. The method of claim 40, wherein the step of instructing the graphics hardware device to use the updated pixel data to complete the first model and to display the completed model as a viewable image comprises the steps of:

adjusting the pitch, yaw, and/or roll of the completed model; and

displaying the adjusted model as a viewable image.

42. The method of claim 36, further comprising the steps of:

creating a second texture map memory buffer;

transferring even lines of the pixel data from the source image to the first texture map memory buffer;

transferring odd lines of the pixel data from the source image to the second texture map memory buffer;

instructing the graphics hardware device to use the pixel data from the first texture map memory buffer to complete the first model and display the completed model as a viewable panoramic image;

instructing the graphics hardware device to use the pixel data from the second texture map memory buffer to complete the first model and display the completed model as a viewable panoramic image; and

repeating the second, third, fourth and fifth steps so as to sequentially display a plurality of viewable images, wherein the plurality of viewable images may be displayed at an interlaced video frequency rate.

43. The method of claim 42, further comprising the steps of:

shifting the texture coordinates of the first texture map memory buffer in the vertical direction by one half of a distance spanned by a pixel;

producing a second model including the shifted texture map coordinates of the first texture map memory buffer;

transferring the second model to the graphics hardware device;

instructing the graphics hardware device to use the pixel data from the first texture map memory buffer to complete the second model and display the completed second model as a viewable panoramic image;

instructing the graphics hardware device to use the pixel data from the second texture map memory buffer to complete the first model and display the completed first model as a viewable panoramic image; and

repeating the fourth and fifth steps so as to sequentially display a plurality of viewable images, wherein the plurality of viewable images may be displayed at an interlaced video frequency rate.

44. The method of claim 43, wherein the second model comprises one of:

a cube, a hexahedron, a sphere, an ellipsoid, a cylinder, an icosahedron, and an arbitrary three-dimensional model.

45. The method of claim 36, wherein the source image includes a panoramic projection image file.

46. The method of claim 45, wherein the panoramic projection image file comprises one of:

an equi-angular panoramic projection image file, a cylindrical panoramic projection image file, an equi-rectangular panoramic projection image file, a partial equi-rectangular panoramic projection image file, a modified partial equi-rectangular panoramic projection image file, and a cubic panoramic projection image file.

47. The method of claim 36, wherein the first model comprises one of:

a cube, a hexahedron, a sphere, an ellipsoid, a cylinder, an icosahedron, and an arbitrary three-dimensional model.

48. The method of claim 36, wherein the pixel data comprises red, blue, and green image data.

49. The method of claim 36, wherein the pixel data comprises luminance and chroma image data.

50. The method of claim 36, wherein the source image is retrieved from one of:

a camera, a tape deck, a streaming video source, a file, a hard disk, an optical disk, a digital video disk, or a television broadcast.

51. The method of claim 36, wherein the step of instructing the graphics hardware device to use the pixel data to complete the model includes the step of:

interpolating the texture map memory buffer pixel data.

52. The method of claim 36, further comprising the step of:

transmitting the viewable image to a server computer.

53. The method of claim 36, further comprising the step of:

sequentially transmitting a plurality of the viewable images to a server computer.

54. An apparatus for processing images, the apparatus comprising:

means for receiving a source image including pixel data;

a processor for creating a texture map memory buffer, for transferring the pixel data from the source image to the texture map memory buffer, for producing a plurality of vertices for a model of a viewable image, wherein the vertices are representative of one or more points corresponding to one or more space vectors of the source image, and for computing one or more texture map coordinates for each of the vertices, wherein the one or more texture map coordinates are representative of one or more pieces of pixel data in the texture map memory buffer corresponding to one or more pieces of pixel data in the source image, and

a graphics hardware device for receiving the model, including the vertices and the one or more texture map coordinates, for utilizing the pixel data to complete the model, and for displaying the completed model as a viewable image.

55. The apparatus of claim 54, wherein the processor further serves as means for:

adjusting the pitch, yaw, and/or roll of the completed model; and

displaying the adjusted model as a viewable image.

56. The apparatus of claim 54, wherein the source image includes a panoramic projection image file.

57. The apparatus of claim 56, wherein the panoramic projection image file comprises one of:

an equi-angular panoramic projection image file, a cylindrical panoramic projection image file, an equi-rectangular panoramic projection image file, a partial equi-rectangular panoramic projection image file, a modified partial equi-rectangular panoramic projection image file, and a cubic panoramic projection image file.

58. The apparatus of claim 54, wherein the model comprises one of:

a cube, a hexahedron, a sphere, an ellipsoid, a cylinder, an icosahedron, and an arbitrary three-dimensional model.

59. The apparatus of claim 54, wherein the pixel data comprises red, blue, and green image data.

60. The apparatus of claim 54, wherein the pixel data comprises luminance and chroma image data.

61. The apparatus of claim 54, wherein the source image is retrieved from one of:

a camera, a tape deck, a streaming video source, a file, a hard disk, an optical disk, a digital video disk, or a television broadcast.

62. The apparatus of claim 54, wherein the processor further serves as means for:

interpolating the texture map memory buffer pixel data.

63. The apparatus of claim 54, wherein the processor further serves as means for:

transmitting the viewable image to a server computer.

64. The apparatus of claim 54, wherein the processor further serves as means for:

sequentially transmitting a plurality of the viewable images to a server computer.

65. The apparatus of claim 54, further comprising:

a mirror for reflecting the source image;

a camera for capturing the source image reflected by the mirror; and

a digital converter device for converting the captured image into the pixel data.

66. The apparatus of claim 65, further comprising a target apparatus attached to the mirror.