Real-time capture and transformation of hemispherical video images to images in rectilinear coordinates

• US 20070263093A1
• Filed: 05/11/2006
• Published: 11/15/2007
• Est. Priority Date: 05/11/2006
• Status: Active Grant

• Current Assignee: Intergraph Software Technologies Company (Hexagon AB)
• Original Assignee: Intergraph Software Technologies Company (Hexagon AB)
• Inventors: Grindstaff, Gene, Acree, Elaine, Whitaker, Sheila
• Granted Patent: US 8,160,394 B2
• Time in Patent Office: Days
• US Class Current: 348/207.990
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method of transforming an input image from a hemispherical source to an output image in rectilinear coordinates is disclosed. The method includes receiving data representative of an input image originating from a hemispherical camera and having a size defined by an input image height and an input image width. Once the data is received, an output image width and output image height representative of a size of a rectilinear output image is calculated based on the size of the input image. A rectilinear output image is then generated by mapping input image pixels to locations within the width and height of the output image, without reference to the optical characteristics of the hemispherical camera.

Claims

1. A method of transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the method comprising:

receiving data representative of an input image originating from a hemispherical camera, and having a size defined by an input image height and an input image width;

calculating an output image width and an output image height representative of a size of a rectilinear output image based on the size of the input image;

generating a rectilinear output image by mapping pixels to locations within the width and height of the output image without reference to the optical characteristics of the hemispherical camera.

2. The method of claim 1, the method further comprising:

generating a look-up table containing designated locations of the pixels located within the input image and corresponding locations within the output image.

3. The method of claim 2, wherein generating a look-up table further comprises:

calculating an inverse mapping for each input pixel by determining a location of each input image pixel wherein the input image pixel location is represented by a column and a row.

4. The method of claim 3, wherein generating a look-up table further comprises:

calculating a look-up table index for each pixel within the input image, the look-up table index containing an association of a column and row within the output image with a pixel from the input image.

5. The method of claim 3, further comprising:

converting the location of each pixel within the input image into a normalized x,y coordinate system.

6. The method of claims 3, wherein generating a look-up table further comprises:

adjusting the column of each input image pixel such that the column lies within the input image width; and

adjusting the row of each input image pixel such that the row lies within the input image height.

7. The method of claim 4, wherein generating a look-up table further comprises:

adjusting the look-up table index column for each input pixel such that the look-up table index column is equal to that input pixel'"'"'s column; and

adjusting the look-up table index row for each input pixel such that the look-up table index row is equal to that input pixel'"'"'s row.

8. The method of claim 6, wherein adjusting the column of each input image pixel such that each input pixel lies within the initial input image width further comprises:

adjusting a value for the column of the input image that is less than zero such that the value for the column is equal to zero; and

adjusting a value for the column of the input image that is greater than the input image width such that the value for the column is equal to the input image width.

9. The method of claim 6, wherein adjusting the row of each input image pixel such that each input pixel lies within the initial input image height further comprises:

adjusting a value for the row of the input image that is less than zero such that the value for the row is equal to zero; and

adjusting a value for the row of the input image that is greater than the input image height such that the value for the row is equal to the input image height.

10. The method of claim 2, wherein generating the rectilinear output image further comprises:

using the look-up table to determine the location on the rectilinear output image to which to map the input pixel.

11. A method of generating a look-up table for transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the method of comprising:

receiving data representative of an input image originating from a hemispherical camera, and having a size defined by an input image height and an input image width;

determining a position within the look-up table for the input image pixels, wherein the position within the look-up table represents an output row and column; and

calculating an inverse mapping for each input pixel by determining a location of pixels within the input image, wherein the location is defined by a column and a row.

12. The method of claim 11, further comprising:

adjusting the column position in the look-up table for the pixels within the input image such that it is equal to that pixel'"'"'s input column; and

adjusting the row position in the look-up table for the input pixels such that it is equal to that pixel'"'"'s input row.

13. The method of claim 11, further comprising:

calculating a width and height defining a size of an output image based on the size of the input image.

14. The method of claim 11, further comprising:

converting the location of each pixel within the input image into a normalized x,y coordinate system.

15. The method of claim 11, further comprising:

adjusting the column of the input images pixel such that the column lies within the initial input image width; and

adjusting the row of the input image pixels such that the row lies within the initial input image height.

16. A method of transforming an input image from a hemispherical source to an output image in rectilinear coordinates using a look-up table, wherein the hemispherical input image is represented by a size and contains a number of pixels and the rectilinear output image is represented by a size calculated based on the input image size, the method comprising:

determining a location within the output image to transfer each input image pixel, wherein the output image location is represented by a column and a row;

determining an input image pixel associated with the output image location based on the row and column of the output image location using the look-up table; and

transferring the associated input image pixel to the output image location.

17. A surveillance system comprising.

a first hemispherical camera producing a data stream representative of a first series of input images;

an image filter configured to capture the input images within the video stream;

an image processor configured to transform the captured images to an output image in rectilinear coordinates without reference to the optical characteristics of the camera; and

a graphics renderer configured to render the rectilinear output image in real-time.

18. The surveillance system of claim 17, the system further comprising:

a second hemispherical camera producing a second data stream representative of a second series of images, wherein the second hemispherical camera is offset from the first hemispherical camera, substantially reducing blind spots.

19. The surveillance system of claim 18 wherein the first data stream is arranged as a number of pixels, each pixel having an associated location defined by a row and a column.

20. The surveillance system of claim 19 wherein the output images contain a plurality of pixel values associated with address information, wherein the address information is associated with a column and a row.

21. The surveillance system of claim 20 wherein the image processor is further configured to generate a look-up table containing the locations of pixels located within the input image and corresponding locations within the output image and transform the series of input images to images in rectilinear coordinates using the look-up table.

22. A computer program product for transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the computer code comprising:

computer code for receiving data representative of an input image from a hemispherical camera, and having a size defined by an input image height and input image width;

computer code for calculating an output image width and an output image height representative of a size of a rectilinear output image based on the size of the input image;

computer code for generating a rectilinear output image by mapping pixels to locations within the width and height of the output image without reference to the optical characteristics of the hemispherical camera;

23. The computer program product of claim 22, further comprising:

computer code generating a look-up table containing designated locations of the pixels located within the input image and corresponding locations within the output image

24. The computer program product of claim 23, wherein the computer code for generating a look-up table further comprises:

computer code for calculating an inverse mapping for each input pixel by determining a location of each input image pixel wherein the input image pixel location is represented by a column and a row.

25. The computer program product of claim 24 wherein the computer code for generating the look-up table further comprises:

computer code calculating a look-up table index for each pixel within the input image, the look-up table index containing an association of a column and row within the output image with a pixel from the input image.

26. The computer program product of claim 24, wherein the computer code for generating the look-up table further comprises:

computer code for converting the location of each pixel within the input image into a normalized x,y coordinate system.

27. The computer program product of claim 24, wherein the computer code for generating the look-up table further comprises:

computer code for adjusting the column of each input image pixel such that the column lies within the input image width; and

computer code for adjusting the row of each input image pixel such that the row lies within the input image height.

28. The computer program product of claim 24, wherein the computer code for generating the look-up table further comprises:

computer code for adjusting the look-up table index column for each input pixel such that the look-up table index column is equal to that input pixel'"'"'s column; and

computer code for adjusting the look-up table index row for each input pixel such that the look-up table index row is equal to that input pixel'"'"'s row.

29. The computer program product of claim 27, wherein the computer code for adjusting the column of each input image pixel such that each input pixel lies within the initial input image width further comprises:

computer code for adjusting a value for the column of the input image that is less than zero such that the value for the column is equal to zero; and

computer code for adjusting a value for the column of the input image that is greater than the input image width such that the value for the column is equal to the input image width.

30. The computer program product of claim 27, wherein the computer code for adjusting the row of each input image pixel such that each input pixel lies within the initial input image height further comprises:

computer code adjusting a value for the row of the input image that is less than zero such that the value for the row is equal to zero; and

compute code for adjusting a value for the row of the input image that is greater than the input image height such that the value for the row is equal to the input image height

31. The computer program product of claim 24, wherein computer code for generating the rectilinear output image further comprises:

computer code for using the look-up table to determine the location on the rectilinear output image to which to map the input pixel.

32. A computer program product for generating a look-up table for transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the computer code comprising:

computer code for receiving data representative of an input image originating from a hemispherical camera, and having a size defined by an input image height and an input image width;

computer code for determining a position within the look-up table for the input image pixels, wherein the position within the look-up table represents an output row and column; and

computer code for calculating an inverse mapping for each input pixel by determining a location of pixels within the input image, wherein the location is defined by a column and a row.

33. The computer program product of claim 32, further comprising:

computer code for adjusting the column position in the look-up table for the pixels within the input image such that it is equal to that pixel'"'"'s input column; and

computer code for adjusting the row position in the look-up table for the input pixels such that it is equal to that pixel'"'"'s input row.

34. The computer program product of claim 32, further comprising:

computer code for calculating a width and height defining a size of an output image based on the size of the input image.

35. The computer program product of claim 32, further comprising:

computer code for converting the location of each pixel within the input image into a normalized x,y coordinate system.

36. The computer program product of claim 32, further comprising:

computer code for adjusting the column of the input images pixel such that the column lies within the initial input image width; and

computer code for adjusting the row of the input image pixels such that the row lies within the initial input image height.

37. A computer program product for transforming an input image from a hemispherical source to an output image in rectilinear coordinates using a look-up table, wherein the hemispherical input image is represented by a size and contains a number of pixels and the rectilinear output image is represented by a size calculated based on the input image size, the computer code comprising:

computer code for determining a location within the output image to transfer each input image pixel, wherein the output image location is represented by a column and a row;

computer code for determining an input image pixel associated with the output image location based on the row and column of the output image location using the look-up table; and

computer code for transferring the associated input image pixel to the output image location.

38. A method of transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the method comprising:

receiving data representative of an input image originating from a hemispherical camera, wherein the data is arranged as a number of pixels defining a size including a height and a width;

calculating a width and height representative of a size of a rectilinear output image based on the size of the input image;

generating a rectilinear output image by mapping input image pixels to a location within the width and height of the output image without reference to the optical characteristics of the camera.

39. The method of claim 38, wherein generating a rectilinear output image further comprises:

mapping input image pixels to a location within the width and height of the output image such that only a single input image pixel is mapped to an output image location and the output image does not contain Moire effects.

40. The method of claim 38, wherein generating a rectilinear out image further comprises mapping a single input image pixel to a plurality of locations on the output image.

41. A computer program product for transforming an input image from a hemispherical source to an output image in rectilinear coordinates, the computer code comprising:

computer code for receiving data representative of an input image originating from a hemispherical camera, wherein the data is arranged as a number of pixels defining a size including a height and a width;

computer code for calculating a width and height representative of a size of a rectilinear output image based on the size of the input image;

computer code for generating a rectilinear output image by mapping input image pixels to a location within the width and height of the output image without reference to the optical characteristics of the camera.

42. The computer program product of claim 41, wherein the computer code for generating a rectilinear output image further comprises:

computer code mapping input image pixels to a location within the width and height of the output image such that only a single input image pixel is mapped to an output image location and the output image does not contain Moire effects.

43. The computer program product of claim 41, wherein the computer code for generating a rectilinear out image further comprises mapping a single input image pixel to a plurality of locations on the output image.