System and method for registration of cubic fisheye hemispherical images

• US 7,865,013 B2
• Filed: 07/10/2008
• Issued: 01/04/2011
• Est. Priority Date: 12/07/2000
• Status: Expired due to Term

• Current Assignee: iLOOKabout, Inc.
• Original Assignee: iLOOKabout, Inc.
• Inventors: Hack, Jeff
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US12/216,770
• Publication Number:
  US 20080284797A1
• Time in Patent Office: 908 Days
• Field of Search: 382/154, 382/285, 382/167, 382/274, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/167
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations

Abstract

Systems and methods for registering hemispheric images obtained using a fisheye lens for panoramic viewing, relating to spatial alignment and color balancing of complement half-side images derived from the hemispheric images. The systems and methods for spatial alignment determine the displacement of the digitized recorded images using a single translation and rotation model of distortion, which further evaluates the center of projection, and the extent of translation and rotation. The system and method for color balancing iteratively increase (or decrease) the values of pixels near the edge (discontinuity) between the two half-sides of an image, each time taking into consideration the average difference of pixel color across a smaller strip of the edge near the pixel. This invention removes the color differential but does not remove any detail underlying the features of the image.

Claims

1. A non-transitory computer readable medium having processor-executable instructions configured to cause one or more processors to perform the steps of:

(A) receiving a plurality of images in at least one input file, an edge being defined where a first given row of a first of said images meets a second given row of a second of said images and said edge is normal to a plurality of columns of said images;

(B) calculating a width of a strip based on an iteration value, said strip being oriented parallel to said edge;

(C) calculating a plurality of difference values across said edge, one of said difference values for each respective one of said columns, each respective one of said difference values being based on a plurality of first color values of said first image inside said strip and a plurality of second color values of said second image inside said strip with said strip centered about said respective column;

(D) adjusting said first color values and said second color values in each of said columns by a weighted scheme based on both (i) said respective difference value and (ii) a respective distance normal from said edge;

(E) repeating the steps (B) through (D) for each of a plurality of said iteration values, each successive one of said iteration values decreasing said width of said strip; and

(F) storing said images as adjusted in an output file.

2. A non-transitory computer readable medium according to claim 1, wherein step (B) comprises the sub-step of:

calculating said width as w=2^h

where h decreases from m to 0, and where m is chosen such that 2^m+1 is less than a width of a full side of a viewing cube.

3. A non-transitory computer readable medium according to claim 2, wherein step (C) comprises the sub-step of:

calculating said difference values as Δ

_i=A(0,i)−

B(0,i) over i=0, . . . , n

where Δ

_i are the difference values at column i across the edge, where one side of the edge is represented by two dimensional array A(j,i), and wherein the other side of the edge is represented by two dimensional array B(j,i).

4. A non-transitory computer readable medium according to claim 3, wherein step (C) comprises the sub-step of:

calculating said respective difference value as a sum of a plurality of squares of a plurality of differences between (i) a plurality of mask values of a mask and (ii) said first color values and said second color values aligned with said mask.

5. A non-transitory computer readable medium according to claim 4, wherein step (D) comprises the sub-step of:

calculating an average of said difference values in said strip as;

6. A non-transitory computer readable medium according to claim 5, wherein step (D) further comprises the sub-step of:

adjusting said first color values by;

7. A non-transitory computer readable medium according to claim 5, further comprising the step of:

ending said repeating in response to said average of said difference value becoming less than a threshold.

8. A non-transitory computer readable medium according to claim 7, wherein said threshold comprises a measure of noise in a neighborhood of said edge.

9. A non-transitory computer readable medium according to claim 1, wherein said processor executable instructions comprise a stand-alone program.

10. A non-transitory computer readable medium according to claim 1, wherein said processor executable instructions comprise comprises an executable module within a program.

11. A non-transitory computer readable medium according to claim 1, wherein said edge is formed along a straight line.

12. One or more processors including structure configured to:

(A) receive a plurality of images in at least one input file, an edge being defined where a first given row of a first of said images meets a second given row of a second of said images and said edge is normal to a plurality of columns of said images;

(B) calculate a width of a strip based on an iteration value, said strip being oriented parallel to said edge;

(C) calculate a plurality of difference values across said edge, one of said difference values for each respective one of said columns, each respective one of said difference values being based on a plurality of first color values of said first image inside said strip and a plurality of second color values of said second image inside said strip with said strip centered about said respective column;

(D) adjust said first color values and said second color values in each of said columns by a weighted scheme based on both (i) said respective difference value and (ii) a respective distance normal from said edge;

(E) repeat the steps (B) through (D) for each of a plurality of said iteration values, each successive one of said iteration values decreasing said width of said strip; and

(F) store said images as adjusted in an output file.

13. One or more processors according to claim 12, wherein the one or more processors includes structure configured to, in calculating the width of the strip:

calculate said width as w=2^h

where h decreases from m to 0, and where m is chosen such that 2^m+1 is less than a width of a full side of a viewing cube.

14. One or more processors according to claim 13, wherein the one or more processors includes structure configured to, in calculating the plurality of difference values:

calculate said difference values as Δ

_i=A(0,i)−

B(0,i) over i=0, . . . , n

where Δ

_i are the difference values at column i across the edge, where one side of the edge is represented by two dimensional array A(j,i), and wherein the other side of the edge is represented by two dimensional array B(j,i).

15. One or more processors according to claim 14, wherein the one or more processors includes structure configured to, in calculating the plurality of difference values:

calculate said respective difference value as a sum of a plurality of squares of a plurality of differences between (i) a plurality of mask values of a mask and (ii) said first color values and said second color values aligned with said mask.

16. One or more processors according to claim 15, wherein the one or more processors includes structure configured to, in adjusting said first color values and said second color values:

calculate an average of said difference values in said strip as;

17. One or more processors according to claim 16, wherein the one or more processors includes structure configured to, in adjusting said first color values and said second color values:

adjust said first color values by;

18. One or more processors according to claim 16, wherein the one or more processors includes structure configured to:

end said repeating in response to said average of said difference value becoming less than a threshold.

19. One or more processors according to claim 18, wherein said threshold comprises a measure of noise in a neighborhood of said edge.

20. One or more processors according to claim 12,wherein said edge is formed along a straight line.