PANORAMA IMAGE STITCHING

• US 20100194851A1
• Filed: 02/03/2009
• Published: 08/05/2010
• Est. Priority Date: 02/03/2009
• Status: Abandoned Application

• Current Assignee: Aricent Incorporated (Capgemini SE)
• Original Assignee: Aricent Incorporated (Capgemini SE)
• Inventors: ULIYAR, Mithun, PASUPALETI, Sirish Kumar, GUPTA, P.S.S.B.K
• Application Number:
  US12/365,059
• Publication Number:
  US 20100194851A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image

Abstract

Systems and methods are disclosed for generation of a panoramic image of a scene. In an implementation, the method includes acquiring a plurality of images (e.g. first image and a second image) of the scene. Subsequent to image acquisition, the plurality of images is registered based on spatial relations of image data in an overlap region between the images. The spatial relations may correspond to distance and angle between a plurality of features in the first and the second images respectively. The registered images are merged based at least in part on a block based mean of the overlap region to generate the panoramic image. Block based merging is utilized to normalize spatially varying intensity differences of the first image and the second image.

Claims

1. A method for generating a panoramic image of a scene, the method comprises:

acquiring a first image and a second image of the scene;

registering the first and the second images based at least in part on spatial relations of image data in an overlap region between the first and the second images; and

merging the registered images based at least in part on a block based mean of the overlap region between the first and second images to generate the panoramic image.

2. The method as in claim 1, wherein the acquiring comprises rotating an image capturing device about an axis and/or moving the image capturing device substantially parallel to a plane of the first image, the rotating and/or moving being performed after acquiring the first image and before acquiring the second image.

3. The method as in claim 1, wherein the acquiring comprises establishing an overlap region between the first image and second image, the extent of overlap region lying in the range of 40% to 70%.

4. The method as in claim 1, wherein the acquiring comprises capturing the second image subsequent to a change in image intensity with respect to the first image.

5. The method as in claim 1, wherein the registering comprises detecting one or more features in the first image and second image.

6. The method as in claim 5, wherein the detecting is performed by Harris corner detection technique.

7. The method of claim 5, wherein the registering further comprises matching the one or more detected features based at least in part on the spatial relations of image data in the overlap region.

8. The method of claim 7, wherein the matching comprises:

selecting a pair of feature points in a left image and a right image of the scene, the left image and right image corresponding to the first image and the second image respectively;

determining a first left distance and a first right distance between the selected pair of features in the left image and the right image respectively; and

determining a first left angle and a first right angle between the selected pair of features in the left image and the right image respectively.

9. The method of claim 8, wherein the spatial relations correspond to distance and angle between the selected pair of features in the left image and right image respectively.

10. The method of claim 8, wherein the matching further comprises:

computing a first adaptive distance threshold corresponding to the first left distance;

determining a first difference between the first left distance and the first right distance; and

comparing the determined first difference with the first adaptive distance threshold.

11. The method of claim 10, wherein the matching further comprises:

determining, based on the comparing, a difference between the first left angle and the first right angle; and

determining a first rotation direction based at least in part on the first left angle, first right angle and the determined difference between the first left angle and the first right angle.

12. The method of claim 11, wherein the matching further comprises:

comparing the difference between the first left angle and the first right angle with an angle threshold; and

selecting, based on the comparing with the angle threshold, a second and a third feature in the left image and the right image respectively.

13. The method of claim 12, wherein the matching further comprises:

determining a second left distance between the second feature and one of the selected pair of features in the left image;

determining a second right distance between the third feature and one of the selected pair of feature in the right image;

determining a second left angle between the second feature and the one of selected pair of features in the left image; and

determining a second right angle between the third feature and the one of selected pair of features in the right image.

14. The method of claim 13, wherein the matching further comprises:

computing a second adaptive distance threshold corresponding to the second left distance;

determining a second difference between the second left distance and the second right distance; and

comparing the determined second difference with the second adaptive distance threshold.

15. The method of claim 14, wherein the matching further comprises:

determining, based on the comparing of the determined second difference with the second adaptive threshold, a difference between the second left angle and the second right angle; and

determining a second rotation direction based at least in part on the second left angle, second right angle and the determined difference between the second left angle and the second right angle.

16. The method of claim 15, wherein the matching further comprises:

comparing the first rotation direction and the second rotation direction; and

setting, based on the comparison of the first and second rotation directions, a rotation flag value to 1.

17. The method of claim 16, wherein the matching further comprises:

discarding one or more of the first rotation direction and the second rotation direction if either of the corresponding first and second angle differences are less than or equal to 3.

18. The method of claim 10, wherein the determining of the first or second adaptive distance thresholds comprises:

setting the first or the second adaptive distance thresholds to 2 if the first left distance and second left distance is less than 35 respectively;

setting the first or second adaptive distance thresholds to 4 if the first left distance and second left distance is less than 100 respectively; and

setting the first or second adaptive distance thresholds to 6 if the first left distance and second left distance is greater than 100 respectively.

19. The method of claim 11, wherein the determining of the first and second rotation direction comprises adjusting the differences between the first, second left and right angles respectively, the adjusting resulting in assigning a value of −

1 or +1 to the first and second rotation direction to represent anti-clockwise direction and clockwise direction respectively.

20. The method of claim 12, wherein the comparing the difference between the first left angle and the first right angle with a angle threshold comprises setting the angle threshold to a value of +15 or −

15 degrees.

21. The method as in claim 17, wherein the matching further comprises generating a set of matched pair of features based at least in part on distance between the selected pair of features in the left and right image respectively.

22. The method as in claim 21, wherein the matching further comprises referring the generated set of matched pair of features as best match set if the number of matched pair of features in the generated set exceeds a best match count and stopping further processing for the matching.

23. The method as in claim 22, wherein the best match count is 16.

24. The method as in claim 21, wherein the matching further comprises referring the generated set of matched pair of features as predominant match set if the number of matched pair of features in the generated set exceeds a predominant match count.

25. The method as in claim 24, wherein the predominant match count is 10.

26. The method as in claim 24, wherein the matching further comprises stopping further processing when the number of predominant match sets exceeds a predominant set threshold.

27. The method as in claim 26, wherein the predominant threshold is 15.

28. The method of claim 1, wherein the merging comprises stitching and blending the first image and the second image to obtain an interim panoramic image.

48. The method of claim 14, wherein the determining of the first or second adaptive distance thresholds comprises:

setting the first or the second adaptive distance thresholds to 2 if the first left distance and second left distance is less than 35 respectively;

setting the first or second adaptive distance thresholds to 4 if the first left distance and second left distance is less than 100 respectively; and

setting the first or second adaptive distance thresholds to 6 if the first left distance and second left distance is greater than 100 respectively.

49. The method of claim 15, wherein the determining of the first and second rotation direction comprises adjusting the differences between the first, second left and right angles respectively, the adjusting resulting in assigning a value of −

1 or +1 to the first and second rotation direction to represent anti-clockwise direction and clockwise direction respectively.

29. The method of merging one or more registered images of a scene to generate a panoramic image of the scene, the method comprising:

computing, for each of a plurality of color planes associated with the scene, block wise mean intensity differences in an overlap region between the one or more registered images, the block being characterized by a predefined dimension;

smoothing a curve representing random variations in the computed mean intensity differences by a second order polynomial fit curve;

finding a linear fit that corresponds to a straight line between first and last values of the smoothened curve using a first order least squares polynomial fit; and

determining a maximum slope change point on the first order least squares polynomial fit curve, the maximum slope change point corresponding to x-coordinate at which distance between the linear fit and the first order least squares polynomial fit curve is maximum.

30. The method of claim 29, wherein the block has a dimension of up to 20 rows and up to 50 columns on each side of a stitch point between the one or more registered images, the stitch point lying on a stitching line for stitching the one or more registered images.

31. The method of claim 29, wherein the performing comprises:

determining maximum slope change points for intensity differences corresponding to each of the plurality of color planes; and

computing an average of the determined maximum slope change points to determine a final slope change point.

32. The method of claim 29, wherein the plurality of color planes correspond to one of the color spaces comprising:

RGB, CMYK, gray space, and the like.

33. The method of claim 31, wherein the performing comprises:

determining a final linear fit that corresponds to a first straight line between the first value and the maximum slope change point and a second straight line between the maximum slope change point and the last value on the first order least squares polynomial fit curve.

34. The method of claim 31, wherein the performing comprises normalizing an interim panoramic image, obtained by stitching and blending of the one or more registered images, by using the mean intensity difference data associated with the final linear fit.

50. The method of claim 31, wherein the plurality of color planes correspond to one of the color spaces comprising:

RGB, CMYK, gray space, and the like.

35. A computing based system for generating panoramic image of a scene, the system comprising:

an image acquisition module configured to acquire a first image and a second image of the scene;

an image registration module configured to register the first and the second images based at least in part on spatial relations of image data in an overlap region between the first and the second images; and

an image merging module configured to merge the registered images based at least in part on a block based mean of the overlap region between the first and second images to generate the panoramic image.

36. The system of claim 35, wherein the image registration module is further configured to detect one or more features in the first and second images.

37. The system of claim 35, wherein the image registration module comprises a feature matching module configured to match one or more detected features based at least in part on the spatial relations of image data in the overlap region.

38. The system of claim 35, wherein the image registration module is further configured to estimate a rotational transformation model for mapping the second image to the first image.

39. The system of claim 35, wherein the image registration module is further configured to:

resize the first and second image to small dimensions prior to detecting and matching of features in the first and second image; and

scale the first and second image to original size prior to merging of the first and second image.

40. The system of claim 35, wherein the image merging module is further configured to stitch and blend the registered images to obtain an interim panoramic image.

41. The system of claim 40, wherein the image merging module comprises an intensity correction module configured to perform a block based intensity correction on the interim panoramic image.

42. A method for matching features points of left image and right image of a scene captured for generating a panoramic image of the scene, the method comprising:

determining number of feature points in the left image and the right image;

selecting a pair of feature points each from the left image and the right image respectively;

matching geometrical properties of the selected pair of feature points in the left image and the selected pair of feature points in the right image, the geometric properties comprising distance and angle between the selected pair of feature points in respective images; and

upon finding a match, storing the selected pairs of feature points.

43. The method of claim 42, wherein the determining comprises detecting a plurality of feature points in the left image and right image.

44. The method of claim 42, wherein the matching comprises:

determining a distance and an angle between the selected pair of feature points in the left image;

determining distance and angle between the selected pair of feature points in the right image; and

comparing the determined distances and the angles corresponding to the left image and the right image to determine whether a match has been found.

45. The method of claim 44, wherein the matching further comprises:

selecting a new feature point in the left image;

determining distance and angle between the new feature point and one of the previously selected pair of feature points in the left image;

selecting a new feature point in the right image;

determining distance and angle between the new feature point and one of the previously selected pair of feature points in the right image; and

comparing the determined distances and the angles corresponding to the new corner feature points in the left image and the right image respectively to determine whether a new match has been found.

46. The method of claim 44, wherein the comparing further comprises determining an adaptive distance threshold based at least in part on the distance between the feature points of the left image.

47. The method of claim 44, wherein the comparing further comprises setting an angle threshold.

51. The method of claim 45, wherein the comparing further comprises determining an adaptive distance threshold based at least in part on the distance between the feature points of the left image.

52. The method of claim 45, wherein the comparing further comprises setting an angle threshold.