System and method for rectified mosaicing of images recorded by a moving camera

• US 20060120625A1
• Filed: 11/10/2005
• Published: 06/08/2006
• Est. Priority Date: 08/20/1999
• Status: Abandoned Application

• Current Assignee: Emaki Incorporated, Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Original Assignee: Emaki Incorporated, Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Inventors: Miyazawa, Takeo, Arora, Chetan, Zomet, Assaf, Peleg, Shmuel
• Application Number:
  US11/271,465
• Publication Number:
  US 20060120625A1
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A system is described for generating a rectified mosaic image from a plurality of individual images, the system comprising a quadrangular region defining module, a warping module and a mosaicing module. The quadrangular region defining module is configured to define in one individual image a quadrangular region in relation to two points on a vertical anchor in the one individual image and mappings of two points on a vertical anchor in at least one other individual image into the one individual image. The warping module is configured to warp the quadrangular region to a rectangular region. The mosaicing module configured to mosaic the quadrangular region to the mosaic image. The system further generates a mosaic from a plurality of panoramic images, the system comprising a motion determining module, a normalizing module, a strip selection module, and a mosaicing module. The motion determining module is configured to determine image motion between two panoramic images. The normalizing module is configured to normalize respective columns in the panoramic images in relation to the ratio of the image motion thereof to the image motion of a selected column, thereby to generate normalized panoramic images. The strip selection module is configured to select strips of the normalized panoramic images. The mosaicing module is configured to mosaic the selected strips together.

Claims

1-27. -27. (canceled)

28. A method of producing a mosaic of a scene from a sequence of camera images of the scene acquired at a respective sequence of positions, the method comprising:

determining first and second anchor points in each image;

warping at least one portion of a given image of the camera images that includes the image'"'"'s anchor points using a transform that changes the scale of a region in the portion and leaves the position of the two anchor points invariant; and

for each portion of the at least one portion, placing the portion adjacent a portion of an other image acquired at a position adjacent that at which the given image is acquired so that features in the portions of the given and other images are aligned.

29. A method according to claim 28 wherein warping at least one portion of the given image comprises determining for the given image and the other image, two additional points for each of the at least one portion of the given image and corresponding homologous points for the other image and warping the at least one portion responsive to the additional points.

30. A method according to claim 29 wherein the homologous and additional points correspond under a homography that transforms a portion of the given image to the other image.

31. A method according to claim 29 wherein at least a portion of a line segment between the additional points is located in the region that undergoes the scale change.

32. A method according to claim 31 wherein the at least one portion comprises one portion.

33. A method according to claim 32 and comprising determining the anchor points so that in each image a line segment between the anchor points has a same length.

34. A method according to claim 33 and comprising determining the anchor points so that in each image the line segment between the anchor points has substantially a same direction.

35. A method according to claim 34 wherein the corresponding homologous points in the other image are the anchor points of the other image.

36. A method according to claim 35 wherein warping the one portion comprises warping a quadrilateral defined by the anchor and additional points into a rectangle.

37. A method according to claim 31 wherein the at least one portion comprises two portions.

38. A method according to claim 37 and comprising determining the anchor points so that in each image a line segment between the anchor points has a same length.

39. A method according to claim 38 and comprising determining the anchor points so that in each image the line segment between the anchor points has a same direction.

40. A method according to claim 39 wherein warping a first portion of the two portions comprises warping a quadrilateral defined by the anchor points and the additional points into a rectangle.

41. A method according to claim 40 wherein warping the second portion comprises warping a quadrilateral defined by points collinear with the given image'"'"'s anchor points and the additional points determined for the second portion into a rectangle.

42. A method according to claim 28 and comprising warping at least one portion of each image.

43. A method according to claim 28 and comprising placing portions of each pair of images acquired at adjacent positions adjacent each other so that features in the images are aligned and the line segments between the anchor points of the images are invariant to within a translation.

44. A method according to claim 28 wherein the line segment between the anchor points of an image is substantially perpendicular to a direction of optic flow in the image.

45. A method of producing a mosaic of a scene from a sequence of camera images of the scene acquired at a respective sequence of positions, the method comprising:

determining first and second anchor points in each image;

determining for a given image of the sequence of images and at least one other image of the sequence of images acquired at a position adjacent that at which the given image is acquired, two additional points in the first image and corresponding homologous points in the other image;

warping at least a portion of the given image using a transform that leaves the anchor points in the given image invariant so that a distance between the additional points and the corresponding points are the same; and

placing at least portions of the given and other image adjacent each other so that the additional and corresponding points are aligned.

46. A method according to claim 45 wherein the homologous and additional points correspond under a homography that transforms a portion of the given image to the other image.

47. A method according to claim 45 and comprising determining the anchor points so that in each image a line segment between the anchor points has a same length.

48. A method according to claim 47 and comprising determining the anchor points so that in each image the line segment between the anchor points has a same direction;

49. A method according to claim 45 and comprising placing portions of each pair of images acquired at adjacent positions adjacent each other so that features in the images are aligned and the line segments between the anchor points of the images are invariant to within a translation.

50. A method of producing a mosaic of a scene from a sequence of camera images of the scene acquired at a respective sequence of positions, the method comprising:

determining first and second anchor points in each image so that in each image the line segment between the anchor points has a same length and same direction;

determining for each image at least one quadrilateral region defined by two auxiliary points collinear with the image anchor points and separated by a distance equal to that which separates the anchor points, and two additional points for which at least a portion of a line between them lies in the image;

warping the at least one quadrilateral into a rectangle using a transform under which the positions of the two auxiliary points are invariant; and

aligning rectangles from images acquired at adjacent positions adjacent to each other.

51. A method according to claim 50 wherein for at least one of the quadrilateral regions the two auxiliary points that are collinear with the anchor points are coincident with the anchor points.

52. A method according to claim 51 wherein the two additional points that define a quadrilateral in a given image are homologous with corresponding points in another image acquired at a position adjacent to that at which the given image is acquired.

53. A method according to claim 52 wherein homologous points correspond to the additional points under a homography that transforms a portion of the given image to the other image.

54. A method according to claim 50 and comprising warping at least one portion of each image.

55. A method according to claim 50 and comprising placing portions of each pair of images acquired at adjacent positions adjacent each other so that features in the images are aligned and the line segments between the anchor points of the images are invariant to within a translation.