Image processing method and apparatus and computer-readable storage medium using improved distortion correction

• US 7,313,289 B2
• Filed: 08/29/2001
• Issued: 12/25/2007
• Est. Priority Date: 08/30/2000
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Aoki, Shin, Kitaguchi, Takashi, Murata, Norihiko
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): KIM, CHONG R
• Application Number:
  US09/940,432
• Publication Number:
  US 20020041717A1
• Time in Patent Office: 2,309 Days
• Field of Search: 382/275, 382/285, 382/294, 382/154, 356/12
• US Class Current: 382/275
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

In image processing method and apparatus of the present invention, image distortions caused by oblique imaging are corrected. A feature point of one of a plurality of partially overlapping images corresponding to a common location of an original image, shared by the plurality of partially overlapping images, is determined. A matched point of one of the other partially overlapping images corresponding to the feature point is determined so that a direction of the object plane is calculated based on the feature point and the matched point. One of the plurality of partially overlapping images is selected as a standard image whose image distortions are to be corrected. A distortion-corrected image is generated on a projection plane by projecting the standard image onto the projection plane based on the direction of the object plane such that image distortions in the standard image are eliminated.

Claims

1. An image processing method for correcting image distortions, comprising the steps of:

inputting a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

determining a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and determining a matched point of one of the other partially overlapping images corresponding to the feature point so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

selecting one of the plurality of partially overlapping images as a standard image whose image distortions are to be corrected, the selection being based on one of (1) a direction of a straight-line pattern contained in each image, (2) the feature point and the matched point determined in the determining step, and (3) a calculated direction of the object plane for each of the partially overlapping images; and

generating a distortion-corrected image on a projection plane by projecting the standard image onto the projection plane based on the direction of the tangible object plane such that image distortions in the standard image are eliminated.

2. The image processing method according to claim 1, wherein in said selecting step, one of the plurality of partially overlapping images is automatically selected as the standard image based on the direction of the straight-line pattern contained in each image.

3. The image processing method according to claim 1 wherein in said selecting step, one of the plurality of partially overlapping images is automatically selected as the standard image based on the feature point and the matched point determined by said determining step.

4. The image processing method according to claim 1, wherein in said selecting step, one of the plurality of partially overlapping images is automatically selected as the standard image based on the calculated direction of the object plane for each of the partially overlapping images.

5. The image processing method of claim 1, wherein said standard image is projected with a perspective projection matrix operation.

6. The image processing method of claim 5, wherein said perspective projection matrix is calculated based on coordinates of at least four combinations of feature points of the standard image and matched points corresponding thereto.

7. The image processing method according to claim 1, wherein the tangible object is an image on a planar object plane.

8. The image processing method according to claim 1, wherein the projection plane is parallel to the tangible object plane.

9. The image processing method according to claim 1, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

10. The image processing method according to claim 1, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

11. The image processing method according to claim 1, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.

12. An image processing method for correcting image distortions, comprising the steps of:

inputting a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

determining a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and determining a matched point of one of the other partially overlapping images corresponding to the feature point of said one of the plurality of partially overlapping images so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

selecting one of the plurality of partially overlapping images as a standard image that contains a smallest amount of image distortions among the plurality of partially overlapping images; and

combining the other partially overlapping images, which are projected onto an image surface of the standard image with respect to each of the other partially overlapping images, so that a composite image is generated on the image surface so as to correct image distortions in the standard image.

13. The image processing method of claim 12, wherein said other partially overlapping images are projected with a perspective projection matrix operation.

14. The image processing method of claim 13, wherein a least-square-method is used to find parameters of said perspective projection matrix.

15. The image processing method according to claim 12 wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

16. The image processing method according to claim 12, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

17. The image processing method according to claim 12, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.

18. An image processing apparatus for correcting image distortions, comprising:

an input unit inputting a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

a correspondence detecting unit determining a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and determining a matched point of one of the other partially overlapping images corresponding to the feature point of said one of the plurality of partially overlapping images so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

a standard image setting unit selecting one of the plurality of partially overlapping images as a standard image that contains a smallest amount of image distortions among the plurality of partially overlapping images; and

an image composition unit combining the other partially overlapping images, which are projected onto an image surface of the standard image with respect to each of the other partially overlapping images, so that a composite image is generated on the image surface so as to correct image distortions in the standard image.

19. The image processing apparatus according to claim 18, wherein said standard image setting unit is configured such that a user is required to select the standard image when taking the original image from one of the oblique directions, and wherein said image processing apparatus further comprises a notification unit which notifies the user that the standard image is currently taken.

20. The image processing apparatus of claim 18, wherein said other partially overlapping images are projected with a perspective projection matrix operation.

21. The image processing apparatus of claim 18, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

22. The image processing apparatus of claim 18, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

23. The image processing apparatus of claim 18, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.

24. An image processing apparatus for correcting image distortions, comprising:

an input unit inputting a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

a correspondence detecting unit determining a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and determining a matched point of one of the other partially overlapping images corresponding to the feature point so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

a standard image setting unit selecting one of the plurality of partially overlapping images as a standard image whose image distortions are to be corrected, the selection being based on one of (1) a direction of a straight-line pattern contained in each image, (2) the feature point and the matched point determined by the correspondence detecting unit, and (3) a calculated direction of the object plane for each of the partially overlapping images; and

a distortion correcting unit generating a distortion-corrected image on a projection plane by projecting the standard image onto the projection plane based on the direction of the tangible object plane such that image distortions in the standard image are eliminated.

25. The image processing apparatus according to claim 24, further comprising a plurality of imaging units which respectively input the plurality of partially overlapping images that are generated by taking the tangible object from the oblique directions.

26. The image processing apparatus according to claim 24, wherein said standard image setting unit is configured such that one of the plurality of partially overlapping images is automatically selected as the standard image based on the direction of the straight-line pattern contained in each image.

27. The image processing apparatus according to claim 24, wherein said standard image setting unit is configured such that one of the plurality of partially overlapping images is automatically selected as the standard image based on the feature point and the matched point determined by said correspondence detecting unit.

28. The image processing apparatus according to claim 24, wherein said standard image setting unit is configured such that one of the plurality of partially overlapping images is automatically selected as the standard image based on the calculated direction of the object plane for each of the partially overlapping images.

29. The image processing apparatus of claim 24, wherein said standard image is projected with a perspective projection matrix operation.

30. The image processing apparatus of claim 24, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

31. The image processing apparatus of claim 24, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

32. The image processing apparatus of claim 24, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.

33. A computer-readable storage medium storing program code instructions for causing a computer to execute an image distortion correction processing to correct image distortions, comprising:

first program code means for causing the computer to input a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

second program code means for causing the computer to determine a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and to determine a matched point of one of the other partially overlapping images corresponding to the feature point so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

third program code means for causing the computer to select one of the plurality of partially overlapping images as a standard image whose image distortions are to be corrected, the selection being based on one of (1) a direction of a straight-line pattern contained in each image, (2) the feature point and the matched point determined by the second program code means, and (3) a calculated direction of the object plane for each of the partially overlapping images; and

fourth program code means for causing the computer to generate a distortion-corrected image on a projection plane by projecting the standard image onto the projection plane based on the direction of the tangible object plane such that image distortions in the standard image are eliminated.

34. The computer readable storage medium of claim 33, wherein said distortion-corrected image is generated with a perspective projection matrix operation.

35. The computer readable storage medium of claim 33, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

36. The computer readable storage medium of claim 33, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

37. The computer readable storage medium of claim 33, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.

38. A computer-readable storage medium storing program code instructions for causing a computer to execute an image distortion correction processing to correct image distortions, comprising:

first program code means for causing the computer to input a plurality of partially overlapping images of a tangible object on an object plane, the plurality of partially overlapping images sharing a common location of the tangible object and being created by capturing the tangible object on the tangible object plane from different directions to the tangible object plane;

second program code means for causing the computer to determine a feature point of one of the plurality of partially overlapping images corresponding to the common location of the tangible object, and to determine a matched point of one of the other partially overlapping images corresponding to the feature point of said one of the plurality of partially overlapping images so that a direction of the tangible object plane is calculated based on the feature point and the matched point, the tangible object plane being defined by a spatial orientation of the tangible object;

third program code means for causing the computer to select one of the plurality of partially overlapping images as a standard image that contains a smallest amount of image distortions among the plurality of partially overlapping images; and

fourth program code means for causing the computer to combine the other partially overlapping images, which are projected onto an image surface of the standard image with respect to each of the other partially overlapping images, so that a composite image is generated on the image surface so as to correct image distortions in the standard image.

39. The computer readable storage medium of claim 38, wherein said other partially overlapping images are projected with a perspective projection matrix operation.

40. The computer readable storage medium of claim 38, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane using a same imaging unit.

41. The computer readable storage medium of claim 38, wherein the plurality of partially overlapping images are created by capturing the tangible object on the object plane at different time instants using a same image unit.

42. The computer readable storage medium of claim 38, wherein the plurality of partially overlapping images are created by moving a same imaging unit to capture the tangible object on the object plane from different directions to the tangible object plane.