Image processing method and apparatus

• US 6,603,885 B1
• Filed: 04/29/1999
• Issued: 08/05/2003
• Est. Priority Date: 04/30/1998
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Enomoto, Jun
• Primary Examiner(s): Lee, Thomas D.
• Assistant Examiner(s): BRINICH, STEPHEN M
• Application Number:
  US09/301,538
• Time in Patent Office: 1,559 Days
• Field of Search: 382/260-269, 382/199-200
• US Class Current: 382/263
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/58 Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction H04N1/62 takes precedence

Abstract

The image processing method and apparatus acquire input image data from the image recorded optically, with a taking lens, acquire the information about the lens used to record the image and perform image processing schemes on the input image data using the acquired lens information, provided that the type of the lens used is identified from the acquired lens information and the intensity of sharpness enhancement of the corresponding image is, altered in accordance with the identified lens type. The characteristics of the taking lens may also be acquired from the lens information and using the obtained lens characteristics as well as the position information for the recorded image, the input image data is subjected to aberration correction for correcting the deterioration in image quality due to the lens characteristics, provided that the information about the focal length of the taking lens effective at the time of image recording is additionally used, or image processing schemes are performed in two crossed directions of the recorded image, or parameters for correcting aberrations in the imaging plane of the taking lens are scaled such that they are related to the output image data on a pixel basis. High-quality prints reproducing high-quality images can be obtained from original images that were recorded with compact cameras, digital cameras and other conventional inexpensive cameras using low-performance lenses.

Claims

1. An image processing method comprising the steps of:

acquiring input image data from an image recorded optically with a lens for taking a picture;

acquiring an information about the lens used to record said image; and

performing image processing schemes including at least sharpness enhancement on said input image data using the acquired information about the lens to produce output image data;

wherein a type of the lens used is identified from said acquired information about the lens and an intensity of the sharpness enhancement of the corresponding image is altered in accordance with the identified type of the lens; and

wherein a type of a film on which said image is recorded is further acquired and the intensity of said sharpness enhancement is altered in accordance with not only said type of the lens but also the acquired type of the film.

2. The image processing method according to claim 1, wherein the intensity of said sharpness enhancement is altered uniformly for an entire part of the image in one frame or, alternatively, the image in said one frame is divided into a plurality of image areas and said intensity is altered for each of the divided image areas.

3. The image processing method according to claim 1, wherein the intensity of said sharpness enhancement is altered independently for each of three primary colors or altered uniformly for said three primary colors.

4. The image processing method according to claim 1, wherein lens characteristics of said taking lens are further acquired from said lens information and using the acquired lens characteristics and a position information for said recorded image, said sharpness enhancement is performed on said input image data at different intensities on a pixel basis.

5. The image processing method according to claim 1, wherein lens characteristics of said taking lens are further acquired from said lens information and using the acquired lens characteristics and a position information for said recorded image, the input image data for the image taken with the lens of the type requiring a change in the intensity of said sharpness enhancement is also subjected to aberration correction for correcting deterioration in image quality derived from said lens characteristics.

6. The image processing method according to claim 5, wherein said aberration correction is for correcting at least one of distortion, chromatic aberration of magnification, deterioration of marginal lumination, and defocusing derived from said lens characteristics.

7. The image processing method according to claim 6, wherein said defocusing is corrected by subjecting the sharpness enhancement to said input image data at different intensities on a pixel basis using said lens characteristics and the position information for said recorded image.

8. The image processing method according to claim 1, wherein lens characteristics of said taking lens are further acquired from said lens information and using the acquired lens characteristics and a position information for said recorded image, at least one of aberration correction for correcting deterioration in image quality derived from said lens characteristics, electronic scaling and sharpness enhancement at different intensities on a pixel basis as said sharpness enhancement is performed on said input image data as one of said image processing schemes.

9. The image processing method according to claim 1, wherein an information about the focal length effective at a time of recording said image is further acquired in addition to said lens information and lens characteristics of said taking lens are acquired from said lens information, and using the acquired lens characteristics, a position information for said recorded image and the information about said focal length, said input image data is subjected to aberration correction for correcting deterioration in image quality derived from said lens characteristics.

10. The image processing method according to claim 9, wherein an information about a diaphragm used to record said image is further acquired and taking account into the obtained diaphragm information, correction of deterioration of marginal lumination is performed on said input image data as said aberration correction.

11. The image processing method according to claim 9, wherein the lens characteristics of said taking lens are calculated in terms of the focal length effective at the time of recording said image using lens characteristics of said taking lens obtained previously at a plurality of focal lengths.

12. The image processing method according to claim 1, wherein lens characteristics of said taking lens are further acquired from said lens information and using the acquired lens characteristics and a position information for said recorded image, said input image data is subjected to the image processing schemes in a first direction of said recorded image and a second direction crossing the first direction.

13. The image processing method according to claim 12, wherein said image processing schemes include at least one of correction of aberrations derived from said taking lens, electronic scaling and said sharpness enhancement.

14. The image processing method according to claim 12, wherein said image processing schemes are performed in said first and second directions independently of each other.

15. The image processing method according to claim 14, wherein an order of said image processing schemes is selectable in said first and second directions.

16. The image processing method according to claim 12, wherein unidirectional image processing is further performed in at least one of said first and second directions.

17. The image processing method according to claim 12, wherein contents of said image processing schemes, are altered in each of said first and second directions.

18. The image processing method according to claim 17, wherein alteration of the contents of said image processing schemes is change in values of parameters in the image processing schemes in each of said first and second directions.

19. The image processing method according to claim 12, wherein if said image processing schemes include at least correction of distortion and chromatic aberration of magnification, either an amount of correction of the chromatic aberration of magnification or an amount of correction of the distortion or both amounts differ between said first and second directions.

20. The image processing method according to claim 12, wherein if said image processing schemes include at least correction of distortion and chromatic aberration of magnification, the correction in whichever of said first and second directions that requires the chromatic aberration of magnification and the distortion to be corrected in smaller amounts precedes the correction in the other direction.

21. The image processing method according to claim 12, wherein if said input image data is acquired by reading photoelectrically said recorded image with line sensors that have said first direction as a main scanning direction and said second direction as an auxiliary scanning direction, said image processing schemes in said second direction further including correction of color divergence caused by said line sensors.

22. The image processing method according to claim 1, wherein if said image processing schemes include aberration correction for correcting aberrations in said image derived from said taking lens, preset parameters for correcting the aberrations which the taking lens used to record said image causes in a plane where said image is focused area scaled with at least one of an electronic scaling ratio for producing said output image data, number of input pixels in said input image data, a size of the input image and a size of the output image to produce aberration correction parameters that are related to said output image data on a pixel basis, and then is performed using said aberration correction to correct the aberrations in said image derived from said taking lens.

23. An image processing apparatus that acquires input image data from an image recorded optically with a lens for taking a picture and performs image processing schemes on said input image data to produce output image data, comprising:

first acquisition means for acquiring a lens information about said taking lens used to record said image;

identifying means for identifying a lens type from the acquired lens information; and

image processing means for performing at least sharpness enhancement of the image;

wherein said image processing means alters an intensity of said sharpness enhancement of the corresponding image in accordance with an identification result of the lens type by said identifying means; and

wherein said image processing means further alters the intensity of said sharpness enhancement in accordance with a film type.

24. The image processing apparatus according to claim 23, wherein said image processing means comprises not only sharpness enhancing means for performing said sharpness enhancement on said image but also storage means for storing lens characteristics related to the lens type and aberration correcting means for receiving the lens characteristics of the corresponding lens type and correcting deterioration in image quality of the image based on a position information of the image and the lens characteristics, wherein correction the deterioration of the image quality by said aberration correcting means is also performed on the image that alters the intensity of said sharpness enhancement.

25. The image processing apparatus according to claim 24, wherein said aberration correcting means corrects at least one of distortion, chromatic aberration of magnification and deterioration of marginal lumination derived from said lens characteristics.

26. The image processing apparatus according to claim 24, wherein said image processing means performs said, sharpness enhancement by said sharpness enhancing means after said correction of the image quality deterioration by said aberration correcting means.

27. The image processing apparatus according to claim 24, wherein said aberration correcting means corrects the chromatic aberration of magnification and the distortion derived from the lens characteristics or further the deterioration of the marginal lumination derived from the lens characteristics, and wherein said aberration correcting means assigns one of three primary colors as a reference color, calculates offsets in image positions of the other colors from the image position of the reference color derived from the chromatic aberration of magnification, uses said offsets derived from the chromatic aberration of magnification and the offset in the image position of said reference color derived from the distortion to calculate appropriate positions of the respective images as corrected not only for the distortion but also for the chromatic aberration of magnification, and corrects the image quality deterioration based on said appropriate positions of the respective images or uses said appropriate positions of the respective images to correct the image quality deterioration and perform electronic scaling.

28. The image processing apparatus according to claim 23, wherein said lens type that requires change in the intensity of said sharpness enhancement of the image is the lens of a single use camera.

29. The image processing apparatus according to claim 23, wherein said image processing means alters the intensity of said sharpness enhancement uniformly for an entire part of the image in one frame or divides the image in said one frame into a plurality of image areas and alters said intensity for each of the divided image areas.

30. The image processing apparatus according to claim 23, wherein said image processing means alters the intensity of said sharpness enhancement independently for each color of three primary colors or uniformly for each color of said three primary colors.

31. The image processing apparatus according to claim 23, further comprising:

second acquisition means for acquiring an information about a focal length of said taking lens effective at the time of recording said image if the taking lens used to record said image is a lens of variable focal length;

wherein said image processing means comprises storage means for storing lens characteristics of the taking lens used to record said image and aberration correcting means for acquiring the lens characteristics of the corresponding lens from said storage means in accordance with the lens information acquired by said first acquisition means and correcting aberrations in the image derived from the taking lens used to record said image using the acquired lens characteristics, the position information of said image and the information about the focal length of said taking lens acquired by said second acquisition means.

32. The image processing apparatus according to claim 31, wherein, when said aberration correcting means corrects deterioration of marginal lumination, said second acquisition means further acquires an information about the diaphragm used at the time of image recording and said aberration correcting means additionally uses said diaphragm information to correct said deterioration of the marginal lumination.

33. The image processing apparatus according to claim 31, wherein said storage means stores, as said lens characteristics, the lens characteristics at a plurality of focal lengths of said lens and said aberration correcting means calculates said lens characteristics at said plurality of the focal lengths in terms of the focal length effective at the time of said image recording that was acquired by said second acquisition means to determine the lens characteristics of said lens at the focal length effective at the time of said image recording.

34. The image processing apparatus according to claim 31, wherein said aberration correcting means assigns one of three primary colors as a reference color, calculates offsets in image positions of the other colors from the image position of the reference color derived from the chromatic aberration of magnification, uses said offsets derived from the chromatic aberration of magnification and the offset in the image position of said reference color derived from the distortion to calculate appropriate positions of the respective images as corrected for both the distortion and the chromatic aberration of magnification, and corrects the distortion and the chromatic aberration of magnification based on said appropriate positions or uses said appropriate positions to perform electronic scaling.

35. The image processing apparatus according to claim 23, further including storage means for storing lens characteristics of said taking lens in accordance with the lens information about said taking lens and wherein said image processing means has bi-directional image processing means for performing the image processing schemes on said input image data in a first direction of the recorded image and a second direction crossing the first direction, using an information about pixel positions of an input,image and the characteristics of the related taking lens as read from said storage means in accordance with the taking.lens information acquired by said first acquisition means.

36. The image processing apparatus according to claim 35, wherein said bi-directional image processing means has at least one of a first and a second distortion correcting part that correct distortion in said first and second directions, respectively;

a first and a second magnification chromatic aberration correcting part that correct chromatic aberration of magnification in said first and second directions, respectively;

a first and a second marginal lumination deterioration correcting part that correct deterioration of marginal lumination in said first and second directions, respectively;

a first and a second defocusing correcting part that correct defocusing in said first and second directions, respectively;

a first and a second electronic scaling part that perform electronic scaling in said first and second directions, respectively; and

a first and a second sharpening part that perform sharpness enhancement in said first and second directions, respectively.

37. The image processing apparatus according to claim 35, wherein said bi-directional image processing means has a first image processing part and a second image processing part that perform image processing schemes independently of each other in said first and second directions, respectively.

38. The image processing apparatus according to claim 37, wherein said bidirectional image processing means is capable of selecting an order of said image processing schemes that are performed by said first and second image processing parts.

39. The image processing apparatus according to claim 35, further including unidirectional image processing means for performing unidirectional image processing in at least one of said first and second directions.

40. The image processing apparatus according to claim 39, wherein if said input image data is acquired by line sensors that read the image recorded on the film, said unidirectional image processing means has a color divergence correcting part that corrects the color divergence derived from said line sensors.

41. The image processing apparatus according to claim 35, further including control means for altering contents of said image processing schemes in each of said first and second directions.

42. The image processing apparatus according to claim 41, wherein said control means alters the contents of said image processing schemes by changing a kind or degree of said image processing schemes.

43. The image processing apparatus according to claim 41, wherein said control means alters the contents of said image processing schemes by changing values of parameters in the image processing schemes in each of said first and second directions.

44. The image processing apparatus according to claim 43, wherein said parameters in said image processing schemes are at least one of a filter coefficient of a filter used a correction coefficient for correction of distortion;

a correction coefficient for the correction of chromatic aberration of magnification;

a correction coefficient for correction of deterioration of marginal lumination;

a correction coefficient for correction of defocusing;

an electronic scaling ratio; and

a coefficient of the sharpness enhancement.

45. The image processing apparatus according to claim 35, wherein if said bi-directional image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, either an amount of correction of chromatic aberration of magnification or an amount of correction of distortion or both amounts in said distortion and magnification chromatic aberration correcting parts differ between said first and second directions.

46. The image processing apparatus according to claim 35, wherein if said bi-directional image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, correction in whichever of said first and second directions that requires the chromatic aberration of magnification and distortion to be corrected in smaller amounts precedes the correction in the other direction.

47. The image processing apparatus according to claim 35 which, if said input image data is acquired by photoelectric reading of said recorded image with line sensors that have said first direction as a main scanning direction and said second direction as an auxiliary scanning direction, further includes a color divergence correcting part that corrects the color divergence derived from said line sensors in said second direction.

48. The image processing apparatus according to claim 23, wherein said image processing means further includes:

storage means for storing parameters for correcting aberrations which the lens used to record said image causes in an imaging plane where said image is focused;

selection means for selecting a parameter which corrects the aberrations that the related taking lens causes on said imaging plane from said storage means in accordance with the lens information acquired by said first acquisition means;

conversion means by which the parameter for correcting the aberrations on the imaging plane as selected by said selection means is scaled with at least one of ant electronic scaling ratio for producing said output image data, number of input pixels in said input image data, size of an input image and the size of the output image, whereby said parameter is converted to an aberration correcting parameter that is related to said output image data on a pixel basis; and

aberration correcting means which corrects the aberrations of the image derived from the image taking lens using the thus obtained, pixel-dependent aberration correcting parameter.

49. The image processing apparatus according to claim 48, wherein said image is one that is recorded on a photographic film and the size of said input image is equal to the size of the image as it is read from said photographic film.