Image processing apparatus and program

• US 8,792,743 B2
• Filed: 01/27/2011
• Issued: 07/29/2014
• Est. Priority Date: 02/03/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Komatsu, Satoru, Kawai, Takashi, Machida, Jun-ichi
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US13/576,347
• Publication Number:
  US 20120301042A1
• Time in Patent Office: 1,279 Days
• Field of Search: None
• US Class Current: 382/254
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 1/60 Colour correction or control H04N1/54 takes precedence
• CPC Class Codes: H04N 1/6072 adapting to different types of images, e.g. characters, graphs, black and white image portions
• CPC Class Codes: H04N 1/628 Memory colours, e.g. skin or sky

Abstract

An image processing apparatus that can be applied to a wide range of general images and that is capable of improving shitsukan of materials depicted in an image by interactively adjusting multiple physical image characteristics of an image based on human latent sensations. Physical image-characteristic parameters are extracted from an input image. The image-characteristic parameters are processed using a first numerical conversion equation, and various low-order-sensitivity parameters quantifying general latent sensations are calculated. The low-order-sensitivity parameters are processed using a second numerical conversion equation, and high-order-sensitivity parameters quantifying shitsukan for different materials are calculated. An output image of which the shitsukan in the input image are improved by combining various types of image processing for changing the image characteristics so that the high-order-sensitivity parameters approach predetermined values is generated.

Claims

1. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to various different low-order sensitivity parameters quantifying latent sensations experienced by humans when viewing an image,

wherein the various different low-order sensitivity parameter calculated from the output image are converted by the first numerical converting unit to approach numerical values satisfying a predetermined relationship; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

3. The image processing apparatus according to claim 1, wherein the image characteristics parameter of the output image, the low-order-sensitivity parameters of the output image and the output image are displayable on one screen.

4. The image processing apparatus according to claim 1, wherein the image characteristics parameter of the output image, the high-order-sensitivity parameters of the output image and the output image are displayable on one screen.

2. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image,

wherein the low-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit to approach predetermined numerical values; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

5. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the numerical converting unit to approach predetermined numerical values.

6. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to various different low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image,

wherein the various different low-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit to approach numerical values satisfying a predetermined relationship; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

7. The program according to claim 6, wherein the image characteristics parameter of the output image, the low-order-sensitivity parameters of the output image and the output image are displayable on one screen.

8. The program according to claim 6, wherein the image characteristics parameter of the output image, the high-order-sensitivity parameters of the output image and the output image are displayable on one screen.

9. A method of producing the program according to claim 6, comprising the steps of:

preparing a plurality of evaluation images having various different physical image characteristics independently or interactively changed from the physical image characteristics of an original image depicting a material having specific shitsukan by performing image processing;

measuring degrees of contribution of various different latent sensations to the individual image characteristics through a subject performing comparative examination of the plurality of evaluation images and evaluating a level of sensuous change in the various different latent sensations experienced by humans when viewing an image; and

acquiring the first numerical converting unit by performing statistical processing on measurement results of the degrees of contribution of the various different latent sensations to the individual image characteristics.

10. The method of producing the program according to claim 9, wherein, in the measuring step, the subject is a plurality of subjects.

11. The method of producing the program according to claim 9,

wherein, in the preparing step, the evaluation images are displayed on a display medium on which the output image is displayed, and

wherein the measuring step is performed using the evaluation images displayed on the display medium.

12. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to various different low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image,

wherein the low-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit to approach predetermined numerical values; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

13. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein the high-order-sensitivity parameters calculated from the output image are converted by the numerical converting unit to approach predetermined numerical values.

14. A method of producing the program according to claim 13, comprising the steps of:

preparing a plurality of evaluation images having various different physical image characteristics independently or interactively changed from the physical image characteristics of an original image depicting a material having specific shitsukan by performing image processing;

measuring degrees of contribution of various different latent sensations to the individual image characteristics through a subject performing comparative examination of the evaluation images and evaluating an level of sensuous change in shitsukan of different materials depicted in an image; and

acquiring the numerical converting unit by performing statistical processing on measurement results of the degrees of contribution of the shitsukan to the individual image characteristics.

15. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristic extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to various different low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image; and

a selecting unit configured to select an image region of a specific material in the input image,

wherein the various different low-order-sensitivity parameters calculated from the image region in the output image selected by the selecting unit are converted by the first numerical converting unit to approach numerical values satisfying a predetermined relationship; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein, with respect to the image region selected by the selecting unit, the high-order-sensitivity parameters calculated from the image region of the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

16. The image processing apparatus according to claim 15, further comprising:

a display device configured to display at least one of the input image and the output image,

wherein the low-order-sensitivity parameters are displayed at a position near the image region displayed on the display device.

17. The image processing apparatus according to claim 15, further comprising:

a display device configured to display at least one of the input image and the output image,

wherein the high-order-sensitivity parameters are displayed at a position near the image region displayed on the display device.

18. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image; and

a selecting unit configured to select an image region of a specific material in the input image,

wherein the various different low-order-sensitivity parameters calculated from the image region in the output image selected by the selecting unit are converted by the first numerical converting unit to approach predetermined numerical values; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein, with respect to the image region selected by the selecting unit, the high-order-sensitivity parameters calculated from the image region of the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

19. An image processing apparatus configured to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image; and

a selecting unit configured to select an image region of a specific material in the input image,

wherein the high-order-sensitivity parameters calculated from the image region in the output image selected by the selecting unit are converted by the numerical converting unit to approach predetermined numerical values.

20. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristic extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to various different low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image; and

a selecting unit configured to select an image region of a specific material in the input image,

wherein the various different low-order-sensitivity parameters calculated from the image region of the output image selected by the selecting unit are converted by the first numerical converting unit to approach numerical values satisfying a predetermined relationship; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein, with respect to the image region selected by the selecting unit, the high-order-sensitivity parameters calculated from the image region of the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

21. The program according to claim 20, further comprising a step of:

displaying at least one of the input image and the output image on a display device,

wherein the low-order-sensitivity parameters are displayed at a position near the image region displayed on the display device.

22. The program according to claim 20, further comprising a step of:

displaying at least one of the input image and the output image on a display device,

wherein the high-order-sensitivity parameters are displayed at a position near the image region displayed on the display device.

23. A method of producing the program according to claim 20, comprising the steps of:

preparing a plurality of evaluation images having various different physical image characteristics independently or interactively changed from the physical image characteristics of an original image depicting a material having specific shitsukan by performing image processing;

measuring degrees of contribution of various different latent sensations to the individual image characteristics through a subject performing comparative examination of the plurality of evaluation images and evaluating a level of sensuous change in the various different latent sensations experienced by humans when viewing an image; and

acquiring the first numerical converting unit by performing statistical processing on measurement results of the degrees of contribution of the various different latent sensations to the individual image characteristics.

24. The method of producing the program according to claim 23, wherein, in the measuring step, the subject is a plurality of subjects.

25. The method of producing the program according to claim 23,

wherein, in the preparing step, the evaluation images are displayed on a display medium on which the output image is displayed, and

wherein the measuring step is performed using the evaluation images displayed on the display medium.

26. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristics extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a first numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to low-order-sensitivity parameters quantifying latent sensations experienced by humans when viewing an image; and

a selecting unit configured to select an image region of a specific material in the input image,

wherein the low-order-sensitivity parameters calculated from the image region of the output image selected by the selecting unit are converted by the first numerical converting unit to approach predetermined numerical values; and

a second numerical converting unit configured to be represented by the hierarchical model and to convert the various different low-order-sensitivity parameters to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image,

wherein, with respect to the image region selected by the selecting unit, the high-order-sensitivity parameters calculated from the image region of the output image are converted by the first numerical converting unit and the second numerical converting unit to approach predetermined numerical values.

27. A program encoded on a non-transitory computer readable medium instructing a computer to convert an input image to generate an output image having image shitsukan different from the image shitsukan of the input image by using an adjusting unit capable of individually adjusting various different physical image characteristic extracted from the input image and interactively adjusting the various different physical image characteristics, the apparatus comprising:

a numerical converting unit configured to be represented by a hierarchical model having a first level defining a relationship between the image characteristics and latent sensation received by humans from an image and a second level defining a relationship between the latent sensation and the shitsukan, in correspondence with human shitsukan recognition structure, the first level and the second level being different from each other, and configured to convert various different image-characteristic parameters quantifying the image characteristics to high-order-sensitivity parameters quantifying shitsukan for different materials depicted in an image; and

a selecting unit configured to select an image region of a target material in the input image,

wherein the high-order-sensitivity parameters calculated from the image region of the output image selected by the selecting unit are converted by the numerical converting unit to approach predetermined numerical values.

28. A method of producing the program according to claim 27, comprising the steps of:

preparing a plurality of evaluation images having various different physical image characteristics independently or interactively changed from the physical image characteristics of an original image depicting a material having specific shitsukan by performing image processing;

measuring degrees of contribution of various different latent sensations to the individual image characteristics through a subject performing comparative examination of the evaluation images and evaluating an level of sensuous change in shitsukan of different materials depicted in an image; and

acquiring the numerical converting unit by performing statistical processing on measurement results of the degrees of contribution of the shitsukan to the individual image characteristics.