Image processing device, display device, image processing method, program, and storage medium

• US 8,358,307 B2
• Filed: 12/10/2008
• Issued: 01/22/2013
• Est. Priority Date: 04/21/2008
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Shiomi, Makoto, Sakai, Tamotsu
• Primary Examiner(s): Thompson, James A
• Assistant Examiner(s): Le, Sarah
• Application Number:
  US12/863,212
• Publication Number:
  US 20110043526A1
• Time in Patent Office: 1,504 Days
• Field of Search: 345/428, 345/611, 345/606, 382/164, 382/173, 382/266, 382/269
• US Class Current: 345/428
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard
• CPC Class Codes: H04N 7/0142 the interpolation being edge adaptive edge detection in television signals H04N5/142; edge-driven scaling G06T3/403

Abstract

Upscaling circuits include a difference circuit which calculates a gradation value of a target pixel with use of differentiation or difference of gradation values in vicinity to the target pixel, so that an edge of an article pictured in an image can be extracted; an averaging circuit which calculates a gradation value of a target pixel by averaging gradation values in vicinity to the target pixel; and a correlation operation circuit which calculates a correlation value indicating a correlation between difference image data and averaged image data, the difference image data being produced by carrying out the difference operation process on the divided image data, and the averaged image data being produced by carrying out the difference operation process and the averaging process on the divided image data. Further, an interpolation process is carried out on the divided image data with an interpolation method using the calculated correlation value.

Claims

1. An image processing device comprising:

a dividing process section which divides inputted image data into a plural pieces of divided image data; and

a plurality of upscaling process sections which enhance resolution of each of the plural pieces of divided image data, respectively, wherein the dividing process section producing the plural pieces of divided image data so that each one of the plural pieces of divided image data shares a portion with its adjacent one of the plural pieces of divided image data, so that border areas of that one and its adjacent one of the plural pieces of divided image data overlap each other,

each of the plurality of upscaling process sections including;

a difference operation section which carries out a difference operation process in which a gradation value of a target pixel is calculated by an operation with use of differentiation or difference of gradation values in vicinity to the target pixel, the gradation value of the target pixel being used for extracting an edge of an article pictured in an image,

an averaging process section which carries out an averaging process in which an average value of the gradation values in vicinity to the target pixel is calculated as a gradation value of the target pixel;

a correlation operation section which carries out calculation process for obtaining a correlation value indicative of a correlation between difference image data and averaged image data, the difference image data being produced by subjecting the plural pieces of divided image data to the difference operation process, and the averaged image data being produced by subjecting the plural pieces of divided image data to the difference operation process and the averaging process;

an interpolation process section which carries out an interpolation process on the plural pieces of divided image data with use of an interpolation method which is selected in accordance with the correlation value;

each of the plurality of upscaling process sections includes an edge-distinguishing section which distinguishes, by comparing the correlation value in each pixel with a predetermined threshold value, an edge part from a non-edge part contained in each of the plural pieces of divided image data;

the interpolation process section carries out the interpolation process on the edge part with an interpolation method with which the edge part stands out more than the non-edge part; and

the difference operation section carries out (i) a horizontal difference operation process for extracting a horizontal edge contained in each of the plural pieces of divided image data and (ii) a vertical difference operation process for extracting a vertical edge contained in each of the plural pieces of divided image data;

when a target pixel is judged as an edge part, the edge-distinguishing section calculates an inclined angle of the edge, the inclined angle of edge being a ratio between a gradation value of a target pixel after the horizontal difference operation process and a gradation value of the target pixel after the vertical difference operation process;

the interpolation process section carries out interpolation process on an interpolation pixel in the edge part with use of gradation values of pixels which are adjacent to the interpolation pixel and selected in the order of distance from a straight line which passes through the interpolation pixel and extends in parallel with an inclining direction of the edge, the number of the pixels to be selected being equal to or smaller than a predetermined number; and

the interpolation process section carries out interpolation process on an interpolation pixel in the non-edge part with use of gradation values of pixels which are adjacent to the interpolation pixel.

2. The image processing device as set forth in claim 1, wherein:

each of the plurality of upscaling process sections carries out the difference operation process, the averaging process, the calculation process of the correlation value, and the interpolation process, concurrently with the other of the plurality of upscaling process sections.

3. The image processing device as set forth in claim 1, wherein:

the difference operation section carries out the difference operation process on a block of 5×

5 pixels which block is centered on a target pixel.

4. The image processing device as set forth in claim 1, wherein:

the portion included in the border areas is image data of 2 lines or more but 5 lines or less.

5. An image processing device comprising:

a dividing process section which divides inputted image data into a plural pieces of divided image data; and

a plurality of upscaling process sections which enhance resolution of each of the plural pieces of divided image data, respectively, wherein

the dividing process section producing the plural pieces of divided image data so that each one of the plural pieces of divided image data shares a portion with its adjacent one of the plural pieces of divided image data, so that border areas of that one and its adjacent one of the plural pieces of divided image data overlap each other,

each of the plurality of upscaling process sections including;

a difference operation section which carries out a difference operation process in which a gradation value of a target pixel is calculated by an operation with use of differentiation or difference of gradation values in vicinity to the target pixel, the gradation value of the target pixel being used for extracting an edge of an article pictured in an image,

an averaging process section which carries out an averaging process in which an average value of the gradation values in vicinity to the target pixel is calculated as a gradation value of the target pixel;

a correlation operation section which carries out calculation process for obtaining a correlation value indicative of a correlation between difference image data and averaged image data, the difference image data being produced by subjecting the plural pieces of divided image data to the difference operation process, and the averaged image data being produced by subjecting the plural pieces of divided image data to the difference operation process and the averaging process; and

an interpolation process section which carries out an interpolation process on the plural pieces of divided image data with use of an interpolation method which is selected in accordance with the correlation value;

each of the plurality of upscaling process sections includes a compensation coefficient calculation section which calculates a compensation coefficient in accordance with the correlation value; and

the interpolation process section carries out, on each interpolation pixel, (i) a first interpolation process for extracting an edge and (ii) a second interpolation process for obtaining continuity of gradation at all azimuths, and then the interpolation process section carries out an interpolation process on the plural pieces of divided image data in which process a result of the first interpolation process and a result of the second interpolation process are weighted in accordance with the compensation coefficient and then the weighted results are added up.

6. A display device comprising:

an image processing device comprising;

a dividing process section which divides inputted image data into a plural pieces of divided image data; and

a plurality of upscaling process sections which enhance resolution of each of the plural pieces of divided image data, respectively;

wherein

the dividing process section producing the plural pieces of divided image data so that each one of the plural pieces of divided image data shares a portion with its adjacent one of the plural pieces of divided image data, so that border areas of that one and its adjacent one of the plural pieces of divided image data overlap each other;

each of the plurality of upscaling process sections including;

a difference operation section which carries out a difference operation process in which a gradation value of a target pixel is calculated by an operation with use of differentiation or difference of gradation values in vicinity to the target pixel, the gradation value of the target pixel being used for extracting an edge of an article pictured in an image;

an averaging process section which carries out an averaging process in which an average value of the gradation values in vicinity to the target pixel is calculated as a gradation value of the target pixel;

a correlation operation section which carries out calculation process for obtaining a correlation value indicative of a correlation between difference image data and averaged image data, the difference image data being produced by subjecting the plural pieces of divided image data to the difference operation process, and the averaged image data being produced by subjecting the plural pieces of divided image data to the difference operation process and the averaging process;

an interpolation process section which carries out an interpolation process on the plural pieces of divided image data with use of an interpolation method which is selected in accordance with the correlation value;

each of the plurality of upscaling process sections includes an edge-distinguishing section which distinguishes, by comparing the correlation value in each pixel with a predetermined threshold value, an edge part from a non-edge part contained in each of the plural pieces of divided image data;

the interpolation process section carries out the interpolation process on the edge part with an interpolation method with which the edge part stands out more than the non-edge part; and

the difference operation section carries out (i) a horizontal difference operation process for extracting a horizontal edge contained in each of the plural pieces of divided image data and (ii) a vertical difference operation process for extracting a vertical edge contained in each of the plural pieces of divided image data;

when a target pixel is judged as an edge part, the edge-distinguishing section calculates an inclined angle of the edge, the inclined angle of edge being a ratio between a gradation value of a target pixel after the horizontal difference operation process and a gradation value of the target pixel after the vertical difference operation process;

the interpolation process section carries out interpolation process on an interpolation pixel in the edge part with use of gradation values of pixels which are adjacent to the interpolation pixel and selected in the order of distance from a straight line which passes through the interpolation pixel and extends in parallel with an inclining direction of the edge, the number of the pixels to be selected being equal to or smaller than a predetermined number; and

the interpolation process section carries out interpolation process on an interpolation pixel in the non-edge part with use of gradation values of pixels which are adjacent to the interpolation pixel; and

a display section which displays an image which is upscaled by the image processing device.

7. An image processing method comprising the steps of:

(a) dividing inputted image data into a plural pieces of divided image data; and

(b) enhancing a resolution of each of the plural pieces of divided image data, respectively, with a plurality of upscaling process sections;

wherein

in the step (a), the plural pieces of divided image data being produced so that each one of the plural pieces of divided image data shares a portion with its adjacent one of the plural pieces of divided image data, so that border areas of that one and its adjacent one of the plural pieces of divided image data overlap with each other,

each of the plurality of upscaling process sections carrying out;

a difference operation process in which a gradation value of a target pixel is calculated by an operation with use of differentiation or difference of gradation values in vicinity to the target pixel, the gradation value of the target pixel being used for extracting an edge of an article pictured in an image;

an averaging process in which an average value of the gradation values in vicinity to the target pixel is calculated as a gradation value of the target pixel;

a calculation process for obtaining a correlation value indicative of a correlation between difference image data and averaged image data, the difference image data being produced by subjecting the plural pieces of divided image data to the difference operation process, and the averaged image data being produced by subjecting the plural pieces of divided image data to the difference operation process and the averaging process;

an interpolation process on the plural pieces of divided image data with use of an interpolation method which is selected in accordance with the correlation value;

each of the plurality of upscaling process sections includes an edge-distinguishing section which distinguishes, by comparing the correlation value in each pixel with a predetermined threshold value, an edge part from a non-edge part contained in each of the plural pieces of divided image data;

the interpolation process is carried out on the edge part with an interpolation method with which the edge part stands out more than the non-edge part;

the difference operation section carries out (I) a horizontal difference operation process for extracting a horizontal edge contained in each of the plural pieces of divided image data and (ii) a vertical difference operation process for extracting a vertical edge contained in each of the plural pieces of divided image data;

when a target pixel is judged as an edge part, the edge-distinguishing section calculates an inclined angle of the edge, the inclined angle of edge being a ratio between a gradation value of a target pixel after the horizontal difference operation process and a gradation value of the target pixel after the vertical difference operation process;

the interpolation process section carries out interpolation process on an interpolation pixel in the edge part with use of gradation values of pixels which are adjacent to the interpolation pixel and selected in the order of distance from a straight line which passes through the interpolation pixel and extends in parallel with an inclining direction of the edge, the number of the pixels to be selected being equal to or smaller than a predetermined number; and

the interpolation process section carries out interpolation process on an interpolation pixel in the non-edge part with use of gradation values of pixels which are adjacent to the interpolation pixel.