Super-resolution device and method

• US 8,014,632 B2
• Filed: 07/26/2007
• Issued: 09/06/2011
• Est. Priority Date: 10/10/2006
• Status: Expired due to Fees

• Current Assignee: Toshiba Visual Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Ida, Takashi, Matsumoto, Nobuyuki, Takeshima, Hidenori, Isogawa, Kenzo, Taguchi, Yasunori
• Primary Examiner(s): Repko; Jason M
• Assistant Examiner(s): Thirugnanam; Gandhi
• Application Number:
  US11/828,397
• Publication Number:
  US 20080107356A1
• Time in Patent Office: 1,503 Days
• Field of Search: 382299-300
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4069 by subpixel displacements
• CPC Class Codes: H04N 7/0125 one of the standards being a high definition standard

Abstract

A super-resolution device and method for setting at least one of a plurality of pixels included in image data as target pixels, the image data including pixels arranged in a screen and pixel values representing brightness, an area including the target pixel and peripheral pixels as a target area, and an area for searching pixel value change patterns in the target pixel area; calculating a difference between a first change pattern and second change pattern; comparing a difference between the first and second change patterns; calculating a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of a decimal-accuracy-vector, an extrapolated vector, and pixel values obtained from the image data.

Claims

1. A super-resolution device, comprising:

a candidate area setting unit configured to set

at least one of a plurality of pixels included in image data as a target pixel, the image data including the plurality of pixels arranged in a screen and corresponding pixel values representing pixel brightness,

an area including the target pixel and pixels in the periphery of the target pixel as a target pixel area within the screen, and

a search area for searching a plurality of change patterns of pixel values of the pixels included in the target pixel area within the screen;

a matching difference calculating unit configured to calculate at least one difference between

a first change pattern of the pixel values of the pixels included in the target pixel area, and

a second change pattern of the pixel values of the pixels included in the search area, said pixels included in the search area including a searched pixel and the pixels in the periphery of the searched pixel;

a difference comparing unit configured to compare at least one difference between the first and second change patterns to obtain a first pixel position with a first minimum difference and a second pixel position in a periphery of the first pixel position with a second difference thereof;

a memory configured to store the first pixel position and the first minimum difference thereof, the second pixel position and the second difference thereof;

a decimal-accuracy-vector calculating unit configured to calculate a position with a minimum difference in the search area with a decimal accuracy on the basis of the first pixel position and the first minimum difference, and the second pixel position and the second difference stored in the memory, and to calculate a decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

an extrapolated vector calculating unit configured to calculate an extrapolated vector of the decimal-accuracy-vector from the position with the minimum difference to a pixel on the screen which is not included in the search area using the decimal-accuracy-vector; and

a super-resolution pixel value calculating unit configured to calculate a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of the decimal-accuracy-vector, the extrapolated vector, and pixel values obtained from the image data.

2. A super-resolution device, comprising:

a candidate area setting unit that configured to set

at least one of a plurality of pixels included in image data as a target pixel, the image data including the plurality of pixels arranged in a screen and corresponding pixel values representing pixel brightness,

an area including the target pixel and pixels in the periphery of the target pixel as a target pixel area within the screen, and

first and second search areas for searching a plurality of change patterns of pixel values of the pixels included in the target pixel area within the screen;

a matching difference calculating unit configured to calculate first and second differences between a first change pattern of the pixel values of the pixels included in the target pixel area and first and second change patterns of pixel values of the pixels included in the first and second search areas, the pixels included in the first and second search areas including first and second searched pixels and pixels in the periphery of the first and second searched pixels;

a difference comparing unit configured to compare differences of the first and second change patterns to obtain first and second pixel positions with first and second minimum differences and a third pixel position in the periphery of the first searched pixel with a third difference thereof and a fourth pixel position in the periphery of the second searched pixel with a fourth difference thereof;

a memory configured to store the first pixel position and the first minimum difference thereof, the second pixel position and the second minimum difference thereof, the third pixel position and the third difference thereof, the fourth pixel position and the fourth difference thereof calculated by the difference comparing unit;

a decimal-accuracy-vector calculating configured to calculate a position with a minimum difference in the first and second search areas with a decimal accuracy on the basis of the first pixel position and the first difference thereof, the second pixel position and the second difference thereof, the third pixel position and the third difference thereof, and the fourth pixel position and the fourth difference thereof stored in the memory, and calculate a first decimal-accuracy-vector and a second decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

an interpolated vector calculating unit configured to calculate an interpolated vector of the first and second decimal-accuracy-vectors from the position with the minimum distance to a pixel on the screen which is not included in the first and second search areas using the first and second decimal-accuracy-vectors; and

a super-resolution pixel value calculating unit configured to calculate a pixel value of a super-resolution image having number of pixels larger than the number of pixels included in the image data on the basis of the decimal-accuracy-vector, the interpolated vector and pixel values obtained from the image data.

3. A super-resolution device comprising:

a candidate area setting unit configured to set

at least one of a plurality of pixels included in image data as a target pixel, the image data including the plurality of pixels arranged in a screen and corresponding pixel values representing pixel brightness,

an area including the target pixel and pixels in the periphery of the target pixel as target pixel area within the screen, and

a search area for searching a plurality of change patterns of pixel values of the pixels included in the target pixel area within the screen;

a matching difference calculating unit configured to calculate a difference between a first change pattern of pixel values of pixels included in the target pixel area and a second change pattern of pixel values of pixels included in the search area, the pixels included in the search area including a searched pixel in the search area and the pixels in the periphery of the searched pixel;

a difference comparing unit configured to compare at least one difference between the first and second change patterns to obtain a first pixel position with a first minimum difference and a second pixel position in a periphery of the first pixel position and a second difference thereof;

a memory configured to store the first pixel position and the first minimum difference thereof, the second pixel position and the second difference thereof calculated by the difference comparing unit;

a decimal-accuracy-vector calculating unit configured to calculate a position with a minimum difference in the search area with a decimal accuracy on the basis of the first pixel position and the first minimum difference thereof and the second pixel position and the second difference thereof stored in the memory, and calculate a decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

a congruent vector calculating unit configured to calculate a congruent vector of the decimal-accuracy-vector from the position with the minimum difference to a pixel on the screen which is not included in the search area using the decimal-accuracy-vector; and

a super-resolution pixel value calculating unit configured to calculate a pixel value of a super-resolution image having number of pixels larger than number of pixels included in the image data on the basis of the decimal-accuracy-vector, the congruent vector, and pixel values obtained from the image data.

4. A super-resolution method, comprising:

setting at least one of a plurality of pixels included in an image data as a target pixel, the image data including the plurality of pixels arranged in a screen and corresponding pixel values representing brightness,

setting an area within the screen including the target pixel and pixels in the periphery of the target pixel as a target pixel area,

setting a search area for searching a plurality of change patterns of the pixel values of the pixels included in the target pixel area within the screen;

calculating a difference between a change pattern of pixel values of pixels included in the target pixel area and a change pattern of pixel values of pixels included in the search area, the pixels in the search area including a searched pixel in the search area and pixels in the periphery of the searched pixel;

comparing differences of the change patterns of the respective pixels in the search area to obtain a first pixel position with a minimum difference and a second pixel position in the periphery of the first pixel with a second difference thereof;

storing the first pixel position and the first difference thereof, the second pixel position and a second difference thereof;

calculating a position with a minimum difference in the search area with a decimal accuracy on the basis of the first pixel position and the first difference thereof and the second pixel position and the second difference thereof, and calculating a decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

calculating an extrapolated vector of the decimal-accuracy-vector from the position with the minimum distance to a pixel on the screen which is not included in the search area using the decimal-accuracy-vector; and

calculating a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of the decimal-accuracy-vector, the extrapolated vector, and pixel values obtained from the image data.

5. A super-resolution method, comprising:

setting at least one of a plurality of pixels included in an image data as a target pixel, the image data including the plurality of pixels arranged in a screen and corresponding pixel values representing the pixel brightness,

setting an area including the target pixel and pixels in the periphery of the target pixel as a target pixel area,

setting first and second search areas for searching a plurality of change patterns of the pixel values of the pixels included in the target pixel area within the screen;

calculating first and second differences between a change pattern of the pixel values of the pixels included in the target pixel area and first and second change patterns of pixel values of pixels included in the first and second areas, the pixels included with the first and second areas including the first and second searched pixels in the first and second search areas and pixels in the periphery of the first and second searched pixels;

comparing differences of the first and second change patterns of the respective pixels in the first and second search areas to obtain first and second pixel positions with a minimum difference, a third pixel position in the periphery of the first pixel with a third difference and a fourth pixel position in the periphery of the second pixel with a fourth difference by a difference comparing unit;

storing the first pixel position and the first difference thereof, the second pixel position and the second difference thereof, the third pixel position and the third difference thereof, the fourth pixel position and the fourth difference thereof calculated;

calculating a position with the minimum difference in the first and second search areas with a decimal accuracy on the basis of the first pixel position and the first difference thereof, the second pixel position and the second difference thereof, the third pixel position and the third difference thereof, and the fourth pixel position and the fourth difference thereof stored in the memory, and calculating a first decimal-accuracy-vector and a second decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

calculating an interpolated vector of the first and second decimal-accuracy-vectors from the position with the minimum distance to a pixel on the screen which is not included in the first and second search areas by using the first and second decimal-accuracy-vectors; and

calculating the pixel value of the super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of the decimal-accuracy-vector, the interpolated vector and pixel values obtained from the image data.

6. A super-resolution method comprising:

setting at least one of a plurality of pixels included in image data as a target pixel, the image data including the plurality of pixels arranged in a screen and pixel corresponding values representing pixel brightness,

setting an area including the target pixel and pixels in the periphery of the target pixel as a target pixel area,

setting a search area for searching a plurality of change patterns of the pixel values of the pixels included in the target pixel area within the screen;

calculating a difference between the change pattern of the pixel values of the pixels included in the target pixel area and the change pattern of the pixel values of the pixels included in the search area, the pixels included in the search area including the searched pixel in the search area and the pixels in the periphery of the searched pixels for the respective pixels in the search area;

comparing differences of the change patterns of the respective pixels in the search area calculated by the matching difference calculating unit to obtain a first pixel position with a minimum difference by a difference comparing unit;

storing the first pixel position and a first difference thereof, a second pixel position in the periphery of the first pixel and a second difference thereof in a memory;

calculating a position with the minimum difference in the search area with a decimal accuracy on the basis of the first pixel position and the first difference thereof and the second pixel position and the second difference thereof stored in the memory, and calculating a decimal-accuracy-vector starting from the target pixel and terminating at the position with the minimum difference;

calculating a congruent vector of the decimal-accuracy-vector from the position with the minimum difference to a pixel on the screen which is not included in the search area using the decimal-accuracy-vector; and

calculating a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of the decimal-accuracy-vector, the congruent vector, and pixel values obtained from the image data.