Super-resolution reconstructing method for enhancing smoothness and sharpness of video image

• US 9,208,537 B1
• Filed: 08/15/2014
• Issued: 12/08/2015
• Est. Priority Date: 07/10/2014
• Status: Expired due to Fees

• Current Assignee: Shenzhen China Star Optoelectronics Technology Co., Ltd. (TCL Technology Group Corp.)
• Original Assignee: Shenzhen China Star Optoelectronics Technology Co., Ltd. (TCL Technology Group Corp.)
• Inventors: Li, Hao, Hu, Houliang, Chu, Liwei
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US14/384,658
• Time in Patent Office: 480 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

A super-resolution reconstructing method includes: providing an original image including multiple original image pixel points arranged in an array; performing edge detecting on the original image by Canny_H method and considering pixel edge information of 4*4 area image range; performing 2*2 times interpolation zooming to edge pixel points and determining a type of the 4*4 area image as pure color entity area or non-pure color entity area according to the pixel edge information; performing a conditional interpolation individually on pure color entity area and non-pure color entity area; and obtaining a pixel 2*2 times interpolation zoomed super-resolution image. In pure color entity area, interpolation is performed according to edge information of 4*4 area image range to maintain border sharpness of pure color entity area and reduce jaggies. In non-pure color entity area, BiCubic_H interpolation is performed to enhance smoothness and sharpness of image.

Claims

1. A super-resolution reconstructing method for enhancing smoothness and sharpness of video image, comprising:

step 1;

providing an original image, wherein the original image comprises a plurality of original image pixel points (P) arranged in an array;

step 2;

performing an edge detecting to the original image by use of Canny_H method and considering pixel edge information of a 4*4 area image;

step 3;

performing a 2*2 times interpolation zooming on edge pixel points and determining a type of the area image as pure color entity area or non-pure color entity area according to the pixel edge information;

step 4;

performing a conditional interpolation individually on pure color entity area and non-pure color entity area; and

step 5;

obtaining a pixel 2*2 times interpolation zoomed super-resolution image.

2. The super-resolution reconstructing method as claimed in claim 1, wherein the step 2 comprises:

step 20;

calculating direction and value of gradient of each of the original image pixel points by way of that using Sobel mask matrices

3. The super-resolution reconstructing method as claimed in claim 2, wherein when a point of edge detecting is the original image pixel point (P11),

when the step 20 determines the original image pixel point (P11) as an edge of a horizontal direction and an edge value as (G(P11)), if satisfying G(P11)>

½

*G(P01) and G(P11)>

½

*G(21), records an edge value of the original image pixel point (P11) as G(P11);

otherwise records the edge value of the original image pixel point (P11) as 0;

when the step 20 determines the original image pixel point (P11) as an edge of a vertical direction and an edge value as (G(P11));

if satisfying G(P11)>

½

*G(P10) and G(P11)>

½

*G(12), records an edge value of the original image pixel point (P11) as G(P11);

otherwise records the edge value of the original image pixel point (P11) as 0;

when the step 20 determines the original image pixel point (P11) as an edge of a 45-degree direction and an edge value as (G(P11));

if satisfying G(P11)>

½

*G(P00) and G(P11)>

½

*G(22), records an edge value of the original image pixel point (P11) as G(P11), otherwise records the edge value of the original image pixel point (P11) as 0; and

when the step 20 determines the original image pixel point (P11) as an edge of a 135-degree direction and an edge value as (G(P11));

if satisfying G(P11)>

½

*G(P02) and G(P11)>

½

*G(20), records an edge value of the original image pixel point (P11) as G(P11);

otherwise records the edge value of the original image pixel point (P11) as 0.

4. The super-resolution reconstructing method as claimed in claim 1, wherein in the step 3 of performing 2*2 times interpolation zooming on edge pixel points, and an interpolation order is that:

starting from the original image pixel point in a first row and first column, traversing the original image pixel points row by row and column by column, and interpolating 3 new pixel point when traverses each of the original image pixel points.

5. The super-resolution reconstructing method as claimed in claim 1, wherein when the type of the area image is pure color entity area, performing the interpolation according to a rule by using the edge information of the 4*4 area image.

6. The super-resolution reconstructing method as claimed in claim 1, wherein when the type of the area image is non-pure color entity area, performing a BiCubic_H interpolation in the non-pure color entity area.

7. The super-resolution reconstructing method as claimed in claim 1, wherein when traverses the original image pixel point (P11), the original image pixel points (P02), (P12), (P22) and (P32) are edges and other 12 original image pixel points (P00), (P01), (P03), (P10), (P11), (P13), (P20), (P21), P(23), (P30), P(31), (P33) are not edge, then if:

|F(P10)−

F(P11)|<

0.125*max(F(P10),F(P11)),

|F(P20)−

F(P21)|<

0.125*max(F(P20),F(P21)),

|F(P12)−

F(P13)|<

0.125*max(F(P12),F(P13)) and

|F(P22)−

F(P23)|<

0.125*max(F(P22),F(P23)), the area image is determined as pure color entity area;

otherwise it is determined as non-pure color entity area.

8. The super-resolution reconstructing method as claimed in claim 7, wherein (F(P)) is the sum of RGB values of original image pixel point (P), that is, F(P)=P(R)+P(G)+P(B);

when the area image is determined as pure color entity area, corresponding interpolated points (A), (B) and (C) satisfy that A=P11+P21, B=P11, and C=P11+P21.

9. The super-resolution reconstructing method as claimed in claim 1, wherein when traverses the original image pixel point (P11), the original image pixel points (P10), (P11), (P12) and (P13) are edges and the other 12 original image pixel points (P00), (P01), (P02), (P03), (P20), (P21), (P22), (P23), (P30), (P31), (P32), (P33) are not edge, then if:

|F(P01)−

F(P11)|<

0.125*max(F(P01),F(P11)),

|F(P02)−

F(P12)|<

0.125*max(F(P02),F(P12)),

|F(P21)−

F(P31)|<

0.125*max(F(P21),F(P31)), and

|F(P22)−

F(P32)|<

0.125*max(F(P22),F(P32)), the area image is determined as pure color entity area;

otherwise it is determined as non-pure color entity area.

10. The super-resolution reconstructing method as claimed in claim 9, wherein the (F(P)) is the sum of RGB values of original image pixel point (P), that is, F(P)=P(R)+P(G)+P(B);

when the area image is determined as pure color entity area, corresponding interpolated points (A), (B) and (C) satisfy that A=P21+P22, B=P11+P12, and C=P21.

11. The super-resolution reconstructing method as claimed in claim 1, wherein when traverse the original image pixel point (P11), the original image pixel points (P22), (P23), (P30) and (P31) are edges and the other 12 original image pixel points (P00), (P01), (P02), (P03), (P10), (P11), (P12), (P13), (P20), (P21), (P32), (P33) are not edge, then if:

|F(P02)−

F(P12)|<

0.125*max(F(P02),F(P12)),

|F(P03)−

F(P13)|<

0.125*max(F(P03),F(P13)),

|F(P22)−

F(P32)|<

0.125*max(F(P22),F(P32)), and

|F(P23)−

F(P33)|<

0.125*max(F(P23),F(P33)), the area image is determined as pure color entity area;

otherwise it is determined as non-pure color entity area.

12. The super-resolution reconstructing method as claimed in claim 11, wherein the (F(P)) is the sum of RGB values of original image pixel point (P), that is, F(P)=P(R)+P(G)+P(B);

when the area image is determined as pure color entity area, corresponding interpolated points (A), (B) and (C) satisfy that A=P21+P22, B=P11+P12, and C=P11+P21.

13. The super-resolution reconstructing method as claimed in claim 1, wherein when traverse the original image pixel point (P11), the original image pixel points (P02), (P12), (P20) and (P21) are edges and the other 12 original image pixel points (P00), (P01), (P03), (P10), (P11), (P13), (P22), (P23), (P30), (P31), (P32), (P33) are not edge, then if:

|F(P01)−

F(P11)|<

0.125*max(F(P01),F(P11)),

|F(P02)−

F(P12)|<

0.125*max(F(P02),F(P12)),

|F(P21)−

F(P31)|<

0.125*max(F(P21),F(P31)), and

|F(P22)−

F(P23)|<

0.125*max(F(P22),F(P23)), the area image is determined as pure color entity area;

otherwise it is determined as non-pure color entity area.

14. The super-resolution reconstructing method as claimed in claim 13, wherein the (F(P)) is the sum of RGB values of original image pixel point (P), that is, F(P)=P(R)+P(G)+P(B);

when the area image is determined as pure color entity area, corresponding interpolated points (A), (B) and (C) satisfy that A=P21+P22, B=P11, and C=P21.

15. The super-resolution reconstructing method as claimed in claim 1, wherein when traverse the original image pixel point (P11), the original image pixel points (P10), (P11), (P22) and (P32) are edges and the other 12 original image pixel points (P00), (P01), (P02), (P03), (P12), (P13), (P20), (P21), (P23), (P30), (P31), (P33) are not edge, then if:

|F(P10)−

F(P11)|<

0.125*max(F(P10),F(P11)),

|F(P20)−

F(P21)|<

0.125*max(F(P20),F(P21)),

|F(P12)−

F(P13)|<

0.125*max(F(P12),F(P13)), and

|F(P22)−

F(P23)|<

0.125*max(F(P22),F(P23)), the area image is determined as pure color entity area;

otherwise it is determined as non-pure color entity area.

16. The super-resolution reconstructing method as claimed in claim 15, wherein the (F(P)) is the sum of RGB values of original image pixel point (P), that is, F(P)=P(R)+P(G)+P(B);

when the area image is determined as pure color entity area, corresponding interpolated points (A), (B) and (C) satisfy that A=P11+P21, B=P11, and C=P11+P21.