Video enhancement method and device

• US 10,390,014 B2
• Filed: 01/18/2017
• Issued: 08/20/2019
• Est. Priority Date: 08/14/2014
• Status: Active Grant

• Current Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Inventors: Li, Kai, Wang, Pulin, Ji, Xiaocui
• Primary Examiner(s): Jiang, Zaihan
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US15/409,186
• Publication Number:
  US 20170127063A1
• Time in Patent Office: 944 Days
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component

Abstract

The disclosure discloses a video enhancement method and device, and belongs to the field of image processing. The method includes that: each frame in a video is sequentially acquired; and for each current frame, attribute information reflecting a moving state or luminance is determined, a preset mapping relationship is searched to obtain a control parameter corresponding to the attribute information, and video enhancement operation is executed on the current frame by using the control parameter. The device includes: an acquisition module and an enhancement module. By the disclosure, the problem of processing inflexibility caused by execution of the same enhancement over each frame in the video is solved, each frame in the video may be differentially enhanced, frame-based video enhancement is implemented, a video enhancement effect is effectively improved, and quality of the video is improved.

Claims

1. A video enhancement method, comprising:

sequentially acquiring, by a video enhancement device, each frame in a video;

for each current frame, determining, by the video enhancement device, attribute information, which reflects a moving state or luminance, of the current frame, searching a preset mapping relationship between attribute information and a control parameter to obtain a control parameter of the current frame, and executing a video enhancement operation on the current frame by using the control parameter of the current frame;

or,

for each current frame, dividing, by the video enhancement device, the current frame into multiple regions, determining attribute information, which reflects a moving state or luminance, of each region, searching a preset mapping relationship between attribute information and control parameter to obtain a control parameter of each region, and executing a video enhancement operation on the current frame by using the control parameters of all the regions in the current frame;

wherein the determining the attribute information, which reflects the moving state or the luminance, of the current frame and searching the preset mapping relationship between the attribute information and the control parameter to obtain the control parameter of the current frame comprises;

calculating a Quantization Parameter (QP) of the current frame, and searching a preset mapping relationship between a QP and a control parameter to obtain a control parameter corresponding to the QP;

or, determining a scenario type of the current frame, and searching a preset mapping relationship between a scenario type and a control parameter to obtain a control parameter corresponding to the scenario type;

or, calculating a luminance value of the current frame, and searching a preset mapping relationship between a luminance value and a control parameter to obtain a control parameter corresponding to the luminance value;

or

wherein the determining the attribute information, which reflects the moving state or the luminance, of each region and searching the preset mapping relationship between the attribute information and the control parameter to obtain the control parameter of each region comprises;

calculating a QP of each region, and searching a preset mapping relationship between a QP and a control parameter to obtain the control parameter of each region;

or, determining a scenario type of each region, and searching a preset mapping relationship between a scenario type and a control parameter to obtain the control parameter of each region;

or, calculating a luminance value of each region, and searching a preset mapping relationship between a luminance value and a control parameter to obtain the control parameter of each region,

wherein the control parameter comprises at least one of;

a contrast control parameter, a definition control parameter or a noise reduction control parameter;

wherein the scenario type at least comprises a still scenario and a moving scenario;

the still scenario comprises;

a desktop or document sharing scenario and/or a still person scenario; and

the moving scenario comprises;

at least one of a slight moving scenario, an ordinary moving scenario, a large moving scenario and a strenuous moving scenario.

2. The method according to claim 1, wherein the executing video enhancement operation on the current frame by using the control parameter of the current frame comprises:

calculating a difference between the control parameter of the current frame and a control parameter of a previous frame, and judging whether an absolute value of the difference exceeds a specified threshold value or not;

if the absolute value of the difference exceeds the specified threshold value, calculating a new control parameter according to the control parameter of the previous frame and the threshold value, and executing a video enhancement operation on the current frame by using the new control parameter; and

if the absolute value of the difference does not exceed the specified threshold value, executing a video enhancement operation on the current frame by using the control parameter of the current frame.

3. The method according to claim 1, before executing video enhancement operation on the current frame by using the control parameters of all the regions in the current frame, further comprising:

for each region i in the current frame, judging whether an absolute value of a difference between a control parameter of a neighbouring region and a control parameter of the region i exceeds a specified threshold value or not;

if an absolute value of a difference between a control parameter of at least one neighbouring region and the control parameter of the region i exceeds the specified threshold value, calculating a new control parameter according to a preset algorithm, and determining the new control parameter as the control parameter of the region i; and

if the absolute values of the differences between the control parameters of all the neighbouring regions and the control parameter of the region i do not exceed the specified threshold value, keeping the control parameter of the region i unchanged.

4. The method according to claim 3, wherein calculating the new control parameter according to the preset algorithm comprises:

calculating the new control parameter according to the control parameters of the neighbouring regions, under a principle that the absolute values of the differences with the control parameters of each neighbouring region do not exceed the specified threshold value;

or,

specifying a weight for each neighbouring region, and calculating the new control parameter according to the control parameters and weights of the neighbouring regions.

5. The method according to claim 1, wherein

in the mapping relationship, a larger QP corresponds to a smaller contrast control parameter, and/or, a larger QP corresponds to a smaller definition control parameter, and/or, a larger QP corresponds to a larger noise reduction control parameter;

or,

in the mapping relationship, a higher movement degree represented by a scenario type corresponds to a smaller contrast control parameter, and/or, a higher movement degree represented by a scenario type corresponds to a smaller definition control parameter, and/or, a higher movement degree represented by a scenario type corresponds to a larger noise reduction control parameter;

or,

in the mapping relationship, a larger luminance value corresponds to a larger contrast control parameter, and/or, a larger luminance value corresponds to a larger definition control parameter, and/or, a larger luminance value corresponds to a smaller noise reduction control parameter.

6. The method according to claim 1, after executing video enhancement operation on the current frame by using the control parameter, further comprising:

coding the video subjected to the video enhancement operation, and sending the coded video.

7. The method according to claim 1, wherein the sequentially acquiring each frame in the video comprises:

receiving the video, decoding the video, and sequentially acquiring each frame in the decoded video.

8. A video enhancement device, comprising:

a processor and a memory, a set of program codes being stored in the memory and the processor executing the program codes stored in the memory to implement the following operation;

sequentially acquiring each frame in a video;

for each current frame, determining attribute information, which reflects a moving state or luminance, of the current frame, searching a preset mapping relationship between attribute information and a control parameter to obtain a control parameter of the current frame, and executing a video enhancement operation on the current frame by using the control parameter of the current frame;

or,

for each current frame, dividing the current frame into multiple regions, determining attribute information, which reflects a moving state or luminance, of each region, searching a preset mapping relationship between the attribute information and the control parameter to obtain a control parameter of each region, and executing a video enhancement operation on the current frame by using the control parameters of all the regions in the current frame;

wherein the processor executes the program codes stored in the memory to implement the following operation;

for each current frame, calculating a Quantization Parameter (QP) of the current frame, searching a preset mapping relationship between a QP and a control parameter to obtain a control parameter corresponding to the QP, and executing a video enhancement operation on the current frame by using the control parameter;

or, for each current frame, determining a scenario type of the current frame, searching a preset mapping relationship between a scenario type and a control parameter to obtain a control parameter corresponding to the scenario type, and executing a video enhancement operation on the current frame by using the control parameter;

or, for each current frame, calculating a luminance value of the current frame, searching a preset mapping relationship between a luminance value and a control parameter to obtain a control parameter corresponding to the luminance value, and executing a video enhancement operation on the current frame by using the control parameter, or

wherein the processor executes the program codes stored in the memory to implement the following operation;

for each current frame, dividing the current frame into multiple regions, calculating a QP of each region, searching the preset mapping relationship between the QP and the control parameter to obtain the control parameter of each region, and executing a video enhancement operation on the current frame by using the control parameters of all the regions in the current frame;

or, for each current frame, dividing the current frame into multiple regions, determining a scenario type of each region, search the preset mapping relationship between the scenario type and the control parameter to obtain the control parameter of each region, and executing a video enhancement operation on the current frame by using the control parameters of all the regions in the current frame;

or, for each current frame, dividing the current frame into multiple regions, calculating a luminance value of each region, searching the preset mapping relationship between the luminance value and the control parameter to obtain the control parameter of each region, and executing video a enhancement operation on the current frame by using the control parameters of all the regions in the current frame,

wherein the control parameter comprises at least one of;

a contrast control parameter, a definition control parameter or a noise reduction control parameter;

wherein the processor executes the program codes stored in the memory to implement the following operation that;

the scenario type at least comprises a still scenario and a moving scenario;

the still scenario comprises;

a desktop or document sharing scenario and/or a still person scenario; and

the moving scenario comprises;

at least one of a slight moving scenario, an ordinary moving scenario, a large moving scenario and an strenuous moving scenario.

9. The device according to claim 8, wherein the processor executes the program codes stored in the memory to implement the following operation:

calculating a difference between the control parameter of the current frame and a control parameter of a previous frame, and judging whether an absolute value of the difference exceeds a specified threshold value or not;

if the absolute value of the difference exceeds the specified threshold value, calculating a new control parameter according to the control parameter of the previous frame and the threshold value, and executing a video enhancement operation on the current frame by using the new control parameter; and

if the absolute value of the difference does not exceed the specified threshold value, executing a video enhancement operation on the current frame by using the control parameter of the current frame.

10. The device according to claim 8, wherein the processor executes the program codes stored in the memory to implement the following operation:

before executing a video enhancement operation on the current frame by using the control parameters of all the regions in the current frame, for each region i in the current frame, judging whether an absolute value of a difference between a control parameter of a neighbouring region and a control parameter of the region i exceeds a specified threshold value or not;

if an absolute value of a difference between a control parameter of at least one neighbouring region and the control parameter of the region i exceeds the specified threshold value, calculating a new control parameter according to a preset algorithm, and determining the new control parameter as the control parameter of the region i; and

if the absolute values of the differences between the control parameters of all the neighbouring regions and the control parameter of the region i do not exceed the specified threshold value, keeping the control parameter of the region i unchanged.

11. The device according to claim 10, wherein the processor executes the program codes stored in the memory to implement the following operation:

calculating the new control parameter according to the control parameters of the neighbouring regions, under according to a principle that the absolute values of the differences with the control parameters of each neighbouring region do not exceed the specified threshold value;

or, specifying a weight for each neighbouring region, and calculating the new control parameter according to the control parameters and weights of the neighbouring regions.

12. The device according to claim 8, wherein the processor executes the program codes stored in the memory to implement the following operation that:

in the mapping relationship, a larger QP corresponds to a smaller contrast control parameter, and/or, a larger QP corresponds to a smaller definition control parameter, and/or, a larger QP corresponds to a larger noise reduction control parameter;

or,

in the mapping relationship, a higher movement degree represented by a scenario type corresponds to a smaller contrast control parameter, and/or, a higher movement degree represented by a scenario type corresponds to a smaller definition control parameter, and/or, a scenario type representing a higher movement degree corresponds to a larger noise reduction control parameter;

or,

in the mapping relationship, a larger luminance value corresponds to a larger contrast control parameter, and/or, a larger luminance value corresponds to a larger definition control parameter, and/or, a larger luminance value corresponds to a smaller noise reduction control parameter.

13. The device according to claim 8, wherein the processor executes the program codes stored in the memory to implement the following operation:

coding the video subjected to the video enhancement operation, and sending the coded video.

14. The device according to claim 8 wherein the processor executes the program codes stored in the memory to implement the following operation:

receiving the video, decoding the video, and sequentially acquiring each frame in the decoded video.