Signal processing device and control program for sharpening images

• US 8,891,898 B2
• Filed: 11/08/2010
• Issued: 11/18/2014
• Est. Priority Date: 02/15/2010
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Gohshi, Seiichi
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Niu, Feng
• Application Number:
  US13/577,953
• Publication Number:
  US 20120314968A1
• Time in Patent Office: 1,471 Days
• Field of Search: None
• US Class Current: 382/263
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 5/142 Edging; Contouring
• CPC Class Codes: H04N 5/208 for compensating for attenuation of high frequency components, e.g. crispening, aperture distortion correction

Abstract

A signal processing device subjects a signal indicating a moving image constituted by frames to a sharpening process for the moving image, to output a signal indicating the sharpened moving image, and includes: a motion detection section for detecting motion between the frames for each block region including one or a plurality of adjacent pixels constituting each frame; a sharpening process section for subjecting a signal indicating an image of a block region from which motion has been detected to a nonlinear process by a nonlinear process section and sharpening the signal to generate a harmonic; and a signal output section for outputting, for the block region from which motion was detected, the harmonic produced by sharpening a signal indicating a moving image of the block region, while outputting, for a block region from which motion was not detected, a signal indicating a moving image of the block region.

Claims

1. A signal processing device that subjects a signal indicative of a moving image constituted by a plurality of temporarily consecutive frames to a sharpening process for the moving image, to output a signal indicative of a moving image thus sharpened, each of the frames including a plurality of block regions, each of the plurality of block regions include one or a plurality of adjacent pixels,

the signal processing device configured or programmed to include:

a motion detection section configured or programmed to detect motion between the frames with respect to each of the plurality of block regions;

a first high-frequency component generation section configured or programmed to generate a harmonic produced by sharpening a signal indicative of a moving image of a block region from which the motion has been detected by the motion detection section; and

a signal output section configured or programmed to output, with respect to the block region from which the motion has been detected by the motion detection section, the harmonic having been produced by sharpening a motion region signal that is a signal indicative of a moving image of the block region from which the motion has been detected by the motion detection section, while outputting, with respect to the block region from which the motion has not been detected by the motion detection section, a non-motion region signal that is a signal indicative of a moving image of the block region from which the motion has been detected by the motion detection section,

the first high-frequency component generation section including;

a first low-frequency component removing section configured or programmed to remove, from frequency components of the motion region signal, at least a direct current component, to generate and output a first low-frequency-free signal;

a first nonlinear process section configured or programmed to receive the first low-frequency-free signal, to generate a first nonlinear process signal (i) in which positive and negative signs of the first low-frequency-free signal are retained and (ii) which broadly monotonically increases nonlinearly with respect to the first low-frequency-free signal when values of the first low-frequency-free signal are at least in the vicinity of 0, and to output the first nonlinear process signal; and

an addition section configured or programmed to add the first nonlinear process signal to the motion region signal, to generate the harmonic.

2. The signal processing device as set forth in claim 1, further comprising:

a second high-frequency component generation section configured or programmed to sharpen, before the non-motion region signal is outputted from the signal output section, the non-motion region signal,

the second high frequency component generation section including;

a second low-frequency component removing section configured or programmed to remove, from frequency components of the non-motion region signal, at least a direct current component, to generate and output a second low-frequency-free signal;

a second nonlinear process section configured or programmed to receive the second low-frequency-free signal, to generate a second nonlinear process signal (i) in which positive and negative signs of the second low-frequency-free signal are retained and (ii) which monotonically increases nonlinearly with respect to the second low-frequency-free signal when values of the second low-frequency-free signal are at least in the vicinity of 0, and to output the second nonlinear process signal; and

a second addition section configured or programmed to add the second nonlinear process signal to the non-motion region signal, to sharpen the non-motion region signal.

3. The signal processing device as set forth in claim 2, wherein

the first low-frequency component removing section further includes:

a low-level signal removing section configured or programmed to change, out of signal values of an outgoing signal, signal values whose absolute values are lower than a predetermined lower limit to 0, and

a high-level signal removing section configured or programmed to change, out of the signal values of the outgoing signal, signal values whose absolute values are higher than a predetermined upper limit in such a manner that the absolute values are not higher than the predetermined upper limit while maintaining signs of that signal values, the first nonlinear process section further includes;

a first amplitude adjustment section configured or programmed to adjust an amplitude of the outgoing signal by multiplying the amplitude by a first predetermined magnification value, wherein the first amplitude adjustment section sets the first predetermined magnification value according to a size of motion having been detected by the motion detection section,

the second nonlinear process section further includes;

a second amplitude adjustment section configured or programmed to adjust an amplitude of the second nonlinear process signal by multiplying the amplitude by a second predetermined magnification value, and

the first predetermined magnification value is larger than the second predetermined magnification value.

4. The signal processing device as set forth in claim 1, wherein

the first nonlinear process section includes:

an even exponentiation operation section configured or programmed to generate an even exponentiation signal by raising an incoming signal to an even exponent not less than 2; and

a sign changing section configured or programmed to reverse positive and negative signs of a portion of the even exponentiation signal, the portion is different in sign from the incoming signal, and

the even exponentiation signal including the signs having been reversed by the sign changing section is outputted.

5. The signal processing device as set forth in claim 1, wherein

the first nonlinear process section includes:

an even exponentiation operation section configured or programmed to generate an even exponentiation signal by raising an incoming signal to an even exponent not less than 2;

a differentiation section configured or programmed to generate a differential signal by differentiating the even exponentiation signal; and

a sign changing section configured or programmed to reverse positive and negative signs of a portion of the differential signal, the portion is different in sign from the incoming signal, and

the differential signal including the signs having been reversed by the sign changing section is outputted.

6. The signal processing device as set forth in claim 1, wherein

the first nonlinear process section includes:

an odd exponentiation operation section configured or programmed to raise an incoming signal to an odd exponent not less than 3, and

a resulting signal obtained by raising by the odd exponentiation operation section is outputted.

7. The signal processing device as set forth in claim 1, wherein

the first nonlinear process section includes:

a square root operation section configured or programmed to generate a square root signal by multiplying (i) a square root of an absolute value of a value calculated by dividing an incoming signal by a possible maximum value of the incoming signal, by (ii) the maximum value; and

a sign changing section configured or programmed to reverse positive and negative signs of a portion of the square root signal, the portion is different in sign from the incoming signal, and

the square root signal including the signs having been reversed by the sign changing section is outputted.

8. The signal processing device as set forth in claim 1, wherein

the first nonlinear process section, when values of an incoming signal are around 0, outputs a signal having an absolute value larger than that of the incoming signal.

9. The signal processing device as set forth in claim 1, wherein

the first low-frequency component removing section is a high-pass filter having 3 or more taps.

10. The signal processing device as set forth in claim 1, wherein

the first low-frequency component removing section further includes:

a low-level signal removing section configured or programmed to change, out of signal values of an outgoing signal, signal values whose absolute values are lower than a predetermined lower limit to 0, and

a high-level signal removing section configured or programmed to change, out of the signal values of the outgoing signal, signal values whose absolute values are higher than a predetermined upper limit in such a manner that the absolute values are not higher than the predetermined upper limit while maintaining signs of that signal values.

11. The signal processing device as set forth in claim 10, wherein

the first nonlinear process section further includes:

a first amplitude adjustment section configured or programmed to adjust an amplitude of the outgoing signal by multiplying the amplitude by a first predetermined magnification value.

12. The signal processing device as set forth in claim 11, wherein

the first amplitude adjustment section sets the first predetermined magnification value according to a size of motion having been detected by the motion detection section.

13. The signal processing device as set forth in claim 1, wherein

the motion detection section includes:

a difference calculation section configured or programmed to calculate a difference signal indicative of a difference between adjacent two frames;

a first high frequency component calculation section configured or programmed to calculate, with respect to said each block region, a first high frequency component included in one of the two frames;

a second high frequency component calculation section configured or programmed to calculate, with respect to said each block region, a second high frequency component included in the other of the two frames;

a third high frequency component calculation section configured or programmed to calculate, with respect to said each block region, a third high frequency component by adding a product of the first high frequency component and a first coefficient to a product of the second high frequency component and a second coefficient; and

a division section configured or programmed to calculate, with respect to said each block region, a motion signal by dividing the difference signal by the third high frequency component, wherein

the motion detection section detects motion when the motion signal in said each block region is larger than a predetermined threshold.

14. A signal processing device that subjects a signal indicative of a moving image constituted by a plurality of temporarily consecutive frames to a sharpening process for the moving image, to output a signal indicative of a moving image thus sharpened, each of the frames including a plurality of block regions, each of the plurality of block regions include one or a plurality of adjacent pixels,

the signal processing device configured and programmed to include:

a motion detection section configured or programmed to detect motion between the frames with respect to each of the plurality of block regions;

a first high-frequency component generation section configured or programmed to generate a first sharpening signal by sharpening a signal indicative of a pixel group including pixels positioned adjacently in a main scanning direction of a moving image of a block region from which the motion has been detected by the motion detection section;

a second high-frequency component generation section configured or programmed to generate a second sharpening signal by sharpening a signal indicative of a pixel group including pixels positioned adjacently in a sub-scanning direction of the moving image represented by the first sharpening signal; and

a signal output section configured or programmed to output, with respect to the block region from which the motion has been detected by the motion detection section, the second sharpening signal having been produced by sharpening a motion region signal that is a signal indicative of a moving image of the block region from which the motion has been detected by the motion detection section, while outputting, with respect to the block region from which the motion has not been detected by the motion detection section, a non-motion region signal that is a signal indicative of a moving image of the block region from which the motion has been detected by the motion detection section,

the first high-frequency component generation section including;

a first low-frequency component removing section configured or programmed to remove, from frequency components of a main scanning direction signal that is a signal indicative of a pixel group including pixels positioned adjacently in a main scanning direction of a moving image of the block region from which the motion has been detected by the motion detection section, at least a direct current component, to generate and output a first low-frequency-free signal;

a first nonlinear process section configured or programmed to generate a first nonlinear process signal (i) in which positive and negative signs of the first low-frequency-free signal are retained and (ii) which broadly monotonically increases nonlinearly with respect to the first low-frequency-free signal when values of the first low-frequency-free signal are at least in the vicinity of 0; and

a first addition section configured or programmed to add the first nonlinear process signal to the main scanning direction signal, to generate the first sharpening signal,

the second high-frequency component generation section including;

a second low-frequency component removing section configured or programmed to remove, from frequency components of a sub-scanning direction signal that is a signal indicative of a pixel group including pixels positioned adjacently in a sub-scanning direction of a moving image represented by the first sharpening signal, at least a direct current component, to generate a second low-frequency-free signal;

a second nonlinear process section configured or programmed to generate and output a second nonlinear process signal (i) in which positive and negative signs of the second low-frequency-free signal are retained and (ii) which broadly monotonically increases nonlinearly with respect to the second low-frequency-free signal when values of the second low-frequency-free signal are at least in the vicinity of 0; and

a second addition section configured or programmed to add the second nonlinear process signal to the sub-scanning direction signal, to generate the second sharpening signal.

15. The signal processing device as set forth in claim 14, further comprising:

a third high-frequency component generation section configured or programmed to sharpen, before the second sharpening signal is outputted from the signal output section, a temporal direction signal that is a signal indicative of a pixel group including pixels positioned adjacently in a temporal direction of a moving image represented by the second sharpening signal,

the third high-frequency component generation section including;

a third low-frequency component removing section configured or programmed to remove, from frequency components of the temporal direction signal, at least a direct current component, to generate and output a third low-frequency-free signal;

a third nonlinear process section configured or programmed to generate a third nonlinear process signal (i) in which positive and negative signs of the third low-frequency-free signal are retained and (ii) which broadly monotonically increases nonlinearly with respect to the third low-frequency-free signal when values of the third low-frequency-free signal are at least in the vicinity of 0; and

a third addition section configured or programmed to add the third nonlinear process signal to the temporal direction signal, to sharpen the temporal direction signal.