ADDITION RATIO LEARNING APPARATUS AND METHOD, IMAGE PROCESSING APPARATUS AND METHOD, PROGRAM, AND RECORDING MEDIUM

• US 20120288187A1
• Filed: 04/26/2012
• Published: 11/15/2012
• Est. Priority Date: 05/12/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: ICHIHASHI, HIDEYUKI, NAGANO, TAKAHIRO, OGAWA, NOBUHIRO
• Granted Patent: US 8,824,782 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

There is provided an addition ratio learning apparatus including a noise adding unit that adds noises to data of an image input as a teacher image, a motion compensating unit that sets an image where time addition noise reduction processing is executed as an NR screen and performs motion compensation with respect to the NR screen, a differential feature amount calculating unit that sets an image as an input screen and calculates a differential feature amount, a circulation history specifying unit that counts a circulation history in the time addition noise reduction processing and specifies the circulation history, an addition ratio computing unit that computes an addition ratio on the basis of pixel values, and a time adding unit that performs multiplication by a coefficient determined according to the computed addition ratio to perform weighted addition and executes the time addition noise reduction processing with respect to the input screen.

Claims

1. An addition ratio learning apparatus comprising:

a noise adding unit that adds noises to data of an image input as a teacher image;

a motion compensating unit that sets an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame to which the noises have been added as an NR screen and performs motion compensation with respect to the NR screen;

a differential feature amount calculating unit that sets an image of an n-th frame to which the noises have been added as an input screen and calculates a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

a circulation history specifying unit that counts a circulation history in the time addition noise reduction processing and specifies the circulation history in the time addition noise reduction processing;

an addition ratio computing unit that computes an addition ratio corresponding to the noises, the differential feature amount, and the circulation history on the basis of a pixel value of the teacher image, a corresponding pixel value in the input screen, and a corresponding pixel value in the motion compensated NR screen; and

a time adding unit that performs multiplication by a coefficient determined according to the computed addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby executes the time addition noise reduction processing with respect to the input screen.

2. The addition ratio learning apparatus according to claim 1, further comprising:

a class code determining unit that sets the noises, the differential feature amount, and the circulation history as learning parameters and determines a class code obtained as a combination of the learning parameters,

wherein the addition ratio computing unit computes an addition ratio corresponding to the class code.

3. The addition ratio learning apparatus according to claim 1, further comprising:

a class code determining unit that sets the noises, the differential feature amount, and the circulation history as learning parameters and determines a class code obtained as a combination of the learning parameters; and

a storage unit that stores the addition ratio computed by the addition ratio computing unit as a learning result by the addition ratio learning apparatus, for each class code.

4. The addition ratio learning apparatus according to claim 1, further comprising:

a class code determining unit that sets the noises, the differential feature amount, and the circulation history as learning parameters and determines a class code obtained as a combination of the learning parameters; and

a sample sorting unit that sorts samples of a linear expression generated using the pixel value of the teacher image, the corresponding pixel value in the input screen, and the corresponding pixel value in the motion compensated NR screen for each class code and accumulates the samples,

wherein the addition ratio computing unit adds the samples, calculates a coefficient of the linear expression by a least-square method, and thereby computes the addition ratio.

5. The addition ratio learning apparatus according to claim 1, further comprising:

a plurality of addition ratio computing units,

wherein the plurality of addition ratio computing units share the addition ratio and compute the addition ratio on the basis of a value of each learning parameter.

6. An addition ratio learning method comprising:

causing a noise adding unit to add noises to data of an image input as a teacher image;

causing a motion compensating unit to set an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame to which the noises have been added as an NR screen and perform motion compensation with respect to the NR screen;

causing a differential feature amount calculating unit to set an image of an n-th frame to which the noises have been added as an input screen and calculate a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

causing a circulation history specifying unit to count a circulation history in the time addition noise reduction processing and specify the circulation history in the time addition noise reduction processing;

causing an addition ratio computing unit to compute an addition ratio corresponding to the noises, the differential feature amount, and the circulation history on the basis of a pixel value of the teacher image, a corresponding pixel value in the input screen, and a corresponding pixel value in the motion compensated NR screen; and

causing a time adding unit to perform multiplication by a coefficient determined according to the computed addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby execute the time addition noise reduction processing with respect to the input screen.

7. A program for causing a computer to function as an addition ratio learning apparatus,

wherein the addition ratio learning apparatus includes:

a noise adding unit that adds noises to data of an image input as a teacher image;

a motion compensating unit that sets an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame to which the noises have been added as an NR screen and performs motion compensation with respect to the NR screen;

a differential feature amount calculating unit that sets an image of an n-th frame to which the noises have been added as an input screen and calculates a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

a circulation history specifying unit that counts a circulation history in the time addition noise reduction processing and specifies the circulation history in the time addition noise reduction processing;

an addition ratio computing unit that computes an addition ratio corresponding to the noises, the differential feature amount, and the circulation history on the basis of a pixel value of the teacher image, a corresponding pixel value in the input screen, and a corresponding pixel value in the motion compensated NR screen; and

a time adding unit that performs multiplication by a coefficient determined according to the computed addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby executes the time addition noise reduction processing with respect to the input screen.

8. A recording medium in which the program according to claim 7 is recorded.

9. An image processing apparatus comprising:

a motion compensating unit that sets an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame as an NR screen and performs motion compensation with respect to the NR screen;

a noise measuring unit that sets an image of an n-th frame in data of an input image as an input screen and measures noises of the input screen;

a differential feature amount calculating unit that calculates a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

a circulation history specifying unit that counts a circulation history in the time addition noise reduction processing and specifies the circulation history in the time addition noise reduction processing;

an addition ratio determining unit that determines an addition ratio corresponding to the noises, the differential feature amount, and the circulation history; and

a time adding unit that performs multiplication by a coefficient determined according to the determined addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby executes the time addition noise reduction processing with respect to the input screen.

10. The image processing apparatus according to claim 9, further comprising:

a class code determining unit that determines a class code obtained as a combination of the noises, the differential feature amount, and the circulation history,

wherein the addition ratio determining unit determines an addition ratio corresponding to the class code.

11. The image processing apparatus according to claim 9,

wherein the circulation history specifying unit resets a count of the circulation history when the differential feature amount more than a preset threshold value is calculated.

12. The image processing apparatus according to claim 9,

wherein the circulation history specifying unit resets a count of the circulation history when the differential feature amount more than a threshold value determined in advance according to an amount of the noises is calculated.

13. An image processing method comprising:

causing a motion compensating unit to set an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame as an NR screen and perform motion compensation with respect to the NR screen;

causing a noise measuring unit to set an image of an n-th frame in data of an input image as an input screen and measure noises of the input screen;

causing a differential feature amount calculating unit to calculate a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

causing a circulation history specifying unit to count a circulation history in the time addition noise reduction processing and specify the circulation history in the time addition noise reduction processing;

causing an addition ratio determining unit to determine an addition ratio corresponding to the noises, the differential feature amount, and the circulation history; and

causing a time adding unit to perform multiplication by a coefficient determined according to the determined addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby execute the time addition noise reduction processing with respect to the input screen.

14. A program for causing a computer to function as an image processing apparatus,

wherein the image processing apparatus includes:

a motion compensating unit that sets an image where time addition noise reduction processing is executed with respect to an image of an (n−

1)-th frame as an NR screen and performs motion compensation with respect to the NR screen;

a noise measuring unit that sets an image of an n-th frame in data of an input image as an input screen and measures noises of the input screen;

a differential feature amount calculating unit that calculates a differential feature amount on the basis of a pixel value of a predetermined area forming the input screen and a pixel value of a corresponding area in the motion compensated NR screen;

a circulation history specifying unit that counts a circulation history in the time addition noise reduction processing and specifies the circulation history in the time addition noise reduction processing;

an addition ratio determining unit that determines an addition ratio corresponding to the noises, the differential feature amount, and the circulation history; and

a time adding unit that performs multiplication by a coefficient determined according to the determined addition ratio to perform weighted addition on the pixel value of the input screen and the corresponding pixel value in the motion compensated NR screen and thereby executes the time addition noise reduction processing with respect to the input screen.

15. A recording medium in which a program according to claim 14 is recorded.