Image processing device, image processing method and medium

• US 9,508,025 B2
• Filed: 11/05/2013
• Issued: 11/29/2016
• Est. Priority Date: 11/06/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Senda, Shuji, Shibata, Takashi, Iketani, Akihiko
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): PERLMAN, DAVID S
• Application Number:
  US14/440,382
• Publication Number:
  US 20150302273A1
• Time in Patent Office: 1,120 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

An image processing device according to the present invention includes: a patch generation unit which generates an input patch used for comparison on the basis of an input image; a modification parameter estimation unit which estimates a parameter used in blurred modification on the basis of the input image; a blurred image generation unit which generates a blurred image on the basis of a learning image by using the parameter; a patch pair generation unit which generates a patch pair used to compose a restoration image on the basis of the blurred image and the learning image; a selection unit which selects a patch pair used to compose the restoration image on the basis of the input patch; and a composition unit which composes the restoration image on the basis of the patch pair selected by the selection unit.

Claims

1. An image processing device comprising:

a central processing unit (CPU); and

a memory, connected to the CPU, having instructions stored therein that when executed by the CPU comprise;

a patch generation unit which generates an input patch used for comparison on the basis of an input image;

a modification parameter estimation unit which estimates a parameter used in blurred modification on the basis of the input image;

a blurred image generation unit which generates a blurred image on the basis of a learning image by using the parameter;

a patch pair generation unit which generates a patch pair used to compose a restoration image on the basis of the blurred image and the learning image;

a selection unit which selects a patch pair used to compose the restoration image on the basis of the input patch;

a composition unit which composes the restoration image on the basis of the patch pair selected by the selection unit, and

an evaluation unit which evaluates the patch pair selected by the selection unit, wherein

the modification parameter estimation unit estimates the parameter on the basis of a result of the evaluation.

2. The image processing device according to claim 1, wherein

the evaluation unit evaluates a result of the composition means.

3. The image processing device according to claim 1, wherein

the evaluation unit repeats the evaluation for a predetermined number of times or until the evaluation result is within a predetermined range.

4. The image processing device according to claim 1, wherein

the modification parameter estimation unit estimates a parameter in view of at least one of a color of a periphery of the input image, a shadow, or an object included in the image.

5. An image processing method for an image processing device, which comprising:

a CPU; and

a memory which is connected to the CPU, comprising;

generating an input patch used for comparison on the basis of an input image;

estimating a parameter used in blurred modification on the basis of the input image;

generating a blurred image on the basis of a learning image by using the parameter;

generating a patch pair used to compose a restoration image on the basis of the blurred image and the learning image;

selecting a patch pair used to compose the restoration image on the basis of the input patch;

composing the restoration image on the basis of the selected_patch pair; and

evaluating the selected patch pair, wherein

the parameter is estimated on the basis of a result of the evaluation.

6. The image processing method according to claim 5, wherein evaluating a result of the composition in the evaluation.

7. The image processing method according to claim 5, wherein repeating the evaluation for a predetermined number of times or until the evaluation result is within a predetermined range.

8. The image processing method according to claim 5, wherein

estimating a parameter in view of at least one of a color of a periphery of the input image, a shadow, or an object included in the image in the estimation of the parameter.

9. A computer readable non-transitory medium embodying a program, the program causing a computer, which comprising:

a CPU; and

a memory which is connected to the CPU, to perform a method, the method comprising;

generating an input patch used for comparison on the basis of an input image;

estimating a parameter used in blurred modification on the basis of the input image;

generating a blurred image on the basis of a learning image by using the parameter;

generating a patch pair used to compose a restoration image on the basis of the blurred image and the learning image;

selecting a patch pair used to compose the restoration image on the basis of the input patch;

composing the restoration image on the basis of the selected patch pair; and

evaluating the selected patch pair, wherein

the parameter is estimated on the basis of a result of the evaluation.

10. The computer-readable non-transitory medium according to claim 9, wherein evaluating a result of the composition in evaluating.

11. The computer-readable non-transitory medium according to claim 9, wherein repeating the evaluation for a predetermined number of times or until the evaluation result is within a predetermined range.

12. The computer-readable non-transitory medium according to claim 9, wherein

estimating a parameter in view of at least one of a color of a periphery of the input image, a shadow, or an object included in the image in estimating the parameter.

13. An image processing device comprising:

patch generation means for generating an input patch used for comparison on the basis of an input image;

modification parameter estimation means for estimating a parameter used in blurred modification on the basis of the input image;

blurred image generation means for generating a blurred image on the basis of a learning image by using the parameter;

patch pair generation means for generating a patch pair used to compose a restoration image on the basis of the blurred image and the learning image;

selection means for selecting a patch pair used to compose the restoration image on the basis of the input patch; and

composition means for composing the restoration image on the basis of the patch pair selected by the selection means;

evaluation means for evaluation the patch pair selected by the selection unit, wherein

the modification parameter estimation means estimates the parameter on the basis of a result of the evaluation.