Image processing method and apparatus using trained dictionary

• US 9,443,287 B2
• Filed: 02/03/2015
• Issued: 09/13/2016
• Est. Priority Date: 02/07/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kimura, Yoshinori
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US14/612,756
• Publication Number:
  US 20150227819A1
• Time in Patent Office: 588 Days
• Field of Search: 382/159, 382/254, 382/260, 382/165, 382/155, 382/201, 382/206, 382/209, 348/143, 345/419
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

The image processing method includes providing first dictionaries produced by dictionary learning and second dictionaries corresponding to the first dictionaries, performing, on each first dictionary, a process to approximate the first image by linear combination of elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring multiple linear combination coefficients, and calculating, for each linear combination coefficient, a ratio between a largest coefficient element and a second-largest coefficient element and selecting a specific linear combination coefficient in which the ratio is largest among the multiple linear combination coefficients. The method further includes selecting, from the multiple second dictionaries, a specific dictionary corresponding to the first dictionary for which the specific linear combination coefficient is produced, and producing the second image by using linear combination of the specific linear combination coefficient and elements of the specific dictionary.

Claims

1. An image processing method that uses a processor to convert an image, the method comprising:

performing, on each of multiple first dictionaries produced by dictionary learning, a process to approximate a first image by linear combination of elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring multiple linear combination coefficients;

calculating, by the processor and for each of the multiple linear combination coefficients, a ratio between a largest coefficient element and a second-largest coefficient element among respective coefficient elements of the respective linear combination coefficients, the largest coefficient element having an absolute value that is largest among the respective coefficient elements and the second-largest coefficient element having an absolute value that is second-largest among the respective coefficient elements, thereby obtaining respective ratios of the multiple linear combination coefficients;

selecting a specific linear combination coefficient, among the multiple linear combination coefficients, having the ratio that is largest among the respective ratios of the multiple linear combination coefficients;

selecting, from multiple second dictionaries corresponding to the first dictionaries, a specific dictionary corresponding to the first dictionary for which the specific linear combination coefficient is selected; and

converting the first image into a second image by using linear combination of the specific linear combination coefficient and elements of the specific dictionary.

2. The image processing method according to claim 1 further comprising:

providing a first training image relevant to the first image and a second training image relevant to the second image;

extracting multiple first training partial images from the first training image;

extracting multiple second training partial images from places corresponding to extracted places of the multiple first training partial images in the first training image;

subtracting, from pixel values distributed in each of the multiple first and second training partial images, an average value of the pixel values to produce an AC component;

converting the AC component of each first training partial image and the AC component of each second training partial image into vectors to produce AC vectors;

performing, on each of pairs of the first and second training partial images whose extracted areas correspond to each other, a process to combine the AC vectors of the pair of the first and second training partial images together so as to produce a training vector and thereby producing multiple training vectors;

clustering the training vectors to produce multiple clusters each containing the training vectors similar to one another;

performing, on each of the multiple clusters, a process to produce a dictionary matrix from the cluster by using a dictionary learning algorithm so as to produce multiple dictionary matrices; and

producing the first and second dictionaries from each of the dictionary matrices.

3. The image processing method according to claim 1,

wherein:

the second image is an image showing a result of classifying multiple types of object images contained in the first image, and

the method further comprises;

providing the multiple first and second dictionaries produced by the dictionary learning that uses a first training image relevant to the first image and multiple vector data as a second training image showing the types of the respective object images;

performing, on each of the multiple first dictionaries, a process to approximate a partial image extracted from the first image by the linear combination of the elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring the multiple linear combination coefficients, and then selecting the specific linear combination coefficient from the multiple linear combination coefficients;

producing a classification vector by using the linear combination of the specific linear combination coefficient and the elements of the specific dictionary and determining the type of the object image in the partial image by using the classification vector;

performing, on each of the partial images plurally extracted from the entire first image, the selection of the specific linear combination coefficient and the determination of the type of the object image using the classification vector and the multiple vector data; and

producing the second image depending on the type of the object image determined in each of the partial images.

4. The image processing method according to claim 3, wherein the method determines, as the type of the object image, a type shown by one of the multiple vector data in which a degree of similarity of the classification vector is highest.

5. The image processing method according to claim 3, further comprising:

extracting multiple training partial images from the first training image;

producing an AC component that is acquired by subtracting, from pixel values distributed in each of the multiple training partial images, an average of the pixel values;

converting the AC component of each of the multiple training partial images into a vector to produce an AC vector;

performing, on each of the multiple training partial images, a process to combine the AC vector with the vector data showing the type of the object image of the training partial image from which the AC vector is produced so as to produce a training vector and thereby producing multiple training vectors;

clustering the multiple training vectors to produce multiple clusters each containing the training vectors similar to one another;

performing, on each of the multiple clusters, a process to produce a dictionary matrix from the cluster by using a dictionary learning algorithm to produce multiple dictionary matrices; and

producing the first and second dictionaries from each of the multiple dictionary matrices.

6. A non-transitory computer-readable storage medium storing an image processing program as a computer program to cause a computer to execute image processing, the image processing comprising:

performing, on each of multiple first dictionaries produced by dictionary learning, a process to approximate a first image by linear combination of elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring multiple linear combination coefficients;

calculating, for each of the multiple linear combination coefficients, a ratio between a largest coefficient element and a second-largest coefficient element among respective coefficient elements of the respective linear combination coefficients, the largest coefficient element having an absolute value that is largest among the respective coefficient elements and the second-largest coefficient element having an absolute value that is second-largest among the respective coefficient elements, thereby obtaining respective ratios of the multiple linear combination coefficients;

selecting a specific linear combination coefficient, among the multiple linear combination coefficients, having the ratio that is largest among the respective ratios of the multiple linear combination coefficients;

selecting, from multiple second dictionaries corresponding to the first dictionaries, a specific dictionary corresponding to the first dictionary for which the specific linear combination coefficient is selected; and

converting the first image into a second image by using linear combination of the specific linear combination coefficient and elements of the specific dictionary.

7. The non-transitory computer-readable storage medium according to claim 6,

wherein:

the second image is an image showing a result of classifying multiple types of object images contained in the first image, and

the image processing further comprises;

providing the multiple first and second dictionaries produced by the dictionary learning that uses a first training image relevant to the first image and multiple vector data as a second training image showing the types of the respective object images;

performing, on each of the multiple first dictionaries, a process to approximate a partial image extracted from the first image by the linear combination of the elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring the multiple linear combination coefficients, and then selecting the specific linear combination coefficient from the multiple linear combination coefficients;

producing a classification vector by using the linear combination of the specific linear combination coefficient and the elements of the specific dictionary and determining the type of the object image in the partial image by using the classification vector;

performing, on each of the partial images plurally extracted from the entire first image, the selection of the specific linear combination coefficient and the determination of the type of the object image using the classification vector and the multiple vector data; and

producing the second image depending on the type of the object image determined in each of the partial images.

8. An image processing apparatus configured to perform image processing to convert an image, the image processing apparatus comprising:

a coefficient calculator configured to perform, on each of multiple first dictionaries produced by dictionary learning, a process to approximate a first image by linear combination of elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring multiple linear combination coefficients;

a coefficient selector configured to calculate, for each of the multiple linear combination coefficients, a ratio between a largest coefficient element and a second-largest coefficient element among respective coefficient elements of the respective linear combination coefficients, the largest coefficient element having an absolute value that is largest among the respective coefficient elements and the second-largest coefficient element having an absolute value that is second-largest among the respective coefficient elements, thereby obtaining respective ratios of the multiple linear combination coefficients, and to select a specific linear combination coefficient, among the multiple linear combination coefficients, having the ratio that is largest among the respective ratios of the multiple linear combination coefficients;

a dictionary selector configured to select, from multiple second dictionaries corresponding to the first dictionaries, a specific dictionary corresponding to the first dictionary for which the specific linear combination coefficient is selected; and

an image producer configured to convert the first image into a second image by using linear combination of the specific linear combination coefficient and elements of the specific dictionary.

9. The image processing apparatus according to claim 8,

wherein:

the second image is an image showing a result of classifying multiple types of object images contained in the first image, and

further comprising a provider configured to provide the multiple first and second dictionaries produced by the dictionary learning that uses a first training image relevant to the first image and multiple vector data as a second training image showing the types of the respective object images;

wherein;

the coefficient calculator is configured to perform, on each of the multiple first dictionaries, a process to approximate a partial image extracted from the first image by the linear combination of the elements of the first dictionary so as to produce a linear combination coefficient and thereby acquiring the multiple linear combination coefficients, and then selecting the specific linear combination coefficient from the multiple linear combination coefficients; and

the image producer is configured (a) to producing a classification vector by using the linear combination of the specific linear combination coefficient and the elements of the specific dictionary and determining the type of the object image in the partial image by using the classification vector, (b) to perform, on each of the partial images plurally extracted from the entire first image, the selection of the specific linear combination coefficient and the determination of the type of the object image using the classification vector and the multiple vector data, and (c) to produce the second image depending on the type of the object image determined in each of the partial images.