Image processing apparatus, method of processing image, and program

• US 9,911,067 B2
• Filed: 05/13/2015
• Issued: 03/06/2018
• Est. Priority Date: 05/15/2014
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Hikida, Satoshi
• Primary Examiner(s): Nguyen, Luong T
• Application Number:
  US14/710,700
• Publication Number:
  US 20150332126A1
• Time in Patent Office: 1,028 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 3/4046 using neural networks
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An image processing apparatus calculating to determine one category, to which an input image data belongs, among categories, which are provided to classify image data, includes a convolution processing unit which performs a convolution process using a first coefficient used for calculating to determine the one category and previously learnt; total combination processing units which are prepared for each category performs a total combination process for a processing result obtained by the convolution processing unit using a second coefficient used for calculating to determine the one category and previously learnt, and calculates to determine the one category; and a normalization unit which performs normalization of a processing result obtained by the total combination processing unit for each category, wherein the convolution processing unit performs learning of the first coefficient in common with the categories, and the total combination processing unit performs learning of the second coefficient for each category.

Claims

1. An image processing apparatus that calculates a first certainty factor indicative of a degree of classifying an input image data into at least one first category, the image processing apparatus comprising a hardware processor that executes a program to implement:

a convolution processing unit forming a (n−

1)-th layer of a convolutional neural network, the convolution processing unit being configured to perform a convolution process using a first coefficient previously learnt by the convolution processing unit;

first and second total combination processing units forming a n-th layer of the convolutional neural network, the first and second total combination processing units being

prepared for each of the at least one first category and each of at least one second category, respectively,

configured to perform first and second total combination processes for a processing result obtained by the convolution processing unit using second and third coefficients previously learnt by the first and second total combination processing units, respectively, and

configured to calculate a first candidate certainty factor indicative of the degree of classifying the input image data into the at least one first category and a second candidate certainty factor indicative of a degree of classifying the input image data into the at least one second category, respectively; and

a normalization unit which performs normalization of processing results obtained by the first and second total combination processing units for each of the at least one first category and the at least one second category,

wherein the convolution processing unit performs learning of the first coefficient in common with the at least one first category, and

wherein the first and second total combination processing units perform learning of the second coefficient for each of the at least one first category and learning of the third coefficient for each of the at least one second category.

2. The image processing apparatus according to claim 1,

wherein each of the plurality of categories is a pair of a specific genre and another genre other than the specific genre, and

wherein the first and second total combination processing units calculate the first certainty factor, with which the input image data belongs to the specific genre, and a second certainty factor, with which the input image data belongs to the another genre.

3. The image processing apparatus according to claim 2,

wherein the normalization unit normalizes the first and second certainty factors using [Formula 10] and [Formula 11], respectively, as follows:

4. The image processing apparatus according to claim 1,

wherein the learning is performed by an error inverse propagation method.

5. A method of processing an input image performed in an image processing apparatus that calculates a first certainty factor indicative of a degree of classifying an input image data into at least one first category, the method comprising:

forming a (n−

1)-th layer of a convolutional neural network by performing a convolution process, by a convolution processing unit included in the image processing apparatus, using a first coefficient previously learnt by the convolution processing unit;

performing, by first and second total combination processing units which are included in the image processing apparatus, form a n-th layer of the convolutional neural network, and are respectively prepared for each of the at least one first category and each of at least one second category first and second total combination processes for a processing result obtained by the performing of the convolution process using second and third coefficients previously learnt by the first and second total combination processing units, respectively;

calculating, by the first and second total combination processing units, a first candidate certainty factor indicative of the degree of classifying the input image data into the at least one first category and a second candidate certainty factor indicative of a degree of classifying the input image data into the at least one second category, respectively; and

performing, by a normalization unit, normalization of processing results obtained by the first and second total combination processing units for each of the at least one first category and the at least one second category,

wherein the convolution processing unit performs learning of the first coefficient in common with the at least one first category, and

wherein the first and second total combination processing units perform learning of the second coefficient for each of the at least one first category and learning of the third coefficient for each of the at least one second category.