Data category identification method and apparatus based on deep neural network

• US 10,296,827 B2
• Filed: 11/18/2015
• Issued: 05/21/2019
• Est. Priority Date: 05/31/2014
• Status: Active Grant

• Current Assignee: Xfusion Digital Technologies Company Limited
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Tian, Guangjian, He, Cheng, Fan, Wei
• Primary Examiner(s): Chang, Li Wu
• Application Number:
  US14/944,294
• Publication Number:
  US 20160071010A1
• Time in Patent Office: 1,280 Days
• Field of Search: None
• CPC Class Codes: G06N 3/06 Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A deep neural network to which data category information is added is established locally, to-be-identified data is input to an input layer of the deep neural network generated based on the foregoing data category information, and information of a category to which the to-be-identified data belongs is acquired, where the information of the category is output by an output layer of the deep neural network. A deep neural network is established based on data category information, such that category information of to-be-identified data is conveniently and rapidly obtained using the deep neural network, thereby implementing a category identification function of the deep neural network, and facilitating discovery of an underlying law of the to-be-identified data according to the category information of the to-be-identified data.

Claims

1. A data category identification method based on a deep neural network for an apparatus comprising a memory and a processor, the method comprising:

establishing an initial deep neural network;

generating a linear category analysis function after data category information is added to a locally saved initial linear category analysis function in the memory, the linear category analysis function being generated from an input training sample vector set;

generating an optimization function of the initial deep neural network from a locally saved unsupervised auto-encoding model optimization function in the memory and the linear category analysis function;

acquiring a parameter of the initial deep neural network from the optimization function of the initial deep neural network;

generating the deep neural network from a locally saved classification neural network in the memory, the initial deep neural network, and the parameter of the initial deep neural network, the deep neural network having a multi-layer network structure comprising at least an input layer and an output layer;

receiving to-be-identified data from a user device and inputting the to-be-identified data to the input layer of the deep neural network, the to-be-identified data comprising image information or text information;

applying the deep neural network to the image information or the text information comprising the to-be-identified data to obtain information for a category of the to-be-identified data;

determining information for the category of the to-be-identified data in the image information or the text information in response to applying the deep neural network to the image information or the text information; and

outputting the information for the category by the output layer of the deep neural network to permit categorization of the to-be-identified data, the linear category analysis function ζ

_lda (W) being;

2. The method of claim 1, wherein the optimization function of the initial deep neural network is:

ζ

=α

ζ

_ae(W)+(1−

α

)ζ

_lda(W)

α

being a coefficient of the optimization function of the initial deep neural network, and is preset and acquired based on an application scenario, ζ

_ae (W) being the unsupervised auto-encoding model optimization function, ζ

_lda (W) being the linear category analysis function, and ζ

being the optimization function of the initial deep neural network.

3. The method of claim 1, wherein acquiring the parameter of the initial deep neural network using the optimization function of the initial deep neural network comprises:

acquiring a gradient corresponding to the optimization function of the initial deep neural network; and

acquiring, using a gradient descent algorithm or a quasi-Newton algorithm, the parameter of the initial deep neural network based on the gradient corresponding to the optimization function of the initial deep neural network.

4. The method of claim 1, wherein establishing the deep neural network based on the locally saved classification neural network, the initial deep neural network, and the parameter of the initial deep neural network comprises:

superimposing the classification neural network onto the initial deep neural network to generate a superimposed initial deep neural network that is obtained after superimposition processing; and

establishing, using a backpropagation algorithm, the deep neural network based on the parameter of the initial deep neural network and the initial deep neural network that is obtained after superimposition processing.

10. The method of claim 1, wherein generating the linear category analysis function after data category information is added to the locally saved initial linear category analysis function comprises:

performing relaxation processing on the initial linear category analysis function using a relaxation algorithm;

performing normalization processing on the input training sample vector set to generate a normalized training sample vector set; and

substituting the normalized training sample vector set into the initial linear category analysis function on which relaxation processing has been performed to generate the linear category analysis function.

5. A data category identification apparatus based on a deep neural network, comprising:

a memory comprising instructions; and

a processor coupled to the memory, wherein the instructions cause the processor to be configured to;

establish an initial deep neural network;

generate a linear category analysis function after data category information is added to a locally saved initial linear category analysis function, the linear category analysis function being generated from an input training sample vector set;

generate an optimization function of the initial deep neural network, the optimization function being acquired from a locally saved unsupervised auto-encoding model optimization function and the linear category analysis function;

acquire a parameter of the initial deep neural network from the optimization function of the initial deep neural network;

generate the deep neural network from a locally saved classification neural network, the initial deep neural network, and the parameter of the initial deep neural network, the deep neural network having a multi-layer network structure comprising at least an input layer and an output layer;

receive to-be-identified data from a user device and inputting the to-be-identified data to the input layer of the deep neural network, the to-be-identified data comprising image information or text information;

apply the deep neural network to the image information or the text information comprising the to-be-identified data to obtain information for a category of the to-be-identified data;

determine information for the category of the to-be-identified data in the image information or the text information in response to applying the deep neural network to the image information or the text information; and

output the information for the category by the output layer of the deep neural network to permit categorization of the to-be-identified data, the linear category analysis function ζ

_lda (W) being;

6. The apparatus of claim 5, wherein the optimization function of the initial deep neural network is:

ζ

=α

ζ

_ae(W)+(1−

α

)ζ

_lda(W)

α

being a coefficient of the optimization function of the initial deep neural network, and is preset and acquired based on an application scenario, ζ

_ae (W) being the unsupervised auto-encoding model optimization function, ζ

_lda (W) being the linear category analysis function, and ζ

being the optimization function of the initial deep neural network.

7. The apparatus of claim 5, wherein the instructions further cause the processor to be configured to:

acquire, based on the optimization function of the initial deep neural network and using a backpropagation algorithm, a gradient corresponding to the optimization function of the initial deep neural network; and

acquire, using a gradient descent algorithm or a quasi-Newton algorithm, the parameter of the initial deep neural network based on the gradient corresponding to the optimization function of the initial deep neural network.

8. The apparatus of claim 5, wherein the instructions further cause the processor to be configured to:

superimpose the classification neural network onto the initial deep neural network to generate a superimposed initial deep neural network that is obtained after superimposition processing, and

establish, using a backpropagation algorithm, the deep neural network based on the parameter of the initial deep neural network and the initial deep neural network that is obtained after superimposition processing.

11. The apparatus of claim 5, wherein the instructions further cause the processor to be configured to:

perform relaxation processing on the initial linear category analysis function using a relaxation algorithm;

perform normalization processing on the input training sample vector set to generate a normalized training sample vector set; and

substitute the normalized training sample vector set into the initial linear category analysis function on which relaxation processing has been performed to generate the linear category analysis function.

9. A non-transitory computer readable medium storing codes which, when executed by a processor of a network system causes the processor to:

establish an initial deep neural network;

generate a linear category analysis function after data category information is added to a locally saved initial linear category analysis function, the linear category analysis function being generated from an input training sample vector set;

generate an optimization function of the initial deep neural network, the optimization function being acquired from a locally saved unsupervised auto-encoding model optimization function and the linear category analysis function;

acquire a parameter of the initial deep neural network from the optimization function of the initial deep neural network;

generate the deep neural network from a locally saved classification neural network, the initial deep neural network, and the parameter of the initial deep neural network, the deep neural network having a multi-layer network structure comprising at least an input layer and an output layer;

receive to-be-identified data from a user device and input the to-be-identified data to the input layer of the deep neural network, the to-be-identified data comprising image information or text information;

apply the deep neural network to the image information or the text information comprising to-be-identified data to obtain information for a category of the to-be-identified data;

determine information for the category of the to-be-identified data in the image information or the text information in response to applying the deep neural network to the image information or the text information; and

output the information for the category by the output layer of the deep neural network to permit categorization of the to-be-identified data, the linear category analysis function ζ

_lda (W) being;

12. The non-transitory computer readble medium of claim 9, wherein the processor is further configured to:

perform relaxation processing on the initial linear category analysis function using a relaxation algorithm;

perform normalization processing on the input training sample vector set to generate a normalized training sample vector set; and

substitute the normalized training sample vector set into the initial linear category analysis function on which relaxation processing has been performed in order to generate the linear category analysis function.