Information processing device and non-transitory computer readable medium

• US 10,482,352 B2
• Filed: 08/31/2017
• Issued: 11/19/2019
• Est. Priority Date: 03/15/2017
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Osada, Genki, Thapliya, Roshan
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/693,093
• Publication Number:
  US 20180268262A1
• Time in Patent Office: 810 Days
• CPC Class Codes: G06F 18/2155 characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Information processing including, a first data acquirer that acquires first observational data; a second data acquirer that acquires second observational data; and a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model.

Claims

1. An information processing device comprising:

a first data acquirer that acquires first observational data;

a second data acquirer that acquires second observational data; and

a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model,

wherein the processor further generates the first observational data and the second observational data from the latent factor obtained through the inference, and interpolates a missing portion of at least one of the first observational data and the second observational data respectively acquired by the first data acquirer and the second data acquirer, by using at least one of the generated first observational data and second observational data, and outputs the interpolated portion.

2. The information processing device according to claim 1,

wherein the processor performs the machine learning by a combination of a recurrent neural network (RNN) and a variational autoencoder (VAE).

3. The information processing apparatus according to claim 2,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

4. The information processing device according to claim 1,

wherein the processor infers the latent factor using the training data of the first observational data and the second observational data, and performs the machine learning by comparing a label obtained by the inference with a label given as correct data.

5. The information processing apparatus according to claim 4,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

6. The information processing device according to claim 1,

wherein the processor calculates a first characteristic quantity of the first observational data acquired by the first data acquirer and a second characteristic quantity of the second observational data acquired by the second data acquirer, infers the latent factor based on the first characteristic quantity and the second characteristic quantity, and calculates the first characteristic quantity and the second characteristic quantity so that the first characteristic quantity and the second characteristic quantity are correlated to each other.

7. The information processing apparatus according to claim 6,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

8. The information processing apparatus according to claim 1,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

9. The information processing device according to claim 1,

wherein the first observational data is motion data on a person,

the second observational data is voice data on the person, and

the processor interpolates a missing portion of the second observational data and outputs the interpolated portion.

10. An information processing device comprising:

a first data acquirer that acquires first observational data;

a second data acquirer that acquires second observational data; and

a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model,

wherein the processor infers the latent factor using the training data of the first observational data and the second observational data, generates the first observational data and the second observational data from the latent factor obtained through the inference, and performs the machine learning by comparing the generated first observational data and second observational data with the first observational data and the second observational data acquired by the first data acquirer and the second data acquirer, respectively.

11. The information processing apparatus according to claim 10,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

12. An information processing device comprising:

a first data acquirer that acquires first observational data;

a second data acquirer that acquires second observational data; and

a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model,

wherein the processor performs the machine learning by a combination of a recurrent neural network (RNN) and a variational autoencoder (VAE), and

wherein the processor uses the RNN to calculate a first characteristic quantity of the first observational data acquired by the first data acquirer and a second characteristic quantity of the second observational data acquired by the second data acquirer, and uses the VAE to infer the latent factor using the first characteristic quantity and the second characteristic quantity.

13. The information processing apparatus according to claim 12,

wherein the first observational data is motion data on a person, and

the second observational data is voice data on the person.

14. An information processing device comprising:

a first data acquirer that acquires first observational data;

a second data acquirer that acquires second observational data; and

a processor that performs machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data, and that infers and outputs the emotion from at least one of the first observational data and the second observational data using a trained model,

wherein the processor performs the machine learning by a combination of a recurrent neural network (RNN) and a variational autoencoder (VAE),

wherein the first observational data is motion data on a person, and the second observational data is voice data on the person, and

wherein the processor converts the interpolated second observational data to text data, and outputs the text data.

15. A non-transitory computer readable medium storing a program causing a computer to execute a process, the process comprising:

inputting first observational data and second observational data to the computer;

performing machine learning by semi-supervised learning to output an emotion of a person as a latent factor using training data of the first observational data and the second observational data;

inferring and outputting the emotion from at least one of the first observational data and the second observational data using a trained model;

generating the first observational data and the second observational data from the latent factor obtained through the inference;

interpolating a missing portion of at least one of the first observational data and the second observational data respectively input to the computer, by using at least one of the generated first observational data and second observational data; and

outputting the interpolated portion.