Lifelog recording system

• US 10,133,917 B2
• Filed: 05/03/2016
• Issued: 11/20/2018
• Est. Priority Date: 09/17/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Enomoto, Takaaki, Nakagawa, Satoshi
• Primary Examiner(s): Owens, Tsion B
• Application Number:
  US15/145,381
• Publication Number:
  US 20170083757A1
• Time in Patent Office: 931 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

A data-to-be-recorded generating section generates a piece of data-to-be-recorded by associating an emotion level of a user estimated through an emotion level estimating section with a piece of lifelog data obtained through a lifelog data obtaining section. A log managing section manages whether the piece of data-to-be-recorded should be stored in a memory device based on a selectivity based on the emotion level.

Claims

1. A lifelog recording system that records, as a piece of lifelog data, life information of a user, the system comprising:

a lifelog data obtaining sensor, which is configured to obtain the piece of lifelog data from an action of the user;

a biometric sensor configured to obtain a biometric signal;

an Electronic Control Unit (ECU) configured to estimate an emotion level, which is a quantified emotion state of the user, based on a result of monitoring the action by the user and the biometric signal;

a memory device, which is configured to store the piece of lifelog data,

wherein the ECU is further configured to;

compute probabilities of the piece of lifelog data corresponding to each of a plurality of types of emotions, based on a feature quantity of the result of monitoring and the biometric signal,

add up the probabilities of each emotion to compute an evaluation value for each emotion used in determining the user'"'"'s emotion level,

estimate an emotion with a highest evaluation value as the user'"'"'s emotion,

generate a piece of data-to-be-recorded by associating the emotion level of the user with the piece of lifelog data,

determine whether the piece of data-to-be-recorded should be stored in the memory device based on the associated emotion level, and

store the piece of data-to-be-recorded in the memory device if the piece of data-to-be-recorded has an emotion level in a predefined recording range.

2. The lifelog recording system according to claim 1, wherein, when an available space of the memory device is smaller than a predetermined value, the ECU is configured to delete, from the memory device, a piece of data-to-be-recorded that has the emotion level in a deletion range.

3. The lifelog recording system according to claim 2, wherein, when there are two or more pieces of data-to-be-recorded that are candidates to be deleted, the ECU is configured to delete the pieces of data preferentially from the piece of data of which a recorded time is older.

4. The lifelog recording system according to claim 1, wherein

the user is a driver of a vehicle,

the lifelog data obtaining sensor is configured to obtain, as the piece of lifelog data, a vehicle operating action by the driver during traveling of the vehicle,

the vehicle operating action includes at least one of an operation of an in-vehicle device and an operation related to traveling of the vehicle, and

the ECU is configured to estimate the emotion level of the driver during traveling of the vehicle.

5. A lifelog recording method that is executed by a lifelog recording system comprising an Electronic Control Unit (ECU) to record, as a piece of lifelog data, life information of a user, the method comprising:

obtaining the piece of lifelog data from an action of the user;

obtaining a biometric signal from a biometric sensor;

estimating an emotion level, which is a quantified emotion state of the user, based on a result of monitoring the action by the user and the biometric signal;

storing the piece of lifelog data in a memory device;

computing probabilities of the piece of the lifelog data corresponding to each of a plurality of types of emotions based on a feature quantity of the result of monitoring and the biometric signal;

adding up the probabilities of each emotion to compute an evaluation value used in determining a user'"'"'s emotion level;

estimating an emotion with the highest evaluation value as the user'"'"'s emotion;

generating a piece of data-to-be-recorded by associating the estimated emotion level of the user with the obtained piece of lifelog data;

determining that the piece of data-to-be-recorded should be stored in the memory device based on the associated emotion level being in a predefined recording range; and

storing the piece of data-to-be-recorded in the memory device.