AROUSAL STATE CLASSIFICATION MODEL GENERATING DEVICE, AROUSAL STATE CLASSIFYING DEVICE, AND WARNING DEVICE

• US 20100036290A1
• Filed: 01/21/2008
• Published: 02/11/2010
• Est. Priority Date: 01/19/2007
• Status: Active Grant

• Current Assignee: Asahi Kasei Kabushiki Kaisha (Asahi Kasei Corporation)
• Original Assignee: Asahi Kasei Kabushiki Kaisha (Asahi Kasei Corporation)
• Inventors: Nopsuwanchai, Roongroj, Noguchi, Yoshihiro
• Granted Patent: US 8,400,313 B2
• Time in Patent Office: Days
• US Class Current: 600/595
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

There is provided an arousal state classification model generating device for generating a blink waveform pattern model and an arousal state pattern model based on data on a blink waveform, which are preferably used for accurately estimating an arousal level with respect to unspecified object persons, an arousal state classifying device for classifying the arousal state of an object person, and a warning device. In the arousal state classification model generating device, a first pattern model is generated by learning a statistical model by using as learning data first feature data extracted from the blink data of at least one of the eyes of each object person at the time of blinking and blink waveform identification information. A second pattern model is generated by learning a statistical model by using as learning data second feature data including data on the occurrence ratio of each specific type of blink waveform in the sequence of analysis intervals and arousal state information data in which arousal state information indicating the arousal state of each object person is provided to each sequence of analysis intervals.

Claims

1-23. -23. (canceled)

24. An arousal state classification model generating device for generating a statistical model to determine an arousal state of an object person, the arousal state classification model generating device characterized by comprising:

learning data storage means for storing first feature data extracted from blink data of at least one eye of each object person at the time of blinking, and blink waveform identification information data in which blink waveform identification information indicating a specific type of blink waveform is provided to each blink in the blink data;

blink waveform pattern model generation means for learning a statistical model by using as learning data the first feature data and the blink waveform identification information data stored in the learning data storage means, and generating a first pattern model having as an input the first feature data and having as an output a likelihood for the blink waveform identification information in relation to the first feature data;

feature data generation means for generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms in an analysis interval based on the blink waveform identification information data stored in the learning data storage means; and

arousal state pattern model generation means for learning a statistical model by using as learning data the second feature data generated by the feature data generation means and arousal state information data in which arousal state information indicating the arousal state of the object person is provided to each sequence of the analysis intervals, and generating a second pattern model having as an input the second feature data and having as an output a likelihood for the arousal state information in relation to the second feature data.

25. The arousal state classification model generating device according to claim 24, characterized in that the blink data is electro-oculogram (EOG) waveform data or moving picture of eye region.

26. The arousal state classification model generating device according to any one of claims 24 and 25, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.

27. The arousal state classification model generating device according to claim 24, characterized in that an HMM (Hidden Markov Model) is used for the statistical model.

28. The arousal state classification model generating device according to claim 24, further comprising:

blink data storage means for storing the blink data of at least one eye of the object person at the time of blinking;

feature data extraction means for extracting the first feature data from the blink data acquired from the blink data storage means;

first pattern model storage means for storing the first pattern model generated by the blink waveform pattern model generation means;

second feature data storage means for storing the second feature data generated by the feature data generation means; and

second pattern model storage means for storing the second pattern model generated by the arousal state pattern model generation means.

29. An arousal state classifying device for classifying an arousal state of an object person, the arousal state classifying device characterized by comprising:

blink data acquisition means for acquiring blink data of at least one eye of the object person at the time of blinking;

a first pattern model generated by the arousal state classification model generating device according to claim 24;

first feature data extraction means for extracting first feature data corresponding to the first pattern model from the blink data acquired by the blink data acquisition means;

blink waveform identification means for identifying a specific type of blink waveform corresponding to the first feature data extracted by the first feature data extraction means based on the first feature data and the first pattern model;

second feature data generation means for generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms based on an identification result by the blink waveform identification means with respect to the blink data of the object person acquired in a sequence of analysis intervals;

a second pattern model generated by the arousal state classification model generating device; and

arousal state classification means for classifying the arousal state of the object person based on the second feature data generated by the second feature data generation means and the second pattern model.

30. The arousal state classifying device according to claim 29, characterized in that the blink data is electro-oculogram (EOG) waveform data or moving picture of eye region.

31. The arousal state classifying device according to any one of claims 27 and 28, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.

32. The arousal state classifying device according to any one of claims 27 to 29, characterized in that an HMM (hidden Markov model) is used for the first pattern model and the second pattern model.

33. A warning device comprising:

the arousal state classifying device according to claim 29; and

warning means for warning the object person based on a classification result of the arousal state in the arousal state classifying device.

34. A vehicle comprising the warning device according to claim 33.

35. An arousal state classification model generating method for generating a statistical model to determine an arousal state of an object person, the arousal state classification model generating method characterized by comprising:

a learning data storage step of storing first feature data extracted from blink data of at least one eye of each object person at the time of blinking, and blink waveform identification information data in which blink waveform identification information indicating a specific type of blink waveform is provided to each blink in the blink data;

a blink waveform pattern model generation step of learning a statistical model by using as learning data the first feature data and the blink waveform identification information data stored in the learning data storage step, and generating a first pattern model having as an input the first feature data and having as an output a likelihood for the blink waveform identification information in relation to the first feature data;

a feature data generation step of generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms in an analysis interval based on the blink waveform identification information data stored in the learning data storage step; and

an arousal state pattern model generation step of learning a statistical model by using as learning data the second feature data generated in the feature data generation step and arousal state information data in which arousal state information indicating the arousal state of the object person is provided to each sequence of the analysis intervals, and generating a second pattern model having as an input the second feature data and having as an output a likelihood for the arousal state information in relation to the second feature data.

36. The arousal state classification model generating method according to claim 35, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.

37. The arousal state classification model generating method according to any one of claims 35 and 36, further comprising:

a blink data storage step of storing the blink data of at least one eye of the object person at the time of blinking in blink data storage means;

a feature data extraction step of extracting the first feature data from the blink data stored in the blink data storage means;

a first pattern model storage step of storing the first pattern model generated in the blink waveform pattern model generation step;

a second feature data storage step of storing the second feature data generated in the feature data generation step; and

a second pattern model storage step of storing the second pattern model generated in the arousal state pattern model generation step.

38. An arousal state classifying method for classifying an arousal state of an object person, the arousal state classifying method comprising:

a blink data acquisition step of acquiring blink data of at least one eye of the object person at the time of blinking;

a first feature data extraction step of extracting first feature data corresponding to a first pattern model generated in the arousal state classification model generating method according to claim 35 from the blink data acquired in the blink data acquisition step;

a blink waveform identification step of identifying a specific type of blink waveform corresponding to the first feature data extracted in the first feature data extraction step based on the first feature data and the first pattern model;

a second feature data generation step of generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms based on an identification result in the blink waveform identification step with respect to the blink data of the object person acquired in a sequence of analysis intervals; and

an arousal state classification step of classifying the arousal state of the object person based on the second feature data generated in the second feature data generation step and a second pattern model generated in the arousal state classification model generating method.

39. The arousal state classifying method according to claim 38, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.

40. An arousal state classification model generating program for generating a statistical model to determine an arousal state of an object person, causing a computer to function as:

a learning data storage means for storing first feature data extracted from blink data of at least one eye of each object person at the time of blinking, and blink waveform identification information data in which blink waveform identification information indicating a specific type of blink waveform is provided to each blink in the blink data;

blink waveform pattern model generation means for learning a statistical model by using as learning data the first feature data and the blink waveform identification information data stored in the learning data storage means, and generating a first pattern model having as an input the first feature data and having as an output a likelihood for the blink waveform identification information in relation to the first feature data;

feature data generation means for generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms an analysis intervals based on the blink waveform identification information data stored by the learning data storage means; and

arousal state pattern model generation means for learning a statistical model by using as learning data the second feature data generated by the feature data generation means and arousal state information data in which arousal state information indicating the arousal state of the object person is provided to each of the sequences of analysis intervals, and generating a second pattern model having as an input the second feature data and having as an output a likelihood for the arousal state information in relation to the second feature data.

41. The arousal state classification model generating program according to claim 40, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.

42. The arousal state classification model generating program according to any one of claims 40 and 41, causing the computer to function as:

blink data storage means for storing the blink data of at least one eye of the object person at the time of blinking;

feature data extraction means for extracting the first feature data from the blink data stored in the blink data storage means;

first pattern model storage means for storing the first pattern model generated by the blink waveform pattern model generation means;

second feature data storage means for storing the second feature data generated by the feature data generation means; and

second pattern model storage means for storing the second pattern model generated by the arousal state pattern model generation means.

43. An arousal state classifying program for classifying an arousal state of an object person, causing a computer to function as:

blink data acquisition means for acquiring blink data of at least one eye of the object person at the time of blinking;

first feature data extraction means for extracting first feature data corresponding to a first pattern model generated by the arousal state classification model generating program according to claim 40 from the blink data acquired by the blink data acquisition means;

blink waveform identification means for identifying a specific type of blink waveform corresponding to the first feature data extracted in the first feature data extraction means based on the first feature data and the first pattern model;

second feature data generation means for generating second feature data including data on an occurrence ratio of each of the specific types of blink waveforms based on an identification result by the blink waveform identification means with respect to the blink data of the object person acquired in a sequence of analysis intervals; and

arousal state classification means for classifying the arousal state of the object person based on the second feature data generated in the second feature data generation means and a second pattern model generated by the arousal state classification model generating program.

44. The arousal state classifying program according to claim 43, characterized in that the data on the occurrence ratio is a time variation of the occurrence ratio or a time variation of the number of occurrences.