Feeling monitoring system

• US 9,465,978 B2
• Filed: 04/04/2013
• Issued: 10/11/2016
• Est. Priority Date: 04/10/2012
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Nozoe, Satoshi, Hachisuka, Satori, Enya, Takeshi, Sakai, Makoto
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): HU, FRED H.
• Application Number:
  US14/390,806
• Publication Number:
  US 20150078632A1
• Time in Patent Office: 1,286 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/7415 Sound rendering of measured values, e.g. by pitch or volume variation A61B5/741 takes precedence
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A feeling monitoring system includes a facial image acquisition portion, a feeling information acquisition section, and a feeling display generation section. The facial image acquisition portion is capable of acquiring a facial image of an occupant individually sitting in each seating position of two or more seating positions in a vehicle. The feeling information acquisition section acquires feeling information representing the feeling of the occupant based on the facial image acquired from the facial image acquisition portion. The feeling display generation section generates a feeling display corresponding to the feeling information acquired by the feeling information acquisition section.

Claims

1. A feeling monitoring system comprising:

a facial image acquisition portion that acquires a facial image of an occupant individually sitting in each seating position of a plurality of seating positions in a vehicle, the plurality of seating positions including a driver'"'"'s seat in a front area of the vehicle, a front-passenger seat in the front area of the vehicle, and a rear-passenger seat in a rear area of the vehicle;

a computer configured to;

detect, in a seating position detection section, the seating position of the occupant based on the facial image acquired from the facial image acquisition portion,

acquire, by a feeling information acquisition section, feeling information representing feeling of the occupant based on the facial image acquired from the facial image acquisition portion, and

generate, by a feeling display generation section, a feeling display corresponding to the feeling information acquired by the feeling information acquisition section; and

a storage portion that stores a plurality of feeling-type bases, wherein feeling-type basis is a basis pertaining to a type of feeling information, the feeling-type basis being previously stored correspondingly for each of the driver'"'"'s seat, the front-passenger seat, and the rear-passenger seat included in the plurality of seating positions, the feeling-type basis previously stored corresponding for each one of the seating positions is configured to include a plurality of types of feeling, wherein

the feeling display generation section;

determines which one of the plurality of feeling-type bases that were previously stored in the storage portion corresponds to the seating position that was detected by the seating position detection section,

determines which one of the plurality of types of feeling, included in the feeling-type basis determined to correspond to the seating position that was detected, agrees with the feeling information acquired by the feeling information acquisition section,

generates the feeling display that corresponds to the feeling information that is determined to agree with one of the plurality of types of feeling included in the feeling-type basis stored in the storage portion, wherein the feeling-type basis is the one of the plurality of feeling-type bases determined to correspond to a subject seating position detected by the seating position detection section among the plurality of seating positions, responsive to the feeling information acquired by the feeling information acquisition section,

when the subject seating position among the plurality of seating positions is detected as the seating position of the occupant by the seating position detection section and the feeling display corresponding to the feeling information of the occupant is generated by the feeling display generation section,

the feeling display generation section uses, as the feeling-type basis that corresponds to the feeling information of the occupant, the feeling-type basis previously stored in the storage portion for the subject seating position, as detected by the seating position detection section, of the occupant, the feeling display generation section supports different feeling displays based on identical feeling information for an identical occupant in different seating positions.

2. The feeling monitoring system according to claim 1,

wherein the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein the first storage portion stores the basis pertaining to the type of the feeling information specific to the occupant, and

wherein the feeling display generation section uses the basis pertaining to the type of the feeling information specific to the occupant based on a result of identification by the occupant identification section.

3. The feeling monitoring system according to claim 1,

wherein the storage portion further stores correspondence between an intensity of the feeling and a mode of the feeling display,

wherein the feeling information contains the intensity of the feeling, and

wherein the feeling display generation section determines the mode of the feeling display generated based on the correspondence between the intensity of the feeling and the mode of the feeling display.

4. The feeling monitoring system according to claim 3,

wherein the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein the correspondence stored in the storage portion is specific to the occupant, and

wherein the feeling display generation section uses the correspondence specific to the occupant based on a result of identification by the occupant identification section.

5. The feeling monitoring system according to claim 1, wherein the computer is further configured to:

cause a display to display the feeling display in connection with the seating position of the occupant corresponding to the feeling display.

6. The feeling monitoring system according to claim 5,

wherein the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein:

the feeling display generation section generates the feeling display corresponding to the feeling information acquired by the feeling information acquisition section based on the occupant identified by the occupant identification section; and

the display portion shows the feeling display in connection with the occupant identified by the occupant identification section as well as in connection with the seating position of the occupant identified by the occupant identification section.

19. The feeling monitoring system according to claim 1, wherein the feeling display generation section generates the feeling display which simultaneously represents the feeling information for each of a plurality of occupants which are detected by the seating position detection section in the two or more seating positions in the vehicle.

7. A feeling monitoring system comprising:

a facial image acquisition portion that acquires a facial image of an occupant individually sitting in each seating position of a plurality of seating positions in a vehicle, the plurality of seating positions including a driver'"'"'s seat in a front area of the vehicle, a front-passenger seat in the front area of the vehicle, and a rear-passenger seat in a rear area of the vehicle;

a computer configured to

detect, in a seating position detection section, the seating position of the occupant based on the facial image acquired by the facial image acquisition portion,

acquire, in a feeling information acquisition section, feeling information representing a feeling of the occupant based on the facial image acquired from the facial image acquisition portion, and

generate, in a feeling display generation section, a feeling display corresponding to the feeling information acquired by the feeling information acquisition section; and

a storage portion that stores a plurality of feeling-intensity-mode correspondences, wherein a feeling-intensity-mode correspondence is a correspondence between (i) a feeling intensity contained in the feeling information and (ii) a feeling-display mode that is a mode of feeling display, the feeling-intensity-mode correspondence being previously stored correspondingly for each of the plurality of seating positions, the feeling-intensity-mode correspondence previously stored corresponding for each one of the seating positions is configured to include correspondence for a plurality of feeling intensities,

wherein;

the feeling display generation section

determines which one of the plurality of feeling-intensity-mode correspondences that were previously stored in the storage portion corresponds to the seating position that was detected by the seating position detection section,

determines which one of the plurality of feeling intensities, included in the feeling-intensity-mode correspondence determined to correspond to the seating position that was detected, agrees with the feeling information acquired by the feeling information acquisition section,

generates the feeling display in the feeling-display mode corresponding to the feeling information that is determined to agree with the one of the plurality of feeling intensities included in the feeling-intensity-mode correspondence stored in the storage portion, wherein the feeling-intensity-mode correspondence is the one of the plurality of feeling-intensity-mode correspondences determined to correspond to a subject seating position detected by the seating position detection section among the plurality of seating positions, responsive to the feeling information acquired by the feeling information acquisition section,

when the subject seating position among the plurality of seating positions is detected as the seating position of the occupant by the seating position detection section and the feeling display corresponding to the feeling information of the occupant is generated by the feeling display generation section,

the feeling display generation section uses, as the feeling-intensity-mode correspondence that corresponds to the feeling information of the occupant, the feeling-intensity-mode correspondence previously stored in the storage portion for the subject seating position, as detected by the seating position detection section, of the occupants, the display generation section supports different feeling displays based on identical feeling information for an identical occupant in different seating positions.

8. The feeling monitoring system according to claim 7,

wherein the storage portion further stores a basis pertaining to a type of feeling information, and

wherein the feeling display generation section generates the feeling display with respect to the feeling information agreeing with the basis pertaining to the type of the feeling information.

9. The feeling monitoring system according to claim 8,

wherein the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein the basis pertaining to the type of the feeling information stored in the storage portion is specific to the occupant, and

wherein the feeling display generation section uses the basis pertaining to the type of the feeling information specific to the occupant based on a result of identification by the occupant identification section.

10. The feeling monitoring system according to claim 7,

wherein the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein the correspondence stored in the storage portion is specific to the occupant, and

wherein the feeling display generation section uses the correspondence specific to the occupant based on a result of identification by the occupant identification section.

11. The feeling monitoring system according to claim 7, wherein the computer is further configured to:

cause a display to display the feeling display in connection with the seating position of the occupant corresponding to the feeling display.

12. The feeling monitoring system according to claim 11, wherein

the computer is further configured to identify, in an occupant identification section, the occupant individually,

wherein the feeling display generation section generates the feeling display corresponding to the feeling information acquired by the feeling information acquisition section based on the occupant identified by the occupant identification section;

and

wherein the display portion shows the feeling display in connection with the occupant identified by the occupant identification section as well as in connection with the seating position of the occupant identified by the occupant identification section.

20. The feeling monitoring system according to claim 7, wherein the feeling display generation section generates the feeling display which simultaneously represents the feeling information for each of a plurality of occupants which are detected by the seating position detection section in the two or more seating positions in the vehicle.

13. A feeling monitoring method for monitoring feelings of occupants in a vehicle, the vehicle has a plurality of seating positions, comprising:

acquiring, by a facial image acquisition portion, a facial image of an occupant individually sitting in each seating position of a plurality of seating positions in a vehicle, the plurality of seating positions including a driver'"'"'s seat in a front area of the vehicle, a front-passenger seat in the front area of the vehicle, and a rear-passenger seat in a rear area of the vehicle;

detecting, in a seating position detection section of a computer, the seating position of the occupant based on the facial image acquired from the facial image acquisition portion;

acquiring, by a feeling information acquisition section of the computer, feeling information representing feeling of the occupant based on the facial image acquired from the facial image acquisition portion;

generating, by a feeling display generation section of the computer, a feeling display corresponding to the feeling information acquired by the feeling information acquisition section; and

providing a storage portion that stores a plurality of feeling-type bases, wherein a feeling-type basis is a basis pertaining to a type of feeling information, the feeling-type basis being previously stored corresponding for each of the driver'"'"'s seat, the front-passenger seat, and the rear-passenger seat included in the plurality of seating positions, the feeling-type basis previously stored correspondingly for each one of the seating positions is configured to include a plurality of types of feeling, wherein

the feeling display generation section;

determines which one of the plurality of feeling-type bases that were previously stored in the storage portion corresponds to the seating position that was detected by the seating position detection section,

determines which one of the plurality of types of feeling, included in the feeling-type basis determined to correspond to the seating position that was detected, agrees with the feeling information acquired by the feeling information acquisition section,

generates the feeling display that corresponds to the feeling information that is determined to agree with one of the plurality of types of feelings included in the feeling-type basis stored in the storage portion, wherein the feeling-type basis is the one of the plurality of feeling-type bases determined to correspond to a subject seating position detected by the seating position detection section among the plurality of seating positions, responsive to the feeling information acquired by the feeling information acquisition section,

when the subject seating position among the plurality of seating positions is detected as the seating position of the occupant by the seating position detection section and the feeling display corresponding to the feeling information of the occupant is generated by the feeling display generation section,

the feeling display generation section uses, as the feeling-type basis that the feeling information of the occupant, the feeling-type basis previously stored in the storage portion for the subject seating position, as detected by the seating position detection section, of the occupant, the feeling display generation section supports different feeling displays based on identical feeling information for an identical occupant in different seating positions.

14. The feeling monitoring method according to claim 13, further comprising

identifying, in an occupant identification section, the occupant individually,

wherein the first storage portion stores the basis pertaining to the type of the feeling information specific to the occupant, and

wherein the feeling display generation section uses the basis pertaining to the type of the feeling information specific to the occupant based on a result of identification by the occupant identification section.

15. The feeling monitoring method according to claim 13,

wherein the storage portion further stores correspondence between an intensity of the feeling and a mode of the feeling display,

wherein the feeling information contains the intensity of the feeling, and

wherein the feeling display generation section determines the mode of the feeling display generated based on the correspondence between the intensity of the feeling and the mode of the feeling display.

16. The feeling monitoring method according to claim 15, further comprising,

identifying, in an occupant identification section of the computer, the occupant individually,

wherein the correspondence stored in the storage portion is specific to the occupant, and

wherein the feeling display generation section uses the correspondence specific to the occupant based on a result of identification by the occupant identification section.

17. The feeling monitoring method according to claim 13, further comprising

causing a display that interfaces to the computer to display the feeling display in connection with the seating position of the occupant corresponding to the feeling display.

18. The feeling monitoring method according to claim 17, further comprising

identifying, in an occupant identification section of the computer, the occupant individually,

wherein:

the feeling display generation section generates the feeling display corresponding to the feeling information acquired by the feeling information acquisition section based on the occupant identified by the occupant identification section; and

the display portion shows the feeling display in connection with the occupant identified by the occupant identification section as well as in connection with the seating position of the occupant identified by the occupant identification section.