Multisensory interface control method, multisensory interface control apparatus, and multisensory interface system

• US 10,563,997 B2
• Filed: 10/12/2015
• Issued: 02/18/2020
• Est. Priority Date: 10/23/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Lee, Ryo Kyung, Hachisuka, Keisuke, Okuno, Eiichi
• Primary Examiner(s): Edouard, Patrick N
• Assistant Examiner(s): Shen, Peijie
• Application Number:
  US15/520,324
• Publication Number:
  US 20170314951A1
• Time in Patent Office: 1,590 Days
• Field of Search: None
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G09B 3/045 with invisibly printed correct answer; the answer being rendered visible by special optical means, e.g. filters; mirrors
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions

Abstract

A multisensory interface control method for a multisensory interface that includes a plurality of information transmission channels, each of which performing information transmission by utilizing one or a plurality of five senses is provided. The multisensory interface control method includes: estimating a user status indicating at least one of an emotional status and an attentional status of a user; setting weightings each of which indicates a degree of importance of a corresponding information transmission channel in accordance with the user status estimated; and changing allocations of a plurality of sets of partial content that configure content transmitted via the multisensory interface, the allocations changed in accordance with the weightings.

Claims

1. A multisensory interface control method for a multisensory interface that includes a plurality of information transmission channels, each of which performing information transmission by utilizing at least two of a plurality of five senses, the multisensory interface control method comprising:

estimating a user status indicating at least one of an emotional status and an attentional status of a driver who is driving a vehicle;

setting weightings, each of which indicates a degree of importance of a corresponding information transmission channel in accordance with the user status estimated, wherein a degree of importance of at least one information transmission channel using an inhibitively affected sense is lowered, a degree of importance of at least one information transmission channel other than the information transmission channel using the inhibitively affected sense is raised; and

changing allocations of a plurality of sets of partial content that configure content transmitted via the multisensory interface, the allocations being changed in accordance with the weightings,

wherein;

the multisensory interface control method is implemented in a drive assistance system for the driver;

the at least one information transmission channel using an inhibitively affected sense performs the information transmission via a first sense of the five senses; and

the at least one information transmission channel other than the information transmission channel using the inhibitively affected sense performs the information transmission via a second sense of the five senses.

2. A multisensory interface control apparatus that controls a multisensory interface that includes a plurality of information transmission channels, each of which performing information transmission by utilizing at least two of a plurality of five senses, the multisensory interface control apparatus comprising:

a user status estimation portion that estimates a user status indicating at least one of an emotional status and an attentional status of a driver who is driving a vehicle;

a weighting setting portion that sets weightings, each of which indicates a degree of importance of a corresponding information transmission channel in accordance with the user status estimated by the user status estimation portion, wherein a degree of importance of at least one information transmission channel using an inhibitively affected sense is lowered, a degree of importance of at least one information transmission channel other than the information transmission channel using the inhibitively affected sense is raised; and

a content adjustment portion that changes allocations of a plurality of sets of partial content that configure content transmitted via the multisensory interface, the allocations being changed in accordance with the weightings,

wherein;

the multisensory interface control apparatus is implemented in a drive assistance system for the driver;

the at least one information transmission channel using an inhibitively affected sense performs the information transmission via a first sense of the five senses; and

the at least one information transmission channel other than the information transmission channel using the inhibitively affected sense performs the information transmission via a second sense of the five senses.

3. The multisensory interface control apparatus according to claim 2, wherein:

the weightings are set in accordance with a correlation between sensitivities of the five senses of the user and at least one of the emotional status and the attentional status of the user.

4. The multisensory interface control apparatus according to claim 2, wherein:

the user status estimation portion estimates the user status based on at least one of an expression of the user and a biosignal of the user.

5. A multisensory interface system comprising:

a multisensory interface that includes a plurality of information transmission channels each of which performs information transmission by utilizing at least one of a plurality of five senses; and

the multisensory interface control apparatus according to claim 2.

6. The multisensory interface control apparatus according to claim 2, wherein:

in accordance with the weightings set by the weighting setting portion,

the content adjustment portion changes the allocations of the plurality of sets of partial content, causing an information transmission volume of the information transmission channel having a high degree of importance to be increased, and the information transmission volume of the information transmission channel having a low degree of importance to be decreased.

7. The multisensory interface control apparatus according to claim 2, wherein:

in accordance with the weightings set by the weighting setting portion,

the content adjustment portion allocates a partial content having a high degree of importance to an information transmission channel having a high degree of importance, and allocates a partial content having a low degree of importance to an information transmission channel having a low degree of importance;

a degree of importance including the high degree and the low degree of importance is determined whether an inhibitively affected sense according to the user status is used or not, the user status being defined to show an emotional status and attentional status of the driver.

8. The multisensory interface control apparatus according to claim 2, wherein:

the inhibitively affected sense is a sense of which a sensitivity is lowered in accordance with the user status.

9. A multisensory interface control apparatus that controls a multisensory interface that includes a plurality of information transmission channels, each of which performing information transmission by utilizing at least two of a plurality of five senses, the multisensory interface control apparatus comprising:

a user status estimation portion that estimates a user status indicating an emotional status of a driver who is driving a vehicle;

a weighting setting portion that sets weightings, each of which indicates a degree of importance of a corresponding information transmission channel in accordance with the user status estimated by the user status estimation portion, wherein a degree of importance of at least one information transmission channel using an inhibitively affected sense is lowered, a degree of importance of at least one information transmission channel other than the information transmission channel using the inhibitively affected sense is raised; and

a content adjustment portion that changes allocations of a plurality of sets of partial content that configure content transmitted via the multisensory interface, the allocations being changed in accordance with the weightings,

wherein;

the multisensory interface control apparatus is implemented in a drive assistance system for the driver;

the at least one information transmission channel using an inhibitively affected sense performs the information transmission via a first sense of the five senses; and

the at least one information transmission channel other than the information transmission channel using the inhibitively affected sense performs the information transmission via a second sense of the five senses.