Multimodal User Interface Design

• US 20140267035A1
• Filed: 03/03/2014
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Abandoned Application

• Current Assignee: Sirius XM Connected Vehicle Services Inc. (Liberty Media Corporation)
• Original Assignee: Sirius XM Connected Vehicle Services Inc. (Liberty Media Corporation)
• Inventors: Schalk, Thomas Barton, Hirschenberger, Frank, Faoro, Paola, He, Yan
• Application Number:
  US14/195,242
• Publication Number:
  US 20140267035A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A multimodal user interface is provided. A human machine interface in a vehicle utilizes a plurality of modalities. A cognitive model for secondary driving tasks indicates a best use of one or more particular modalities for performing each secondary driving task.

Claims

1. A method for providing a multimodal user interface, which comprises:

providing in a vehicle;

a multi-modal input module defining human-machine interface design rules; and

a human-machine interface (HMI) that utilizes a plurality of modalities; and

providing a secondary driving task cognitive model indicating when to use one or more particular modalities of the HMI for performing each secondary driving task dependent upon the HMI design rules.

2. The method according to claim 1, wherein the plurality of modalities include at least two of speech, touch, gesture, vision, sound, and haptic feedback.

3. The method according to claim 2, wherein:

speech, touch, and gesture are input modalities; and

vision, sound, and haptic feedback are output modalities.

4. The method according to claim 1, which further comprises providing the cognitive model with all or a subset of the plurality of modalities for any given secondary driving task.

5. The method according to claim 2, which further comprises activating gesture detection prior to detection of gesture input.

6. The method according to claim 5, wherein gesture input includes at least one of a specific gesture:

in a particular location relative to a touch display;

using a particular hand shape; and

with particular temporal properties.

7. The method according to claim 6, which further comprises providing a notification that gesture input is ready to be detected.

8. The method according to claim 6, which further comprises allowing gesture detection to time out when a valid gesture is not detected.

9. The method according to claim 6, which further comprises providing three-dimensional gesture input.

10. The method according to claim 9, wherein the three-dimensional gesture input is at least one of translational motion of a hand and movement of the hand itself.

11. The method according to claim 6, further comprising using gesture input to:

move images on a display of the vehicle;

zoom the image in and out;

control volume;

control fan speed; and

close applications.

12. The method according to claim 6, which further comprises using gesture input to highlight individual icons on a display of the vehicle.

13. The method according to claim 6, which further comprises waking up a vehicle system using gesture input.

14. The method according to claim 1, which further comprises using haptic feedback to interrupt the secondary driving task.

15. The method according to claim 1, which further comprises:

first, initiating the secondary driving task with a tap; and

second, prompting a user to speak information or a command.

16. The method according to claim 1, which further comprises prompting a user with audio or text to enter user input.

17. The method according to claim 16, wherein the audio is human voice or text-to-speech (TTS).

18. The method according to claim 16, wherein:

a user is prompted using text; and

a head-up display displays the text to the user.

19. The method according to claim 3, which further comprises using a conventional speech button to manage both speech and touch input modalities.

20. A method for providing a multimodal user interface, which comprises:

providing in a vehicle;

a multi-modal input module defining human-machine interface design rules; and

a human-machine interface (HMI) that utilizes a plurality of modalities; and

providing a secondary driving task cognitive model indicating when to use one or more particular modalities of the HMI for performing each secondary driving task dependent upon HMI design rules;

initiating a secondary task; and

interrupting the secondary task the secondary task to ensure safety.