Multimode user interface of a driver assistance system for inputting and presentation of information

• US 9,103,691 B2
• Filed: 11/12/2008
• Issued: 08/11/2015
• Est. Priority Date: 11/12/2007
• Status: Active Grant

• Current Assignee: Volkswagen AG
• Original Assignee: Volkswagen AG
• Inventors: Wller, Christoph, Neugebauer, Moritz, Fabian, Thomas, Wehling, Ulrike, Missall, Markus, Horna, Gnter
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US12/741,491
• Publication Number:
  US 20110022393A1
• Time in Patent Office: 2,463 Days
• Field of Search: 704/236, 704/250, 704/255, 704/270, 704/270.1, 704/275
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

In a method for multimode information input and/or adaptation of the display of a display and control device, input signals of different modality are detected which are supplied via the device to a voice recognition unit, thus initiating a desired function and/or display as an output signal, which are displayed on the device and/or output by voice output. Touch and/or gesture input signals are provided on or to the device for selection of an object intended for interaction and activation of the voice recognition unit and for the vocabulary which is provided for interaction to be restricted with the selection of the object and/or activation of the voice recognition unit as a function of the selected object, on the basis of which a voice command from the restricted vocabulary is added to the selected object as an information input and/or for adaptation of the display, via the voice recognition unit.

Claims

1. A method for multimode information input for a touchscreen device in a motor vehicle, the method comprising:

detecting, by the touchscreen device, an input motion of a user, the input motion comprising a physical gesture of the user that is (a) a contactless physical gesture with respect to a surface of the touchscreen device or (b) a touching of the surface of the touchscreen device,

automatically activating a voice recognition means associated with the touchscreen device in response to a detection of the input motion performed during the input motion and prior to completion of the input motion, and

after the activation of the voice recognition means and after completion of the input motion, determining, by the touchscreen device, a selection of one or more objects that are displayed by the touchscreen device, wherein the determination is based on an evaluation of the completed input motion,

in response to the one or more displayed objects or in response to the activation of the voice recognition means, automatically implementing a restriction of a vocabulary, which vocabulary is provided for interaction, wherein the restriction is based on the selected one or more displayed objects,

receiving, by the voice recognition means, a voice command associated with the selected one or more displayed objects and defined by the restricted vocabulary, wherein the voice recognition means is configured to receive the voice command as an information input or for adapting a display of the touchscreen device.

2. The method according to claim 1, wherein the input motion of the user comprises at least one of a long press touch, a double click touch, a two-finger touch, or a drag gesture.

3. The method according to claim 1, comprising providing the displayed one or more objects only after automatically activating the voice recognition means.

4. The method according to claim 1, wherein the voice recognition means is active only for a presettable, defined period of time after being activated, which defined period of time limits when the voice command is receivable from the user.

5. The method according to claim 1, wherein the received voice command is (a) a single word or (b) a vocable comprising multiple words.

6. The method according to claim 1, comprising superimposing on the display a text image that corresponds to the received voice command, wherein the text image that corresponds to the received voice command is superimposed on the display for a presettable, defined period of time.

7. The method according to claim 1, comprising, in response to receiving a voice command which does not match the one or more selected objects, providing at least one of a graphic display or an acoustic display indicating that the voice command does not match the one or more selected objects.

8. The method according to claim 1, wherein the touchscreen device comprises one or more proximity sensors, and wherein the detection of the input motion of the user comprises contactless detection of the physical gesture by the one or more proximity sensors.

9. A multimode information input system configured to receive multimode interactions of a user, the multimode information input system comprising:

a processor; and

logic instructions stored in non-transitory computer-readable media and executable by the processor to;

detect an input motion at a touchscreen device, the input motion comprising a physical gesture of a user that is (a) a contactless physical gesture relative to a surface of the touchscreen device or (b) a user touching of the surface of the touchscreen device,

automatically activate a voice recognition means associated with the touchscreen device in response to a detection of the input motion performed during the input motion and prior to completion of the input motion, and

after the activation of the voice recognition means and after completion of the input motion, determine a selection of one or more objects that are displayed at the touchscreen device, wherein the determination is based on an evaluation of the completed input motion,

in response to the selection of the one or more displayed objects or in response to the activation of the voice recognition means, automatically implement a restriction of a vocabulary, which vocabulary is provided for interaction, wherein the restriction is based on the selected one or more displayed objects,

receive, by the voice recognition means, a voice command associated with the selected one or more displayed objects and defined by the restricted vocabulary, wherein the voice recognition means is configured to receive the voice command as an information input or for adapting a display of the touchscreen device.

10. The multimode information system according to claim 9, wherein depending on the detected input motion from the user, the logic instructions are executable to generate control signals for a system selected from the group consisting of:

a navigation system comprising system functions selected from the group consisting of;

route planning, correction of at least one of start and destination inputs, point of interest search, point of interest selection, point of interest interaction, and communications functions, and

an entertainment system comprising system functions related to a selection and a correction of music titles.

11. The multimode information system according to claim 9, wherein the touchscreen device comprises one or more proximity sensors, and wherein detection of the input motion of the user comprises contactless detection of the physical gesture by the one or more proximity sensors.