Vehicle-based Multi-modal Interface

• US 20160150020A1
• Filed: 11/20/2014
• Published: 05/26/2016
• Est. Priority Date: 11/20/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Glass, Mike, Bapat, Vikram Padmakar, Schulz, Kristoffer S., Farmer, Jason Ryan, Murillo, Oscar E.
• Granted Patent: US 10,116,748 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/11 Instrument graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2360/111 for controlling multiple devices
• CPC Class Codes: B60K 2360/1442 Emulation of input devices
• CPC Class Codes: B60K 2360/151 Instrument output devices for configurable output
• CPC Class Codes: B60K 2360/164 Infotainment
• CPC Class Codes: B60K 2360/573 Mobile devices controlling vehicle functions
• CPC Class Codes: B60K 2360/589 Wireless data transfers
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60K 35/80 Arrangements for controlling instruments
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04M 1/6083 by interfacing with the vehicle audio system
• CPC Class Codes: H04M 1/6091 : including a wireless interface
• CPC Class Codes: H04M 1/72412 : using two-way short-range wireless interfaces

Abstract

Various embodiments enable mobile devices, such as phones and the like, to integrate with an in-vehicle information/entertainment system to enable the user to control the in-vehicle information/entertainment system by way of their mobile phone. Users can leverage the functionality of their mobile phone to promote an in-vehicle experience which can be contextually tailored to the user'"'"'s or the vehicle'"'"'s context. Yet other embodiments can purvey an in-vehicle experience through a cloud based service.

Claims

1. A computer-implemented method comprising:

establishing a connection between a mobile device and an in-vehicle information/entertainment system;

receiving a communication from the mobile device, the communication being associated with an input of a first modality type that was received by the mobile device, the input being associated with accomplishing a task;

performing one or more actions associated with the task;

receiving input of a second modality type that is different from the first modality type, the input of the second modality type being associated with performing the task;

performing one or more actions associated with the input of the second modality type for performing the task;

wherein the in-vehicle information/entertainment system is configured to receive input of at least a third modality type that is different from the first and second modality types to perform one or more actions associated with the task or one or more actions associated with a different task.

2. The computer-implemented method of claim 1, wherein the input of the first modality type comprises voice input.

3. The computer-implemented method of claim 2, wherein the input of the second modality type comprises touch input received by way of a console of the in-vehicle information/entertainment system.

4. The computer-implemented method of claim 2, wherein the input of the second modality type comprises a jog dial input received by way of a jog dial associated with the console of the in-vehicle information/entertainment system.

5. The computer-implemented method of claim 1, wherein at least some actions can be performed using the first input modality or one or more of the second input modality or the third modality.

6. The computer-implemented method of claim 1, wherein at least some actions can be performed using the first input modality or one or more of the second input modality or the third modality, wherein the first input modality comprises voice input.

7. The computer-implemented method of claim 1, wherein at least one of the first, second, or third modality types enables navigation of objects that are rendered on a console of the in-vehicle information/entertainment system.

8. The computer-implemented method of claim 1, wherein at least one of the first, second, or third modality types enables navigation of objects that are rendered on a console of the in-vehicle information/entertainment system, and wherein the objects comprise tiles.

9. The computer-implemented method of claim 1, wherein the input of the first modality type comprises voice input, and the input of the second modality type is configured to disambiguate an ambiguity associated with the input of the first modality type.

10. The computer-implemented method of claim 1, wherein the in-vehicle information/entertainment system is configured to enable actions associated with one particular task can be interrupted when a user'"'"'s context indicates that performable actions associated with a different task are available for performance.

11. One or more computer readable storage media storing computer-readable instructions which, when executed, perform operations comprising:

establishing a connection between a mobile device and an in-vehicle information/entertainment system;

receiving a communication from the mobile device, the communication being associated with an input of a first modality type that was received by the mobile device, the input being associated with accomplishing a task;

performing one or more actions associated with the task;

receiving input of a second modality type that is different from the first modality type, the input of the second modality type being associated with performing the task;

performing one or more actions associated with the input of the second modality type for performing the task;

wherein the in-vehicle information/entertainment system is configured to receive input of at least a third modality type that is different from the first and second modality types to perform one or more actions associated with the task or one or more actions associated with a different task.

12. The one or more computer readable storage media of claim 11, wherein the input of the first modality type comprises voice input.

13. The one or more computer readable storage media of claim 11, wherein the input of the first modality type comprises voice input, and wherein the input of the second modality type comprises touch input received by way of a console of the in-vehicle information/entertainment system.

14. The one or more computer readable storage media of claim 11, wherein the input of the first modality type comprises voice input, and wherein the input of the second modality type comprises a jog dial input received by way of a jog dial associated with the console of the in-vehicle information/entertainment system.

15. The one or more computer readable storage media of claim 11, wherein at least some actions can be performed using the first input modality or one or more of the second input modality or the third modality.

16. The one or more computer readable storage media of claim 11, wherein at least one of the first, second, or third modality types enables navigation of objects that are rendered on a console of the in-vehicle information/entertainment system.

17. An automobile comprising:

an in-vehicle information/entertainment system configured to be operably connected to a mobile device and, when connected to the mobile device, perform operations comprising;

receiving a communication from the mobile device, the communication being associated with an input of a first modality type that was received by the mobile device, the input being associated with accomplishing a task;

performing one or more actions associated with the task;

receiving input of a second modality type that is different from the first modality type, the input of the second modality type being associated with performing the task;

performing one or more actions associated with the input of the second modality type for performing the task;

wherein the in-vehicle information/entertainment system is configured to receive input of at least a third modality type that is different from the first and second modality types to perform one or more actions associated with the task or one or more actions associated with a different task.

18. The vehicle of claim 17, wherein the input of the first modality type comprises voice input.

19. The vehicle of claim 17, wherein the input of the first modality type comprises voice input, and wherein the input of the second modality type comprises touch input received by way of a console of the in-vehicle information/entertainment system.

20. The vehicle of claim 17, wherein the input of the first modality type comprises voice input, and wherein the input of the second modality type comprises a jog dial input received by way of a jog dial associated with the console of the in-vehicle information/entertainment system.