Providing a user interface experience based on inferred vehicle state

• US 9,596,643 B2
• Filed: 07/15/2014
• Issued: 03/14/2017
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Paek, Timothy S., Bahl, Paramvir, Addagatla, Sreenivas
• Primary Examiner(s): TAYLOR, NATHAN SCOTT
• Application Number:
  US14/332,235
• Publication Number:
  US 20140329487A1
• Time in Patent Office: 973 Days
• Field of Search: 455/41.2, 4554041-4042, 455/418, 340/441, 340575-576
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/1472 Multi-touch gesture
• CPC Class Codes: B60K 2360/148 Instrument input by voice
• CPC Class Codes: B60K 2360/151 Instrument output devices for configurable output
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/182 Distributing information between displays
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/566 Mobile devices displaying vehicle information
• CPC Class Codes: B60K 2360/573 Mobile devices controlling vehicle functions
• CPC Class Codes: B60K 2360/586 Wired data transfers
• CPC Class Codes: B60K 2360/5894 SIM cards
• CPC Class Codes: B60K 2360/5899 Internet
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 2360/834 Docking arrangements
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/26 using acoustic output
• CPC Class Codes: B60K 35/28 : characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 : Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/50 : Instruments characterised by their means of attachment to or integration in the vehicle B60K35/231 takes precedence
• CPC Class Codes: B60K 35/80 : Arrangements for controlling instruments
• CPC Class Codes: B60K 35/85 : Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: G06F 1/1632 : External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1694 : the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/0481 : based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/90 : Services for handling of emergency or hazardous situations, e.g. earthquake and tsunami warning systems [ETWS]
• CPC Class Codes: H04W 48/04 : based on user or terminal location or mobility data, e.g. moving direction, speed

Abstract

A mobile device is described herein that provides a user interface experience to a user who is operating the mobile device within a vehicle. The mobile device provides the user interface experience using mode functionality. The mode functionality operates by receiving inference-input information from one or more input sources. At least one input source corresponds to at least one movement-sensing device, provided by the mobile device, that determines movement of the mobile device. The mode functionality then infers a state of the vehicle based on the inference-input information and presents a user interface experience that is appropriate for the vehicle state. In one scenario, the mode functionality can also infer that the vehicle is in a distress condition. In response, the mode functionality can solicit assistance for the user.

Claims

1. A computer implemented method, comprising:

determining that a mobile device is in a vehicle based upon at least one of mobile device-provided input information provided by the mobile device or vehicle-provided input information provided by the vehicle;

automatically switching the mobile device from a handheld mode to a vehicle mode based upon the at least one of the mobile device-provided input information or the vehicle-provided input information;

evaluating the at least one of the mobile device-provided input information or the vehicle-provided input information to select a current vehicle state of the vehicle having a corresponding level of driving complexity, the current vehicle state being selected from a plurality of predetermined vehicle states associated with different levels of driving complexity; and

,

generating a user interface experience for a user who is operating the vehicle based at least in part on the corresponding level of driving complexity of the current vehicle state, wherein the user interface experience imposes attention-related demands on the user and wherein the user interface experience replaces at least some input commands with other input commands that are associated with the corresponding level of driving complexity of the current vehicle state.

2. The method of claim 1, further comprising:

obtaining the mobile device-provided input information from one or more of;

an accelerometer device of the mobile device;

a gyro device of the mobile device;

a vision device of the mobile device;

a magnetometer device of the mobile device;

or a position-determining device of the mobile device,

wherein the mobile device-provided input information is evaluated to select the current vehicle state.

3. The method of claim 2, wherein the mobile device-provided input information is obtained from the position-determining device and the position-determining device comprises a GPS device.

4. The method of claim 1, further comprising:

obtaining the vehicle-provided input information, wherein the vehicle- provided input information indicates whether passengers are present in the vehicle and the vehicle-provided input information is evaluated to select the current vehicle state.

5. The method of claim 1, further comprising:

obtaining and evaluating both the vehicle-provided input information and the mobile device-provided input information to select the current vehicle state.

6. The method of claim 1, wherein the evaluating comprises:

compiling a movement signature from the at least one of the mobile device-provided input information or the vehicle-provided input information.

7. The method of claim 6, wherein the evaluating comprises comparing the movement signature to a plurality of different movement signatures associated with different classes of movement to determine the current vehicle state.

8. The method of claim 7, wherein the different classes of movement correspond to different variations in speed of the vehicle and the current vehicle state characterizes an extent to which the speed of the vehicle varies.

9. The method of claim 7, wherein the different classes of movement correspond to different amounts of braking of the vehicle and the current vehicle state characterizes an extent to which the brakes of the vehicle are being applied.

10. The method of claim 1, wherein, in the vehicle mode, the generating comprises sending the generated user interface experience to the vehicle for presentation by the vehicle.

11. The method of claim 1, wherein the at least some input commands that are replaced are touch input commands and the other input commands are voice-based or gesture-based commands.

12. The method of claim 1, wherein the generating comprises presenting the user interface experience on the mobile device.

13. A mobile device, comprising:

a display;

a processor; and

a computer-readable medium storing computer readable instructions which, when executed by the processor, cause the processor to;

allow a user to specify that the mobile device operate in either a handheld mode or a vehicle mode;

in the vehicle mode;

obtain sensor information from one or more sensors, and

use the sensor information to predict a route that a vehicle is likely to take to reach a specified or predicted destination;

based at least on the predicted route, determine a predicted future vehicle state of the vehicle, the predicted future vehicle state having an attention profile that characterizes a level of attention and a type of attention which is appropriate for the user to maintain while operating the vehicle; and

configure a user interface of the mobile device to comply with the attention profile for the predicted future vehicle state.

14. The mobile device of claim 13, wherein:

an application with which the mobile device interacts includes a plurality of resources, the plurality of resources including at least a first group of resources adapted for application in a first vehicle state, and a second group of resources adapted for application in a second vehicle state, and

the computer readable instructions, when executed by the processor, cause the processor to use either the first group of resources or the second group of resources based at least on the predicted future vehicle state.

15. The mobile device of claim 13, further comprising the one or more sensors.

16. The mobile device of claim 15, wherein:

the one or more sensors indicate a location of the mobile device, and

the computer readable instructions, when executed by the processor, cause the processor to consult prior routes traveled by the mobile device to predict the route.

17. The mobile device of claim 13, wherein the computer readable instructions, when executed by the processor, cause the processor to:

determine predicted driving conditions associated with the predicted route; and

determine the predicted future vehicle state based at least on the predicted driving conditions.

18. The mobile device of claim 13, wherein:

the one or more sensors are provided by the vehicle, and the computer readable instructions, when executed by the processor,

cause the processor to obtain the sensor information from the vehicle.

19. A method comprising:

operating a mobile device having a handheld mode and a vehicle mode in the vehicle mode;

while in the vehicle mode;

obtaining sensor information from one or more sensors, and

using the sensor information to predict a route that a vehicle is likely to take to reach a specified or predicted destination;

based at least on the predicted route, determining a predicted future vehicle state of the vehicle, the predicted future vehicle state having an attention profile that characterizes a level of attention and a type of attention which is appropriate for a user to maintain while operating the vehicle; and

configuring a user interface of the mobile device in accordance with the attention profile for the predicted future vehicle state.

20. The method of claim 19, performed entirely by the mobile device.