Providing a user interface experience based on inferred vehicle state

• US 8,811,938 B2
• Filed: 12/16/2011
• Issued: 08/19/2014
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Paek, Timothy S., Bahl, Paramvir, Addagatla, Sreenivas
• Primary Examiner(s): TAYLOR, NATHAN SCOTT
• Application Number:
  US13/327,786
• Publication Number:
  US 20130157607A1
• Time in Patent Office: 977 Days
• Field of Search: 455/41.2, 4554041-4042, 455/418, 340/441, 340575-576
• US Class Current: 455/404.2
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/1472 Multi-touch gesture
• CPC Class Codes: B60K 2360/148 Instrument input by voice
• CPC Class Codes: B60K 2360/151 Instrument output devices for configurable output
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/182 Distributing information between displays
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/566 Mobile devices displaying vehicle information
• CPC Class Codes: B60K 2360/573 Mobile devices controlling vehicle functions
• CPC Class Codes: B60K 2360/586 Wired data transfers
• CPC Class Codes: B60K 2360/5894 SIM cards
• CPC Class Codes: B60K 2360/5899 Internet
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 2360/834 Docking arrangements
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/26 using acoustic output
• CPC Class Codes: B60K 35/28 : characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 : Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/50 : Instruments characterised by their means of attachment to or integration in the vehicle B60K35/231 takes precedence
• CPC Class Codes: B60K 35/80 : Arrangements for controlling instruments
• CPC Class Codes: B60K 35/85 : Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: G06F 1/1632 : External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1694 : the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/0481 : based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/90 : Services for handling of emergency or hazardous situations, e.g. earthquake and tsunami warning systems [ETWS]
• CPC Class Codes: H04W 48/04 : based on user or terminal location or mobility data, e.g. moving direction, speed

Abstract

A mobile device is described herein that provides a user interface experience to a user who is operating the mobile device within a vehicle. The mobile device provides the user interface experience using mode functionality. The mode functionality operates by receiving inference-input information from one or more input sources. At least one input source corresponds to at least one movement-sensing device, provided by the mobile device, that determines movement of the mobile device. The mode functionality then infers a state of the vehicle based on the inference-input information and presents a user interface experience that is appropriate for the vehicle state. In one scenario, the mode functionality can also infer that the vehicle is in a distress condition. In response, the mode functionality can solicit assistance for the user.

Claims

1. A processor implemented method, comprising:

receiving inference-input information from one or more input sources when a mobile device is in a vehicle, wherein an individual input source is a camera positioned to capture a space in front of a display of the mobile device;

inferring a vehicle state of the vehicle based on the inference-input information;

generating an attention profile from the inference-input information;

predicting future driving conditions that may be encountered by the vehicle before the future driving conditions are actually encountered; and

,

presenting a user interface experience for a user operating the vehicle, the user interface experience imposing attention-related demands on the user, where the attention-related demands are selected based on the vehicle state, the attention profile, and the predicted future driving conditions, wherein the user interface experience disallows touch control of the mobile device when the user is operating the vehicle and monitors input from the camera to detect user gestures for controlling the mobile device.

2. The method of claim 1, wherein another individual input source comprises at least one movement-sensing device comprising one or more of:

an accelerometer device;

a gyro device;

a vision device;

a magnetometer device; and

a position-determining device.

3. The method of claim 2, wherein the position-determining device comprises a GPS device.

4. The method of claim 1, wherein another individual input source comprises at least one sensor device associated with the vehicle, the sensor device being external to the mobile device.

5. The method of claim 1, wherein another individual input source comprises at least one sensor device that is associated with a mount which holds the mobile device within the vehicle.

6. The method of claim 1, wherein said inferring comprises determining whether the vehicle is moving or not moving, based on the inference-input information.

7. The method of claim 1, wherein said inferring comprises classifying a manner in which the vehicle is moving, based on the inference-input information.

8. The method of claim 1, wherein said inferring comprises determining a location of the vehicle based on the inference-input information, and assessing the attention-related demands associated with that location.

9. The method of claim 1, wherein said inferring comprises determining that the vehicle is in a distress condition based on the inference-input information, wherein the inference-input information contains evidence that the mobile device is located in the vehicle, and other evidence that at least one of:

the vehicle has abruptly decelerated or accelerated; and

the mobile device has become dislodged from a mount that holds the mobile device within the vehicle at a time of abrupt acceleration or deceleration.

10. The method of claim 1, wherein said presenting comprises at least partially enabling or restricting an output mode that delivers an output presentation to the user.

11. The method of claim 1, wherein said presenting comprises changing a type of content that is provided by an output presentation that is delivered to the user.

12. The method of claim 1, wherein said presenting comprises changing at least one property of an output presentation that is delivered to the user.

13. The method of claim 1, wherein said presenting comprises controlling a vehicle system that is associated with the vehicle, where the vehicle system is external to the mobile device.

14. The method of claim 1, wherein said presenting comprises attempting to obtain assistance for the user in response to a determination that the vehicle is in a distress condition.

15. A mobile device, comprising:

a display and a processor configured to control the display;

a camera configured to capture a space in front of the display;

an interface module configured to receive inference-input information from one or more input sources, at least one input source configured to provide an indication whether the mobile device is in a vehicle being operated by a user of the mobile device;

a state detection module configured to infer a vehicle state of the vehicle based on the inference-input information, the vehicle state having an attention profile that characterizes a level of attention and a type of attention which is appropriate for the user to maintain while operating the vehicle;

an experience presentation module configured to switch a user interface on the display from a handheld mode to a vehicle mode when the mobile device is in the vehicle, the experience presentation module further configured to disallow touch control of the mobile device according to the vehicle state and monitor input from the camera to detect user gestures for controlling the mobile device;

the interface module, the state detection module, and the experience presentation module being implemented by the processor.

16. The mobile device functionality of claim 15, wherein an application with which the mobile device interacts includes a plurality of resources, the plurality of resources including at least a first group of resources adapted for application in a first vehicle state, and a second group of resources adapted for application in a second vehicle state,

wherein the experience presentation module is configured to use either the first group of resources or the second group of resources based on the vehicle state.

17. A computer readable storage device for storing computer readable instructions, the computer readable instructions executable by one or more processing devices, the computer readable instructions comprising:

receiving inference-input information from one or more input sources, at least one input source configured to provide an indication whether a mobile device is in a vehicle being operated by a user of the mobile device;

determining an attention profile that characterizes a level of attention and a type of attention which is appropriate for the user to maintain while operating the vehicle;

switching a user interface on a display of the mobile device from a handheld mode to a vehicle mode when the mobile device is in the vehicle being operated by the user, wherein the vehicle mode disallows touch control of the mobile device, and wherein the user interface in the vehicle mode reflects the attention profile; and

,

in an instance when the user interface is in the vehicle mode, monitoring input from a space in front of the display of the mobile device to detect user gestures for controlling the mobile device.

18. The computer readable storage device of claim 17, wherein the computer readable instructions cause the one or more processing devices to perform further acts comprising controlling the mobile device based upon the monitoring.