MULTI-MODAL USER EXPRESSIONS AND USER INTENSITY AS INTERACTIONS WITH AN APPLICATION

• US 20140104197A1
• Filed: 12/07/2012
• Published: 04/17/2014
• Est. Priority Date: 10/12/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Khosravy, Moe, Lawler, Duncan, DeMaio, Pasquale
• Granted Patent: US 10,139,937 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G01C 21/3626 Details of the output of route guidance instructions traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G09B 29/00 Maps; Plans; Charts; Diagrams, e.g. route diagram star maps G09B27/04; devices for holding or supporting maps A47B97/02; for computing purposes G06G1/14, G09B1/16; display boards G09F

Abstract

Architecture that enables single and multi-modal interaction with computing devices, as well as interpreting user intensity (or liveliness) in the gesture or gestures. In a geospatial implementation, a multi-touch interaction can involve the detection and processing of tactile pressure (touch sensitive) to facilitate general navigation between two geographical points. This is further coupled with providing detailed information that facilitates navigation and turn-by-turn directions. This includes the use of time and/or pressure to release or increase the zoom level of map tiles, the touching of the two geographical points and speaking to obtain directions between these two geographical points, and the blending of tiles to create a compelling user experience, where the map is in different levels of zoom on the same view.

Claims

1. A system, comprising:

a sensing component that senses and interprets multi-modal user expressions as interactions for controlling an application;

an analysis component that analyzes the user expressions for user intensity, wherein the application operates according to the user expressions and the user intensity; and

a microprocessor configured to execute computer-executable instructions associated with at least one of the sensing component or the analysis component.

2. The system of claim 1, wherein one of the user expressions is to touch a touch sensitive display, using one or more touches, two geographical points on a map application for directions between an origin and a destination.

3. The system of claim 2, wherein the map application blurs map tiles in a path between the origin and the destination while retaining a detailed focus of the origin and the destination.

4. The system of claim 1, wherein the sensing component senses the user intensity as defined by one or more of the user expressions and the analysis component analyzes the intensity to cause a zoom-in operation or a zoom-out operation of the application.

5. The system of claim 1, wherein the user intensity is defined according to dwell time of a touch on a touch sensitive display and touch pressure of the touch.

6. The system of claim 1, wherein the application renders different levels of view in a single overall view.

7. The system of claim 1, wherein the user expression translates to pressure and dwell of the pressure applied to a touch display surface such that a different pressure results in a correspondingly different application operation.

8. The system of claim 1, wherein the multi-modal user expressions are disambiguated using speech input.

9. A method, comprising acts of:

sensing and interpreting multi-modal user expressions as interactions for controlling a geospatial application; and

interacting with the geospatial application based on the user expressions to cause the geospatial application to render different levels of zoom in a single overall view; and

configuring a processor to execute instructions related to at least one of the acts of sensing or interacting.

10. The method of claim 9, further comprising an act of analyzing the user expressions to detect user intensity, and interacting with the geospatial application based on the user expressions and the user intensity to cause the geospatial application to render the different levels of zoom in the single overall view.

11. The method of claim 9, further comprising blurring map tiles in a path between a geographical origin and a geographical destination while retaining a detailed focus the origin and the destination

12. The method of claim 9, further comprising only fetching zoom levels associated with geographical points of interest of a map.

13. The method of claim 9, further comprising interacting with the geospatial application using multiple touches on a pressure sensitive display to identify and present directions between a geographical origin and a destination.

14. The method of claim 9, further comprising enabling different zoom levels on a geographical point of interest in response to corresponding variations in touch pressure and touch dwell time.

15. The method of claim 9, further comprising touching a geographical origin and a geographical destination on a map of the geospatial application and providing voice input to obtain directions between the origin and the destination.

16. The method of claim 9, further comprising rendering an increased level of detail for a geographical origin and a geographical destination, and a reduced level of detail everywhere else in a map of the geospatial application.

17. A method, comprising acts of:

sensing and analyzing received user gestures;

enabling interaction with an overall map view of a navigation application based on the user gestures to obtain navigation information;

detecting concurrent touch gestures via a touch display as selections of an origin and a destination in the overall map view;

rendering map view portions of both the origin and the destination at different levels of detail than other geographic portions of the overall map view; and

configuring a processor to execute instructions related to at least one of the acts of sensing, enabling, detecting, or rendering.

18. The method of claim 17, further comprising enabling the different levels of detail of map tiles in response to touch pressure variations.

19. The method of claim 17, further comprising enabling the different levels of detail of map tiles in response to touch dwell.

20. The method of claim 17, further comprising concurrently touching the origin and the destination, and speaking, to cause the navigation application to return directions from the origin to the destination.