Multi-modal user expressions and user intensity as interactions with an application

• US 10,139,937 B2
• Filed: 12/07/2012
• Issued: 11/27/2018
• Est. Priority Date: 10/12/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khosravy, Moe, Lawler, Duncan, DeMaio, Pasquale
• Primary Examiner(s): Nguyen, Chanh
• Assistant Examiner(s): Truong, Nguyen H
• Application Number:
  US13/707,726
• Publication Number:
  US 20140104197A1
• Time in Patent Office: 2,181 Days
• Field of Search: 345156-184
• CPC Class Codes: G01C 21/3626 Details of the output of route guidance instructions traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G09B 29/00 Maps; Plans; Charts; Diagrams, e.g. route diagram star maps G09B27/04; devices for holding or supporting maps A47B97/02; for computing purposes G06G1/14, G09B1/16; display boards G09F

Abstract

Architecture that enables single and multi-modal interaction with computing devices, as well as interpreting user intensity (or liveliness) in the gesture or gestures. In a geospatial implementation, a multi-touch interaction can involve the detection and processing of tactile pressure (touch sensitive) to facilitate general navigation between two geographical points. This is further coupled with providing detailed information that facilitates navigation and turn-by-turn directions. This includes the use of time and/or pressure to release or increase the zoom level of map tiles, the touching of the two geographical points and speaking to obtain directions between these two geographical points, and the blending of tiles to create a compelling user experience, where the map is in different levels of zoom on the same view.

Claims

1. A computing device, comprising:

a pressure-sensitive display;

at least one processor; and

a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, performs a method, comprising;

detecting multi-touch input provided on the pressure-sensitive display to determine a first geographical point in a mapping application and a second geographical point in the mapping application;

analyzing an intensity of the detected multi-touch input at the first geographical point and the second geographical point to determine a first zoom level and a first zoom rate for the first geographical point and a second zoom level and a second zoom rate for the second geographical point, wherein the first zoom level is associated with a first level of detail of the first geographical point and the second zoom level is associated with a second level of detail of the second geographical point;

displaying, on the pressure-sensitive display, the first geographical point at the first zoom level concurrently with the second geographic point at the second zoom level, wherein the first geographical point is zoomed in to the first zoom level using the first zoom rate and the second geographic point is zoomed in to the second zoom level using the second zoom rate; and

displaying a path between the first geographical point and the second geographical point, wherein the path is blurred as a blended zoom that changes based, at least in part, on the intensity of the detected multi-touch input at the first geographical point and the second geographical point.

2. The system of claim 1, wherein the multi-touch input associated with the first geographical point and the second geographical point causes directions from the first geographical point to the second geographic point to be output on the pressure-sensitive display.

3. The system of claim 1, wherein a first intensity is associated with a zoom-in operation and a second intensity is associated with a zoom-out operation.

4. The system of claim 1, wherein the intensity is associated with a dwell time of the detected multi-touch input.

5. The system of claim 1, further comprising instructions for analyzing speech input when determining the first zoom level and the second zoom level.

6. A method for controlling a mapping application, comprising:

receiving a first touch input at a first location on a pressure-sensitive display;

receiving a second touch input at a second location on the pressure-sensitive display;

determining an intensity of the first touch input at the first location and an intensity of the second touch input at the second location;

rendering, on the pressure-sensitive display, different levels of zoom at different zoom rates for the first location concurrently with the second location based, at least in part, on the determined intensity of the first touch input and the second touch input, wherein each of the different levels of zoom provide a different level of detail for the first location and the second location respectively; and

displaying a path between the first location point and the second location, wherein the path is blurred as a blended zoom that changes based, at least in part, on the intensity of the first touch input and the second touch input.

7. The method of claim 6, further comprising blurring map tiles in a path between the first location and the second location while retaining the different levels of zoom for the first location and the second location.

8. The method of claim 6, further comprising providing directions between the first location and the second location.

9. The method of claim 6, wherein the different levels of zoom for the first location and the different levels of zoom for the second location are associated with a touch dwell time.

10. The method of claim 6, further comprising providing voice input to obtain directions between the first location and the second location.

11. The method of claim 6, further comprising controlling the different levels of zoom based, at least in part, on one or more received gestures associated with the first touch input or the second touch input.

12. The method of claim 6, wherein the path is associated with a route.

13. A computer-readable storage medium encoding computer executable instructions that, when executed by a processor, performs a method for controlling a mapping application, comprising:

receiving two or more touch inputs on a pressure-sensitive display;

associating an origin on a map of the mapping application with a first touch input of the two or more touch inputs;

associating a destination on the map of the mapping application with a second touch input of the two or more touch inputs;

sensing an intensity of each of the two or more touch inputs;

rendering the origin at a first level of detail and at a first zoom rate based on the intensity of a first one of the two or more touch inputs;

rendering the destination at a second level of detail and at a second zoom rate based on the intensity of a second one of the two or more touch inputs; and

displaying a path between the origin and the destination, wherein the path is blurred as a blended zoom that changes based, at least in part, on the intensity of the first one of the two or more touch inputs at the origin and the second one of the two or more touch inputs at the destination.

14. The computer-readable medium of claim 13, further comprising instructions for enabling different map tiles to have different zoom levels in response to the intensity of the two or more touch inputs.

15. The computer-readable medium of claim 13, further comprising instructions for determining the different levels of detail based, at least in part, on a touch dwell.

16. The computer-readable medium of claim 13, further comprising instructions for returning directions from the origin to the destination based on voice input.

17. The computer-readable storage medium of claim 13, further comprising instructions for zooming out from the origin or the destination based on a first amount of intensity and zooming in on the origin or the destination based on a second amount of intensity.

18. The computer-readable storage medium of claim 13, further comprising instructions for zooming out from the origin or the destination based on a first gesture and zooming in on the origin or the destination based on a second gesture.

19. The computer-readable storage medium of claim 13, further comprising instructions for:

receiving a third touch input on the pressure sensitive display;

associating the third touch input with a geographic region; and

rendering the geographic region associated with the third touch input at a different level of detail than other geographic portions displayed on the mapping application.

20. The computer-readable storage medium of claim 19, wherein the different level of detail associated with the geographic region is based, at least in part, on one of an intensity of the third touch input and a gesture associated with the third touch input.