Response to user input based on declarative mappings

• US 9,152,395 B2
• Filed: 12/13/2010
• Issued: 10/06/2015
• Est. Priority Date: 12/13/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Luengen, Ross, Zou, Song
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US12/966,842
• Publication Number:
  US 20120147031A1
• Time in Patent Office: 1,758 Days
• Field of Search: 345/619, 345/647, 345/649, 345/654, 345/660, 345/665, 345668-672, 345/684, 345/173, 715/863, 715/773, 709/246
• US Class Current: 1/1
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/37 Details of the operation on graphic patterns G09G5/38 takes precedence

Abstract

Systems and methods are provided that enable direct response to user input, e.g., in accordance with storyboards. For instance, applications can predefine storyboards which describe output in terms of received input. The predefined storyboards can be used externally and independently from the applications to process user input on behalf of the applications.

Claims

1. A method executed by at least one processor of a computing device, comprising:

executing a direct manipulation component process and an application process on the computing device, including;

receiving, by the direct manipulation component process executing on the computing device, a declarative mapping from the application process that is also executing on the computing device, wherein the declarative mapping correlates a predefined change to be made in a display output of the computing device to a predefined user input;

receiving, by the direct manipulation component process, the predefined user input; and

in response to receiving the predefined user input, making, by the direct manipulation component process, the predefined change to the display output, the predefined change being made by the direct manipulation component process on behalf of the application process without intervention of the application process, wherein;

the predefined user input is a touch manipulation;

receiving the predefined user input includes receiving the touch manipulation via a touch interface;

executing the direct manipulation component process and the application process on the computing device also includes;

determining whether the received touch manipulation is associated with the declarative mapping; and

,

executing the direct manipulation component process and the application process on the computing device also includes;

receiving, by the direct manipulation component process, additional user input, the additional user input not being the predefined user input; and

forwarding the user input to the application process in response to the user input not being the predefined user input.

2. The method of claim 1, wherein executing the direct manipulation component process and the application process on the computing device also includes:

transmitting the display output to a graphics subsystem configured to present the display output.

3. A method executed by at least one processor of a computing device, comprising:

executing a direct manipulation component process and an application process on the computing device, including;

receiving, by the direct manipulation component process executing on the computing device, a declarative mapping from the application process that is also executing on the computing device, wherein the declarative mapping correlates a predefined change to be made in a display output of the computing device to a predefined user input;

receiving, by the direct manipulation component process, the predefined user input; and

in response to receiving the predefined user input, making, by the direct manipulation component process, the predefined change to the display output, the predefined change being made by the direct manipulation component process on behalf of the application process without intervention of the application process,

wherein executing the direct manipulation component process and the application process on the computing device also includes;

receiving, by the direct manipulation component process, at least one other declarative mapping, each of the at least one other declarative mappings correlating an associated predefined change to be made in the display output of the computing device to an associated predefined user input, the declarative mapping and the at least one other declarative mappings comprising a set of declarative mappings, wherein the set of declarative mappings is associated with a type of touch manipulation;

receiving, by the direct manipulation component process, additional user input; and

in response to receiving the additional user input;

identifying an input manipulation represented by the additional user input; and

selecting, from the set of declarative mappings, at least one declarative mapping that is associated with the identified input manipulation.

4. The method of claim 3, wherein executing the direct manipulation component process and the application process on the computing device also includes:

determining a set of values associated with the identified input manipulation based on the additional user input.

5. The method of claim 4, wherein executing the direct manipulation component process and the application process on the computing device also includes:

determining, based on the at least one declarative mapping, a set of output parameters for the set of values associated with the identified input manipulation.

6. The method of claim 5, wherein executing the direct manipulation component process and the application process on the computing device also includes:

generating an output transform based on the set of output parameters, including;

generating a manipulation on at least one of a region or an object of the application process.

7. A computer storage memory having instructions stored therein, the stored instructions being executable by a processor of a computing device to implement a direct manipulation subsystem on the computing device, and being organized into at least:

an input component on the computing device that is configured to receive;

at least one declarative mapping from at least one application executing on the computing device; and

user input corresponding to at least one touch manipulation of a touch interface;

an output transform component on the computing device that is configured to create at least one output transform, on behalf of the at least one application executing on the computing device and without intervention of the at least one application executing on the computing device, based upon the at least one declarative mapping, wherein the at least one output transform represents at least one response to the user input;

a manipulation analysis component on the computing device that is configured to identify a set of values associated with the user input; and

a parameter determination component on the computing device that is configured to derive at least one output parameter of the at least one output transform, and, based on the at least one declarative mapping, obtain a set of output parameters corresponding to the set of values associated with the user input.

8. The computer storage memory of claim 7, wherein the instructions are further organized into:

an input evaluation component on the computing device that is configured to determine that the received user input corresponds to a touch manipulation for which a declarative mapping of the at least one declarative mapping has been received from the at least one application executing on the computing device.

9. A computer storage memory having instructions stored therein, the stored instructions being executable by a processor of a computing device to implement a direct manipulation subsystem on the computing device, and being organized into at least:

an input component on the computing device that is configured to receive;

at least one declarative mapping from at least one application executing on the computing device; and

user input corresponding to at least one touch manipulation of a touch interface; and

an output transform component on the computing device that is configured to create at least one output transform, on behalf of the at least one application executing on the computing device and without intervention of the at least one application executing on the computing device, based upon the at least one declarative mapping, wherein the at least one output transform represents at least one response to the user input, wherein

the at least one declarative mapping defines a plurality of functions that map a set of input parameters of the user input to a set of output parameters specifying an output for a response to the user input.

10. The computer storage memory of claim 9, wherein the output transform component on the computing device is further configured to generate the at least one output transform independently of the application executing on the computing device to which the user input is directed.

11. A computing device, comprising:

a memory and a processor, the memory and the processor respectively configured to store and execute instructions for the computing device, the instructions comprising;

a direct manipulation component configured to execute on the computing device and to receive;

a set of declarative mappings from an application executing on the computing device; and

touch input for the application executing on the computing device from an operating system of the executing on the computing device, wherein;

the direct manipulation component derives output that responds to the touch input, independently and on behalf of the application executing on the computing device, in accordance with the set of declarative mappings, wherein the set of declarative mappings is associated with a type of touch manipulation.

12. The computing device of claim 11, wherein the direct manipulation component is configured to provide the output to the operating system executing on the computing device to instruct a graphics processing unit of the computing device to apply an output transform specified in the output.

13. The computing device of claim 11, wherein the set of declarative mappings includes respective declarative mappings for a set of characteristics of the touch input.

14. A computing device, comprising:

a memory and a processor, the memory and the processor respectively configured to store and execute instructions for the computing device, the instructions comprising;

a direct manipulation component configured to execute on the computing device and to receive;

a set of declarative mappings from an application executing on the computing device; and

touch input for the application executing on the computing device from an operating system of the executing on the computing device, wherein;

the direct manipulation component derives output that responds to the touch input, independently and on behalf of the application executing on the computing device, in accordance with the set of declarative mappings; and

the set of declarative mappings is associated with a region of a display to which the application executing on the computing device renders application content.

15. The computing device of claim 14, wherein the direct manipulation component is configured to apply the output to the application content to the region, and wherein the region is also associated with the set of declarative mappings.