Control exposure

• US 9,250,713 B2
• Filed: 03/02/2012
• Issued: 02/02/2016
• Est. Priority Date: 12/05/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ewanchuk, Brian J., Olson, Lawrence W., Hillberg, Michael J., Kaza, Akhilesh, Gup, Jeremy B., Lamb, Austin M.
• Primary Examiner(s): Mudrick, Timothy A
• Application Number:
  US13/410,888
• Publication Number:
  US 20130145380A1
• Time in Patent Office: 1,432 Days
• Field of Search: 719/318
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Control exposure techniques are described. In one or more implementations, a determination is made by a computing device as to which of a plurality of controls correspond to one or more inputs detected using one or more magnetometers, cameras, or microphones. A result of the determination is exposed by the computing device to one or more applications that are executed by the computing device.

Claims

1. A method comprising:

receiving inputs detected using one or more magnetometers, cameras, or microphones;

determining one or more applications of a plurality of applications that are a target of the inputs, the determining comprising performing a handshake with the applications to determine which of the applications are available to receive events or subscribed to the events;

converting, by a computing device, the inputs into graphical user interface controls provided by a natural user interface input module for use in a graphical user interface of the one or more applications; and

exposing the graphical user interface controls to the one or more applications as events via an application programming interface.

2. A method as described in claim 1, wherein the one or more cameras are configured to perform depth sensing.

3. A method as described in claim 1, wherein the inputs are recognizable by the computing device as gestures configured to initiate an operation that corresponds to the graphical user interface controls.

4. A method as described in claim 1, wherein the exposing is performed through execution of an operating system by the computing device.

5. A method as described in claim 1, wherein the handshake is performed between an operating system that performs the determining and the exposing and the one or more applications, the handshake usable by the operating system to determine how and what to send to the one or more applications.

6. A system implemented by a computing device, the system comprising:

applications that are executable on hardware of the computing device; and

a module that is implemented at least partially using the hardware of the computing device, the module configured to;

receive one or more inputs that describe a user interaction that is detected without using touch;

determine one or more applications of the applications that are a target of the one or more inputs by performing a handshake with the applications to determine which of the applications are available to receive events or subscribed to the events;

determine which of a plurality of graphical user interface controls provided by the module for use in a graphical user interface of the determined one or more applications is a target of the one or more inputs;

convert the one or more inputs into one or more events that are consumable by the determined one or more applications; and

expose the events to the determined one or more applications, each event identifying a respective graphical user interface control that corresponds to the input and an operation to be performed by the determined one or more applications using the respective graphical user interface control.

7. A system as described in claim 6, wherein the module is implemented as part of an operating system of the computing device.

8. A system as described in claim 6, wherein the one or more events are associated with a particular control.

9. A system as described in claim 6, wherein the one or more inputs are detected without using touch by one or more cameras.

10. A system as described in claim 6, wherein the one or more inputs are detected without using touch by one or more microphones.

11. A system as described in claim 6, wherein the module is further configurable to output the one or more inputs to the one or more applications as a raw stream.

12. One or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to implement an operating system configured to expose one or more events via an application programming interface for one or more graphical user interface controls identified by the operating system from inputs received from one or more cameras of the computing device, the controls being provided by a natural user interface input module for use in a graphical user interface of one or more applications, each event identifying a respective graphical user interface control that corresponds to the input and an operation to be performed by the one or more applications using the respective graphical user interface control the one or more events exposed to one or more applications determined to be a target of the one or more inputs by performing a handshake with the applications to determine which of the applications are available to receive events or subscribed to the events.

13. One or more computer-readable storage media as described in claim 12, wherein the inputs describe one or more gestures made by a user that are detected using the one or more cameras without involving touch.

14. One or more computer-readable storage media as described in claim 12, wherein at least one said application is not configured to process the inputs to identify the one or more controls.