Unified input and invoke handling

• US 10,078,382 B2
• Filed: 09/02/2015
• Issued: 09/18/2018
• Est. Priority Date: 09/04/2014
• Status: Active Grant

• Current Assignee: Home Box Office Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: Home Box Office Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Furtwangler, Nathan J. E., Furtwangler, Brandon C., Parker, J. Jordan C.
• Primary Examiner(s): Duong, Hien Luongvan
• Application Number:
  US14/843,877
• Publication Number:
  US 20160070424A1
• Time in Patent Office: 1,112 Days
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

The described technology is directed towards normalizing input received from various types of device types for providing the input to user interface (UI) elements in a unified way that reflects the invoke intent of the user. Input from different device types is classified as button input, pointer input or command input, and routed to a corresponding button provider, pointer provider or command provider. Each provider includes logic for communicating input data (e.g., events) to a UI element or elements. Based upon the input events, a UI element can execute a corresponding invoke handler to take an invoke action for that particular event.

Claims

1. A method, comprising:

generating, by a system comprising a processor, mappings of input devices to providers, wherein at least two input devices of the input devices are mapped to a single provider of the providers, and at least two providers of the providers are mapped to different input devices of the input devices, wherein a provider of the providers is configured to normalize inputs from the input devices mapped to the provider;

processing, by the system, input from an input device of the input devices, including selecting a selected provider of the providers corresponding to the input device based on the mappings, and routing the input to the selected provider;

normalizing, by the system via the selected provider, the input to generate input-related data;

communicating, by the system, the input-related data from the selected provider to one or more user interface (UI) elements, including providing an input event to a selected UI element; and

performing, by the system, an invoke action at the selected UI element based upon the input event.

2. The method of claim 1, wherein the selecting the selected provider comprises selecting a button provider, and wherein the providing the input event to the selected UI element comprises communicating a button input event and data associated with the input event including a button code to the selected UI element based upon the selected UI element having focus.

3. The method of claim 1, wherein the selecting the selected provider comprises selecting a pointer provider, and wherein the providing the input event to the selected UI element comprises communicating a pointer input event and data associated with the pointer input event to the selected UI element based upon a rendered representation of the UI element at a location on a display screen that is below a current pointer location.

4. The method of claim 3, wherein the communicating the data comprises communicating data associated with a pointer input move input event that indicates that the selected UI element is being hovered over by the pointer.

5. The method of claim 3, wherein the communicating the data comprises communicating data associated with a pointer input move input event that indicates that the selected UI element is no longer being hovered over by the pointer.

6. The method of claim 3, wherein the communicating the data comprises communicating information indicating that a pointer is in an up state with respect to the UI element.

7. The method of claim 3, wherein the communicating the data comprises communicating information indicating that a pointer is in a down state with respect to the UI element.

8. The method of claim 3, wherein the communicating the data comprises communicating information indicating that a pointer is in an engaged state or a disengaged state.

9. The method of claim 1, wherein the selecting the selected provider comprises selecting a command provider, and wherein the providing the input event to the selected UI element comprises filtering to obtain a set of one or more candidate UI elements based upon at least one of initialization or command information, and if the set contains one candidate, selecting the one candidate as the selected UI element for providing the input event including command data to the selected UI element, and if the set contains more than one candidate, disambiguating to select one candidate from the set as the selected UI element for providing the input event including command data to the selected UI element.

10. The method of claim 9, wherein the input comprises a command initiation request without an accompanying command, and further comprising entering, by the system, a targeting phase, including visualizing one or more UI elements to which a subsequent received command is able to be targeted as the selected UI element.

11. A system comprising:

a processor; and

a memory communicatively coupled to the processor, the memory having stored therein computer-executable instructions, comprising;

a user interface (UI) element set comprising one or more UI elements, at least one UI element coupled to an invoke handler that takes an invoke action when executed;

an input namespace component configured to;

generate mappings of input devices to providers, wherein at least two input devices of the input devices are mapped to a single provider of the providers, and at least two providers of the providers are mapped to different input devices of the input devices, wherein a provider of the providers is configured to normalize inputs from the input devices mapped to the provider, and

route input from an input device of the input devices to a selected provider of the providers based upon the mappings; and

the selected provider configured to;

normalize the input to generate input-related data, and

communicate the input-related data to a selected UI element of the UI element set, the selected UI element configured to execute the invoke handler to take the invoke action based upon the input-related data.

12. The system of claim 11, wherein the providers comprise a button provider, a pointer provider, and a command provider.

13. The system of claim 11, wherein the input devices comprise at least one of a physical keyboard input device, a virtual keyboard input device, a gamepad controller input device or a media remote control input device that are mapped to a button provider of the providers, and wherein the input-related data communicated is by the button provider to the selected UI element and comprises a button event and button code representative of which button was pressed.

14. The system of claim 11, wherein the input devices comprise at least one of a pointer input device, touch input device, stylus input device, or skeletal input device that are mapped to a pointer provider of the providers, and wherein the pointer provider communicates the input-related data to the selected UI element based upon a current pointer location relative to a location of a rendered representation of the selected UI element.

15. The system of claim 11, wherein the input devices comprise at least one of a speech input device, physical keyboard input device, or virtual keyboard input device that are mapped to a command provider of the providers, and wherein the command provider communicates the input-related data to the selected UI element based upon at least one of an initiation process or a command.

16. A non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations comprising:

generating mappings of input devices to providers, wherein at least two input devices of the input devices are mapped to a single provider of the providers, and at least two providers of the providers are mapped to different input devices of the input devices, wherein a provider of the providers is configured to normalize inputs from the input devices mapped to the provider;

receiving input from an input device of the input devices;

selecting a selected provider of the providers corresponding to the input device based on the mappings;

normalizing, by the selected provider, the input to generate input-related data;

communicating the input-related data from the selected provider to one or more user interface (UI) elements, including providing an input event to a selected UI element.

17. The non-transitory computer-readable medium of claim 16, wherein the input event is a button event, and the operations further comprise performing an invoke action associated with the button event at the selected UI element.

18. The non-transitory computer-readable medium of claim 16, wherein the input event is a command input event, and the operations further comprise sending command input data to the selected UI element, including filtering UI elements to obtain a candidate set of one or more candidate UI elements based upon initialization or command information, or both initialization and command information, and selecting a candidate UI element as the selected UI element from the candidate set.

19. The non-transitory computer-readable medium of claim 18, wherein selecting the candidate UI element as the selected UI element comprises targeting the candidate set, receiving user input related to a command, and a) if the command applies to only one targeted candidate, selecting the one targeted candidate as the selected candidate, or b) if the command applies more than one targeted candidate, disambiguating between the targeted candidates, including receiving user selection information to select one of the targeted candidates as the selected UI element.

20. The non-transitory computer-readable medium of claim 18, wherein the operations further comprise receiving the command input data at the selected UI element, and performing an invoke action associated with the command input data.