Virtual input device system

• US 10,095,328 B2
• Filed: 12/18/2017
• Issued: 10/09/2018
• Est. Priority Date: 09/04/2014
• Status: Active Grant

• Current Assignee: Home Box Office Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: Home Box Office Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Patwardhan, Makarand V., Parker, J. Jordan C., Furtwangler, Nathan J. E., Corrington, Jesse
• Primary Examiner(s): Mehmood, Jennifer
• Assistant Examiner(s): SUBEDI, DEEPROSE D
• Application Number:
  US15/844,781
• Publication Number:
  US 20180107286A1
• Time in Patent Office: 295 Days
• Field of Search: 703 23- 25, 345156-168, 719324
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

The described technology is directed towards virtual input devices that take application program-directed input from automation and/or remote devices, such as over a network, instead of via actual user input via a physical device, for example. This allows an automation framework to insert input into an application program, such as for automated testing without modifying any of the application'"'"'s other components. The virtual input devices may be object instances or the like that receive their input from function calls based upon the type of input and output events, e.g., to simulate keyboard input/output (I/O), mouse or other pointer I/O, voice, gesture, and other command I/O, and so forth.

Claims

1. A method, comprising:

mapping, by a system comprising a processor, a plurality of input device types to a provider, wherein the provider normalizes input data generated by the input device types to normalized input data that is not distinguishable based on respective input device types of the plurality of input device types;

obtaining, by the system, input from a source having an input device type of the plurality of input device types, wherein the input is directed to user interface element interaction;

normalizing, by the system via the provider, the input into the normalized input data; and

providing, by the system, the normalized input data to a user element.

2. The method of claim 1, wherein normalizing the input into the normalized input data comprises normalizing the input data into normalized button data, normalized pointer data, or normalized command data.

3. The method of claim 1, wherein obtaining the input from the source comprises, receiving the input from a virtual input device.

4. The method of claim 1, wherein obtaining the input from the source comprises, receiving the input from an actual input device.

5. The method of claim 1, further comprising, at the user element, receiving the normalized input data and in response, taking an invoke action.

6. The method of claim 1, further comprising, at the user element, receiving the normalized input data and in response, bubbling the normalized input data up to a parent user element.

7. The method of claim 1, further comprising, tracking the user element as being a currently targeted or currently focused user element, and wherein providing the normalized input data to the user element comprises routing the normalized input data to the currently targeted or currently focused user element based on the tracking the user element.

8. The method of claim 7, wherein the normalized input data comprises normalized button data, and wherein providing the normalized input data to the user element comprises routing the normalized input data to the currently focused user element.

9. The method of claim 7, wherein the normalized input data comprises normalized pointer data, and wherein providing the normalized input data to the user element comprises routing the normalized input data to the currently targeted user element.

10. A system comprising:

a processor; and

a memory communicatively coupled to the processor, the memory having stored therein computer-executable instructions, comprising;

an input namespace component configured to map a plurality of input device types to a provider, wherein the provider normalizes input data generated by the input device types to normalized input data that is not distinguishable based on respective input device types of the plurality of input device types; and

the provider configured to;

receive input from an input device having an input device type of the plurality of input device types; and

normalize the input data into the normalized input data, and

provide the normalized input data to a user interface element.

11. The system of claim 10, wherein the input device is a virtual input device or an actual input device.

12. The system of claim 10, wherein the normalized input data is normalized button data, normalized pointer data, or normalized command data plurality of input devices of different types comprises a pointer device and a button device.

13. The system of claim 10, wherein the plurality of input device types comprises a first button device type and a second button device type, wherein the provider normalizes the input into the normalized input data comprising a corresponding key code regardless of whether the input was obtained from the first button device type or the second button device type.

14. The system of claim 10, wherein the plurality of input device types comprises a first pointer device type and a second pointer device type, wherein the provider normalizes the input into the normalized input data comprising a corresponding pointer command regardless of whether the input was obtained from the first pointer device type or the second pointer device type.

15. The system of claim 10, wherein the user interface element receives the normalized input data, wherein the user interface element has an invoke handler associated with the user interface element, and wherein the invoke handler takes an invoke action based on the normalized input data.

16. The system of claim 10, wherein the user interface element receives the normalized input data, wherein the user interface element does not have an invoke handler associated with the user interface element, and wherein the user interface element bubbles up the normalized input data to a parent element.

17. The system of claim 10, further comprising, a tracking component that tracks which of a plurality of user elements is a currently targeted or currently focused user element, the tracking component coupled to the provider to identify to the provider which user element is to be provided the normalized input data.

18. A non transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a system including a processor to perform operations, the operations comprising:

mapping a plurality of input device types to a provider, wherein the provider normalizes input data generated by the input device types to normalized input data that is not distinguishable based on respective input device types of the plurality of input device types;

obtaining first input data from a first input device having a first input device type of the plurality of input device types;

normalizing, via the provider, the first input data into first normalized input data of the normalized input data; and

providing, by the provider the first normalized input data to a user interface element.

19. The non transitory computer-readable medium of claim 18, wherein the operations further comprise:

obtaining second input data from a second input device having a second input device type of the plurality of input device types;

normalizing, via the provider, the second input data into second normalized input data of the normalized input data; and

providing the second normalized input data to the user interface element.

20. The non transitory computer-readable medium of claim 18, wherein the first input device is a virtual input device.