Human interface device input fusion

• US 10,146,355 B2
• Filed: 03/26/2015
• Issued: 12/04/2018
• Est. Priority Date: 03/26/2015
• Status: Active Grant

• Current Assignee: Lenovo PC International Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: Zaitsev, Grigori, Raux, Antoine Roland, Waltermann, Rod D.
• Primary Examiner(s): Lin, Chun-Nan
• Application Number:
  US14/670,096
• Publication Number:
  US 20160283016A1
• Time in Patent Office: 1,349 Days
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

One embodiment provides a method, including: accepting, at a first input device of an electronic device, a first user input for an object resident on the electronic device; partially completing, using a processor, a user action based on the first user input; accepting, at another type of input device of the electronic device, a user input associated with the first user input for the object; combining, using a processor, the first user input for the object and the user input; and completing, using a processor, the user input action for the object based on the combined first user input and the user input. Other aspects are described and claimed herein.

Claims

1. A method, comprising:

accepting, at a first input device of an electronic device, a first user command for an object resident on the electronic device;

partially completing, using a processor, an action based on the first user command for the object, wherein the action requires two or more user commands to complete;

accepting, prior to completion of the action based on the first user command for the object and within a dynamically determined time period defined by a content of at least one of the first user command for the object and a second user command for the object, at another type of input device of the electronic device, the second user command for the object associated with the first user command for the object;

combining, using a processor, the first user command for the object and the second user command for the object; and

completing, using a processor, the action for the object based on the combined first user command for the object and the second user command for the object.

2. The method of claim 1, wherein the second user command for the object comprises voice input.

3. The method of claim 1, wherein the action is selected from the group consisting of an object copy action, an object cut and paste action, an object movement action, an object share action, and an object transmission action.

4. The method of claim 1, wherein the first user command for the object is an object indication action.

5. The method of claim 4, wherein the second user command for the object is a destination indication.

6. The method of claim 1, wherein the first user command for the object is accepted by detecting touch input.

7. The method of claim 6, wherein the detecting touch input comprises detecting pen input.

8. The method of claim 1, wherein the first user command for the object is accepted by detecting gesture input.

9. The method of claim 8, wherein the detecting gesture input comprises detecting pen input.

10. The method of claim 1, wherein the action is completed by accepting an additional user command for the object.

11. An electronic device, comprising:

a first input device;

another type of input device;

a processor operatively coupled to the first input device and the another type of input device;

a memory device that stores instructions executable by the processor to;

accept, at the first input device, a first user command for an object resident on the electronic device;

partially complete an action based on the first user command for the object, wherein the action requires two or more user commands to complete;

accept, prior to completion of the action based on the first user command for the object and within a dynamically determined time period defined by a content of at least one of the first user command for the object and a second user command for the object, at the another type of input device of the electronic device, the second user command for the object associated with the first user command for the object;

combine the first user command for the object and the second user command for the object; and

complete the action for the object based on the combined first user command for the object and the second user command for the object.

12. The electronic device of claim 11, wherein the second user command for the object comprises voice input.

13. The electronic device of claim 11, wherein the action is selected from the group consisting of an object copy action, an object cut and paste action, an object movement action, an object share action, and an object transmission action.

14. The electronic device of claim 11, wherein the first user command for the object is an object indication action.

15. The electronic device of claim 14, wherein the second user command for the object is a destination indication.

16. The electronic device of claim 11, wherein the first user command for the object is accepted by detecting touch input.

17. The electronic device of claim 16, wherein the detecting touch input comprises detecting pen input.

18. The electronic device of claim 11, wherein the first user command for the object is accepted by detecting gesture input.

19. The electronic device of claim 18, wherein the detecting gesture input comprises detecting pen input.

20. A product, comprising:

a storage device having code stored therewith, the code being executable by the processor and comprising;

code that accepts, from a first input device of an electronic device, a first user command for an object resident on the electronic device;

code that partially completes an action based on the first user command for the object, wherein the action requires two or more user commands to complete;

code that accepts, prior to completion of the action based on the first user command for the object and within a dynamically determined time period defined by a content of at least one of the first user command for the object and a second user command for the object, from another type of input device of the electronic device, the second user command for the object associated with the first user command for the object;

code that combines the first user command for the object and the second user command for the object; and

code that completes the action for the object based on the combined first user command for the object and the second user command for the object.