Using multi-modal input to control multiple objects on a display

• US 20110161890A1
• Filed: 12/31/2009
• Published: 06/30/2011
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Anderson, Glen J., Doherty, Rina A.
• Granted Patent: US 8,977,972 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Embodiments of the invention are generally directed to systems, methods, and machine-readable mediums for implementing gesture-based signature authentication. In one embodiment, a system may include several modal input devices. Each modal input device is capable of retrieving a stream of modal input data from a user. The system also includes modal interpretation logic that can interpret each of the retrieved modal input data streams into a corresponding of set of actions. The system additionally includes modal pairing logic to assign each corresponding set of actions to control one of the displayed objects. Furthermore, the system has modal control logic which causes each displayed object to be controlled by its assigned set of actions.

Claims

1. A modular multi-modal input system, comprising:

a plurality of modal input devices, each modal input device to retrieve a stream of modal input data from a user;

modal interpretation logic to interpret each of the plurality of retrieved modal input data streams into a corresponding of set of actions;

modal pairing logic to assign each corresponding set of actions to control one of a plurality of displayed objects; and

modal control logic to control each displayed object by its assigned set of actions.

2. The modular multi-modal input system of claim 1, wherein the modal pairing logic is operable to dynamically change a control assignment of a first displayed object of the plurality of displayed objects from a first modal input data stream to a second modal input data stream.

3. The modular multi-modal input system of claim 1, wherein the modal pairing logic is operable to dynamically change a first modal input data stream of the plurality of modal input data streams from controlling a first displayed object to a second displayed object.

4. The modular multi-modal input system of claim 1, further comprising:

modal input management logic to manage two or more of the a plurality of modal input devices as a group for the user, the group of modal input devices allowing the user to provide simultaneous control of two or more of the plurality of displayed objects.

5. The modular multi-modal input system of claim 4, wherein at least two of the plurality of modal input devices in the group are hands-free modal input devices.

6. The modular multi-modal input system of claim 4, wherein a first modal input device in the group is of a different modal type than a second modal input device in the group.

7. The modular multi-modal input system of claim 1, wherein at least one modal input device of the plurality of modal input devices is one of a video camera, a microphone, and an accelerometer.

8. A method, comprising:

retrieving a plurality of streams of modal input data from a user;

interpreting each of the plurality of retrieved modal input data streams into a corresponding of set of actions;

assigning each corresponding set of actions to control one of a plurality of displayed objects; and

controlling each displayed object by its assigned set of actions.

9. The method of claim 8, further comprising:

dynamically changing a control assignment of a first displayed object of the plurality of displayed objects from a first modal input data stream to a second modal input data stream.

10. The method of claim 8, further comprising:

dynamically changing a first modal input data stream of the plurality of modal input data streams from controlling a first displayed object to a second displayed object.

11. The method of claim 8, further comprising:

managing two or more of the a plurality of modal input devices as a group for the user, the group of modal input devices allowing the user to provide simultaneous control of two or more of the plurality of displayed objects.

12. The method of claim 11, wherein at least two of the plurality of modal input devices in the group are hands-free modal input devices.

13. The method of claim 11, wherein a first modal input device in the group is of a different modal type than a second modal input device in the group.

14. The method of claim 8, wherein at least one modal input device of the plurality of modal input devices is one of a video camera, a microphone, and an accelerometer.

15. A machine-readable medium having stored thereon instructions, which if executed by a machine causes the machine to perform a method comprising:

retrieving a plurality of streams of modal input data from a user;

interpreting each of the plurality of retrieved modal input data streams into a corresponding of set of actions;

assigning each corresponding set of actions to control one of a plurality of displayed objects; and

controlling each displayed object by its assigned set of actions.

16. The machine-readable medium of claim 15, wherein the performed method further comprises:

dynamically changing a control assignment of a first displayed object of the plurality of displayed objects from a first modal input data stream to a second modal input data stream.

17. The machine-readable medium of claim 15, wherein the performed method further comprises:

dynamically changing a first modal input data stream of the plurality of modal input data streams from controlling a first displayed object to a second displayed object.

18. The machine-readable medium of claim 15, wherein the performed method further comprises:

managing two or more of the a plurality of modal input devices as a group for the user, the group of modal input devices allowing the user to provide simultaneous control of two or more of the plurality of displayed objects.

19. The machine-readable medium of claim 18, wherein at least two of the plurality of modal input devices in the group are hands-free modal input devices.

20. The machine-readable medium of claim 18, wherein a first modal input device in the group is of a different modal type than a second modal input device in the group.

21. The machine-readable medium of claim 15, wherein at least one modal input device of the plurality of modal input devices is one of a video camera and a microphone.