Continuous motion controls operable using neurological data

• US 10,203,751 B2
• Filed: 05/11/2016
• Issued: 02/12/2019
• Est. Priority Date: 05/11/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Keskin, Cem, Shahid, Khuram, Chau, Bill, Kim, Jaeyoun, Koishida, Kazuhito
• Primary Examiner(s): Nguyen, Jennifer T
• Application Number:
  US15/152,403
• Publication Number:
  US 20170329392A1
• Time in Patent Office: 1,007 Days
• Field of Search: 345156-157
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

Computer systems, methods, and storage media for generating a continuous motion control using neurological data and for associating the continuous motion control with a continuous user interface control to enable analog control of the user interface control. The user interface control is modulated through a user'"'"'s physical movements within a continuous range of motion associated with the continuous motion control. The continuous motion control enables fine-tuned and continuous control of the corresponding user interface control as opposed to control limited to a small number of discrete settings.

Claims

1. A computer system configured for using neurological data to modulate a continuous user interface control, the computer system comprising:

one or more processors; and

one or more hardware storage devices having stored thereon computer-executable instructions which are executable by the one or more processors to cause the computing system to perform at least the following;

obtain a generic continuous motion control that maps neurological data obtained from a plurality of users to a set of physical movements within a continuous range of motion of the plurality of users, wherein the generic continuous motion control is sufficient to provide limited responsiveness and accuracy when associated to a generic continuous user interface control to modulate the generic continuous user interface control in response to detected physical movement within the continuous range of motion;

tune the generic continuous motion control to generate a continuous motion control for a particular user by at least mapping neurological data obtained from the particular user while the particular user is performing the set of physical movements within the continuous range of motion;

associate the continuous motion control to a continuous user interface control;

detect user input comprising neurological data associated with a physical movement within the continuous range of motion; and

modulate the continuous user interface control in a manner corresponding to the physical movement of the user within the continuous range of motion.

2. The computing system of claim 1, wherein the set of physical movements within the continuous range of motion includes physical movements that are differentiated from one another based on relative position of a body part within the continuous range of motion.

3. The computing system of claim 1, wherein the set of physical movements within the continuous range of motion includes physical movements that are differentiated from one another based on relative speed at which the physical movements are made within the continuous range of motion.

4. The computing system of claim 1, wherein the continuous range of motion includes movements of a hand, foot, arm, leg, face, or head.

5. The computing system of claim 1, wherein the set of physical movements within the continuous range of motion includes physical movements that are differentiated from one another based on relative force.

6. The computing system of claim 1, wherein the continuous user interface control is an audio volume level control.

7. The computing system of claim 1, wherein continuous user interface control is operable in a virtual reality or an augmented reality environment.

8. The computing system of claim 1, wherein the continuous user interface control is operable with a continuous range of settings as opposed to a limited number of discrete settings.

9. The computing system of claim 1, wherein the user input comprising neurological data is obtained from an electroencephalography sensor contemporaneously worn by the user.

10. The computing system of claim 1, wherein the continuous motion control is operable to modulate the continuous user interface control without input obtained from a camera.

11. The computing system of claim 1, wherein the continuous user interface control is a virtual analog joystick.

12. The computing system of claim 11, wherein the continuous motion control is operable to modulate the virtual analog joystick through a set of thumb movements.

13. The computing system of claim 1, wherein the continuous motion control is operable to modulate the continuous user interface control through foot movements.

14. The computing system of claim 1, wherein the continuous motion control is operable to modulate the continuous user interface control through facial movements.

15. The computing system of claim 1, wherein the continuous motion control maps neurological data obtained from a plurality of users using machine learning techniques.

16. A computer-implemented method for using neurological data to modulate a continuous user interface control, the method being implemented by a computing system that includes at least one processor and one or more hardware storage devices having stored thereon computer-executable instructions that are executable by the at least one processor to cause the computing system to implement the method, the method comprising:

obtaining a generic continuous motion control that maps neurological data obtained from a plurality of users to a set of physical movements within a continuous range of motion of the plurality of users, wherein the generic continuous motion control is sufficient to provide limited responsiveness and accuracy when associated to a generic continuous user interface control to modulate the generic continuous user interface control in response to detected physical movement within the continuous range of motion;

tuning the generic continuous motion control to generate a continuous motion control for a particular user by at least mapping neurological data obtained from the particular user while the particular user is performing the set of physical movements within the continuous range of motion;

associating the continuous motion control to a continuous user interface control;

detecting user input comprising neurological data associated with a physical movement within the continuous range of motion; and

modulating the continuous user interface control in a manner corresponding to the physical movement of the user within the continuous range of motion.

17. The method of claim 16, wherein the set of physical movements within the continuous range of motion includes physical movements that are differentiated from one another based on relative position of a body part within the continuous range of motion and based on relative speed at which the physical movements are made within the continuous range of motion.

18. The method of claim 16, wherein the continuous motion control is operable to modulate the continuous user interface control in an analog fashion.

19. The method of claim 16, wherein the continuous motion control is operable to modulate the continuous user interface control without input obtained from a camera.

20. One or more hardware storage device having stored thereon computer-executable instructions which are executable by one or more processors of a computing system to use neurological data to modulate a continuous user interface control by at least causing the computer system to perform a method that includes the following:

obtaining a generic continuous motion control that maps neurological data obtained from a plurality of users to a set of physical movements within a continuous range of motion of the plurality of users, wherein the generic continuous motion control is sufficient to provide limited responsiveness and accuracy when associated to a generic continuous user interface control to modulate the generic continuous user interface control in response to detected physical movement within the continuous range of motion;

tuning the generic continuous motion control to generate a continuous motion control for a particular user by at least mapping neurological data obtained from the particular user while the particular user is performing the set of physical movements within the continuous range of motion;

associating the continuous motion control to a continuous user interface control;

detecting user input comprising neurological data associated with a physical movement within the continuous range of motion; and

modulating the continuous user interface control in a manner corresponding to the physical movement of the user within the continuous range of motion.