Recognition of free-form gestures from orientation tracking of a handheld or wearable device

• US 9,811,555 B2
• Filed: 09/27/2014
• Issued: 11/07/2017
• Est. Priority Date: 09/27/2014
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Mitri, Nicholas G., Wilkerson, Christopher B., Awad, Mariette
• Primary Examiner(s): DAVIS, TONY O
• Application Number:
  US14/499,133
• Publication Number:
  US 20160092504A1
• Time in Patent Office: 1,137 Days
• Field of Search: 345156, 345173, 345473-474, 348135
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser

Abstract

A user performs a gesture with a hand-held or wearable device capable of sensing its own orientation. Orientation data, in the form of a sequence of rotation vectors, is collected throughout the duration of the gesture. To construct a trace representing the shape of the gesture and the direction of device motion, the orientation data is processed by a robotic chain model with four or fewer degrees of freedom, simulating a set of joints moved by the user to perform the gesture (e.g., a shoulder and an elbow). To classify the gesture, a trace is compared to contents of a training database including many different users'"'"' versions of the gesture and analyzed by a learning module such as support vector machine.

Claims

1. An apparatus, comprising:

hardware logic configurable to cause a computing device to perform actions, the actions comprising;

capturing a sequence of rotation vectors from a device being moved to perform a gesture;

converting the sequence of rotation vectors to a sequence of corresponding device positions using a robotic chain model with at most 4 degrees of freedom; and

wherein the robotic chain model simulates a shoulder and an elbow of a user holding the device while performing the gesture; and

connecting the corresponding device positions to form a trace,

wherein the trace approximates a shape of the gesture; and

extracting features of the trace and using the features to classify the gesture;

wherein the gesture is classified by an algorithm comprising a support vector machine comparing the trace to contents of a training database and performing statistical analysis.

2. The apparatus of claim 1, wherein the hardware logic comprises at least one dedicated chip.

3. The apparatus of claim 1, wherein the hardware logic comprises at least one circuit formed on a general-purpose chip.

4. The apparatus of claim 1, wherein hardware logic on the device being moved executes at least part of the capturing of the sequence of rotation vectors.

5. The apparatus of claim 1, further comprising a wireless transmitter on the device being moved, wherein the wireless transmitter is operable to transmit the sequence of rotation vectors to a remote server.

6. The apparatus of claim 5, wherein at least part of the logic executing the converting of the sequence of rotation vectors comprises hardware logic located on the remote server.

7. The apparatus of claim 5, wherein at least part of the logic executing the connecting of the corresponding device positions to form the trace comprises hardware logic located on the remote server.

8. The apparatus of claim 5, further comprising a wireless receiver on the device being moved, wherein the wireless receiver is operable to receive the trace from the remote server.

9. The apparatus of claim 5, further comprising a display component on the device being moved, wherein the display component is operable to display the trace.

10. A non-transitory machine-readable storage medium programmed with instructions for components of a machine to perform actions, the actions comprising:

capturing a sequence of rotation vectors from a device being moved to perform a gesture;

converting the sequence of rotation vectors to a sequence of corresponding device positions using a robotic chain model with at most 4 degrees of freedom; and

connecting the corresponding device positions to form a trace,

wherein the robotic chain model simulates a shoulder and an elbow of a user holding the device while performing the gesture;

wherein the trace approximates a shape of the gesture; and

extracting features of the trace and using the features to classify the gesture;

wherein the gesture is classified by an algorithm comprising a support vector machine comparing the trace to contents of a training database and performing statistical analysis.

11. The non-transitory machine-readable storage medium of claim 10, wherein the actions further comprise displaying the trace on the device or on an apparatus wirelessly connected to the device.

12. The non-transitory machine-readable storage medium of claim 10, wherein the actions further comprise comparing the trace to a command gesture and, upon detecting a match between the trace and the command gesture, executing a command corresponding to the command gesture.

13. The non-transitory machine-readable storage medium of claim 10, wherein the actions further comprise storing the trace in a training database.

14. The non-transitory machine-readable storage medium of claim 10, wherein the actions further comprise reducing a number of data points defining the trace before classifying the gesture.

15. The non-transitory machine-readable storage medium of claim 14, wherein the reducing of the number of data points comprises fitting the trace to a curve.

16. The non-transitory machine-readable storage medium of claim 15, wherein the curve comprises a Bezier curve.

17. The non-transitory machine-readable storage medium of claim 16, wherein the Bezier curve comprises fewer than 15 control points.

18. The non-transitory machine-readable storage medium of claim 16, wherein the Bezier curve has a predetermined number of control points.

19. The non-transitory machine-readable storage medium of claim 16, wherein the actions further comprise minimizing a number of control points in the Bezier curve.

20. The non-transitory machine-readable storage medium of claim 10, wherein the trace is a three-dimensional trace, and wherein the actions further comprise transforming the trace into a two-dimensional trace.

21. The non-transitory machine-readable storage medium of claim 20, wherein the transforming of the trace into a two-dimensional trace comprises principal component analysis.

22. The non-transitory machine-readable storage medium of claim 10, wherein the actions further comprise sub-sampling the corresponding device positions before forming the trace.

23. The non-transitory machine-readable storage medium of claim 10, wherein the corresponding device positions are referenced to an origin at a shoulder joint of a user holding the device while performing the gesture.

24. The non-transitory machine-readable storage medium of claim 10, wherein the converting of the sequence of rotation vectors to a sequence of corresponding device positions comprises an inverse-kinetic calculation and a forward-kinetic calculation.

25. The non-transitory machine-readable storage medium of claim 10, wherein at least one of the converting of the sequence of rotation vectors to the sequence of corresponding device positions or the connecting of the corresponding device positions to form a trace occurs on a remote server wirelessly connected to the device.