Method and device for touchless signing and recognition

• US 8,904,312 B2
• Filed: 11/07/2007
• Issued: 12/02/2014
• Est. Priority Date: 11/09/2006
• Status: Expired due to Fees

• Current Assignee: Navisense
• Original Assignee: Navisense
• Inventors: Boillot, Marc
• Primary Examiner(s): Pesin, Boris
• Assistant Examiner(s): BADAWI, ANGIE M
• Application Number:
  US11/936,777
• Publication Number:
  US 20110041100A1
• Time in Patent Office: 2,582 Days
• Field of Search: 715/863
• US Class Current: 715/863
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A touchless sensor device (110) for touchless signing and recognition is provided. The sensor device can include a recognition engine (114) for recognizing at least one finger sign (140), and a controller (120) for composing a text from the recognized at least one finger sign and providing predictive texting. A recognized pattern can be an alphanumeric character or a finger gesture. The controller can generate a trace (145) from the finger sign. The trace can include spatio-temporal information (153) that is characteristic to a touchless writing style. The controller can provide text messaging services, email composition services, biometric identification services, phone dialing, and navigation entry services through touchless finger signing.

Claims

1. A touchless user interface device, comprising:

a sensor device for tracking a movement of a finger gesture in a three-dimensional touchless sensing space, and

tracking features comprising a relative location, acceleration and movement of the finger gesture in the three-dimensional sensing space to produce a trace;

and

a state driven gesture recognition engine for

learning gesture characteristics of a textual articulation and a non-stiction writing style in the three-dimensional sensory space from the acceleration and movement features of the trace;

discriminating between a 2D writing style and a 3D writing style of the gesture characteristics; and

classifying the finger gesture from the stylistic information upon identifying in the trace at least one pattern established from a finger start action and a finger stop action.

2. The touchless user interface device of claim 1, further comprising

a display for displaying an alphanumeric character responsive to the finger gesture or a finger swipe;

and

an audio module for audibly presenting the at least one pattern.

3. The touchless user interface device of claim 1, wherein the sensor device is integrated within a mobile device, a headset, or an earpiece.

4. The touchless user interface device of claim 1, further comprising a controller operatively coupled to the sensor device and the state driven gesture recognition engine, the controller:

emitting ultrasonic waveforms from ultrasonic transducers to reflect off a finger in a three-dimensional touchless sensing space;

digitally sampling the reflected ultrasonic waveforms off the finger to produce a sampled reflected ultrasonic waveform;

tracking via pulse echo detection a relative location and movement of the finger in the three-dimensional ultrasonic sensing space from differential time of flight waveform analysis of the sampled reflected ultrasonic waveforms to produce the trace;

weighting a Time of Flight (TOF) ultrasonic distance measurement with a differential Time of Flight (dTOF) ultrasonic measurement for producing the location and the relative displacement as the finger accelerates and decelerates,

creating the trace of the movement from the location and relative displacement received from the sensory device; and

identifying finger start and finger stop actions from the trace for inputting continuous finger gestures to the state driven gesture recognition engine.

5. The touchless user interface device of claim 4, wherein the state driven gesture recognition engine is a neural network or Hidden Markov Model (HMM) that learns to discriminate between the 2D writing style and the 3D writing style to classify the at least one pattern and stylistic information of a writing style from the trace.

6. The touchless user interface device of claim 4, wherein the sensor device projects the touchless sensing space on or above a surface of a mobile device.

7. The touchless user interface device of claim 4, wherein the trace is a differential trajectory of an x, y, and z component of the movement.

8. The touchless user interface device of claim 5, wherein the neural network classifies the pattern using a spatio-temporal trace of the movement of the finger.

9. The touchless user interface device of claim 4, wherein a temporal trace is an acceleration vector.

10. The touchless user interface device of claim 4, wherein the state driven gesture recognition engine is a Neural Network or Hidden Markov Model that learns to discriminate between the 2D writing style and the 3D writing style to classify the at least one pattern from the trace.

11. The touchless user interface device of claim 4, wherein the controller performs biometric identification on a finger signing to verify an identity of a user.

12. The touchless user interface device of claim 4, wherein the controller performs a software action in response to the state driven gesture recognition engine recognizing the finger gesture.

13. A method for touchless sensing, comprising

by way of a sensory device, tracking a movement of a finger gesture in a three-dimensional touchless sensing space, and

tracking features comprising a relative location, acceleration and movement of the finger gesture in the three-dimensional sensing space to produce a trace;

by way of a state driven gesture recognition engine,

learning gesture characteristics of a textual articulation and a non-stiction writing style in the three-dimensional sensory space from the acceleration and movement features of the trace;

discriminating between a 2D writing style and a 3D writing style of the gesture characteristics; and

classifying the finger gesture from the stylistic information upon recognizing in the trace at least one pattern established from a gesture start action and a gesture stop action.

14. The method of claim 13, further comprising a controller that performs differential measurement for finger acceleration and deceleration, and by way of the state machine determines start and stop completions in view of the stylistic information conveyed by the rate of change.

15. The method of claim 13, wherein the controller provides text messaging services for touchless finger signing.

16. The method of claim 13, wherein the controller provides email services for touchless finger signing.

17. The method of claim 13, wherein the controller provides navigation services for touchless finger signing.

18. The method of claim 13, wherein the controller recognizes a touchless finger swipe.

21. The method of claim 13, wherein the sensor device performs the tracking by way of optical or camera elements.

22. The method of claim 13, wherein the sensory device performs the tracking by way of acoustic or ultrasonic sensors.

19. A touchless signing engine comprising:

a sensor device for tracking a finger movement in a touchless sensing space above a display and creating a trace from the finger movement from acceleration features of the finger movement by

emitting ultrasonic waveforms from ultrasonic transducers to reflect off a finger in a three-dimensional touchless sensing space;

digitally sampling the reflected ultrasonic waveforms off the finger to produce a sampled reflected ultrasonic waveform;

producing a relative location and movement of the finger in the three-dimensional ultrasonic sensing space from differential time of flight waveform analysis of the sampled reflected ultrasonic waveforms,

creating and presenting a trace of the location and the relative displacement received from the sensory device on the display

and, a state driven gesture recognition engine for

learning gesture characteristics of a textual articulation and a non-stiction writing style in the three-dimensional sensory space from the acceleration features of the trace;

discriminating between a 2D writing style and a 3D writing style of the gesture characteristics; and

classifying the finger gesture from the stylistic information upon recognizing a pattern from the trace established from a finger start action and a finger stop action.

20. The touchless signing engine of claim 19, further comprising a controller for performing a software action in response to the state driven gesture recognition engine recognizing a finger signor swine.