Method and apparatus for touchless calibration

• US 7,961,173 B2
• Filed: 09/05/2007
• Issued: 06/14/2011
• Est. Priority Date: 09/05/2006
• Status: Expired due to Fees

• Current Assignee: Navisense
• Original Assignee: Navisense
• Inventors: Boillot, Marc
• Primary Examiner(s): Beck, Alexander S
• Assistant Examiner(s): CERULLO, LILIANA P
• Application Number:
  US11/850,634
• Publication Number:
  US 20080055247A1
• Time in Patent Office: 1,378 Days
• Field of Search: 345156-184, 715700-866, 710 1- 74, 178 1801- 1907, 341/7.1, 340/22
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A touchless sensing unit (110) and method (210) for calibrating a mobile device for touchless sensing is provided. The method can include evaluating (214) a finger movement within a touchless sensory space (101), estimating (216) a virtual coordinate system (320) from a range of finger movement, and mapping (218) the virtual coordinate system to a device coordinate system (330).

Claims

1. A method for calibrating touchless sensory controls of a mobile device, comprising:

detecting a first button press;

determining if the first button press is held down within a time duration; and

if so, granting touchless control of the mobile device if the button press is held down within the time duration,

else, starting a range calibration of the mobile device for touchless control if the button press is held down longer than the time duration,

wherein the range calibration comprises;

tracking a first full radial range of touchless finger tip movement corresponding to a left-right radial finger tip track within a virtual coordinate system of a touchless sensory field with respect to a stationary origin of the finger tip movement during the first full radial range;

aligning the first full radial range corresponding to a left-right radial finger tip track with a X principal axis of the virtual coordinate system;

tracking a second full radial range of touchless finger tip movement corresponding to an up-down radial finger tip track within the virtual coordinate system of the touchless sensory field with respect to the stationary origin of the finger tip movement during the second full radial range;

aligning the second full radial range of the up-down radial finger tip track with a Y principal axis of the virtual coordinate system;

mapping both the X principal and Y principal axes of the virtual coordinate system to principal axes of a device coordinate system of a display of the mobile device,

wherein the range calibration establishes how a user'"'"'s hand or finger is oriented with respect to its stationary origin relative to a touchless sensing unit producing the touchless sensing field and maps the principal axes to the display according to the orientated hand or finger,

wherein one or more range calibrations of differing hand orientations at differing origin locations permit respective alignment of the same X and Y principal axes with the device coordinate system to support different touchless user interface control mappings.

2. The method of claim 1, further comprising:

detecting a second button press; and

disabling touchless control of the mobile device in response to the second button press.

3. The method of claim 1, wherein initiating the calibration, further comprises:

presenting a first indicator that calibration is starting;

evaluating the first and second full radial range of finger tip movement within the touchless sensory field;

mapping the first and second full radial range of finger tip movements to a display range; and

presenting a second indicator when the mapping is complete,

wherein the mapping maps at least one location of the finger in the touchless sensory field to at least one location in the display range to assign a virtual coordinate system to the touchless sensory field.

4. The method of claim 3, wherein presenting an indicator includes displaying a visual indicator or audibly playing a sound.

5. The method of claim 1, wherein mapping the range further comprises:

estimating an elliptical trajectory of the virtual coordinate system from a full rotational range of finger movement;

determining and visually presenting an origin of the virtual coordinate system by finding the stationary origin of the finger with respect to a circumferential boundary of the finger movement along the elliptical trajectory in a display range associated with a device coordinate system;

identifying the stationary origin of the finger in the virtual coordinate system from a center of the circumferential boundary;

determining a difference between the stationary origin of the fingerin the virtual coordinate system and a display origin in the device coordinate system; and

warping the virtual coordinate system to the device coordinate system based on the difference.

6. The method of claim 5, wherein the warping includes:

translating the circumferential boundary along the elliptical trajectory through an affine transformation; and

non-linearly mapping the elliptical trajectory onto a rectangular area, wherein the mapping accounts for the movement of the finger along the elliptical trajectory and the first and second full radial range of finger movement along the principal axes of the virtual coordinate system with respect to a principal axes of the device coordinate system

such that finger tip locations within a full radial range of finger tip movement within the elliptical trajectory correspond to a unique point location within a rectangle of the display.

7. The method of claim 5, further comprising:

presenting a third indicator when the warping is complete; and

granting touchless control of the mobile device, wherein the touchless control uses the virtual coordinate system to adjust at least one control in a device coordinate system in response to a finger movement.

8. A method for calibrating a mobile device for touchless control, comprising:

evaluating two full range finger movements within a touchless sensory space each maximally wide across a full radial range of finger motion; and

associating first and second finger movements with at least one calibration range of touchless control for the mobile device,

wherein the associating calibrates the touchless control with the at least one finger movement,

where the calibrating includes

tracking a first full radial range of touchless finger tip movement corresponding to a left-right radial finger tip track within a virtual coordinate system of the touchless sensory field with respect to a stationary origin of the finger tip movement during the first full radial range;

aligning the first full radial range corresponding to a left-right radial finger tip track with a X principal axis of the virtual coordinate system;

tracking a second full radial range of touchless finger tip movement corresponding to an up-down radial finger tip track within the virtual coordinate system of the touchless sensory field with respect to the stationary origin of the finger tip movement during the second full radial range;

aligning the second full radial range of the up-down radial finger tip track with a Y principal axis of the virtual coordinate system;

mapping both the X principal and Y principal axes of the virtual coordinate system to a device coordinate system of a display of the mobile device,

wherein the range calibration establishes how a user'"'"'s hand or finger movement is oriented with respect to its origin location relative to a touchless sensing unit producing the touchless sensing field and the mapping of the principal axes to the display,

wherein one or more range calibrations of differing hand orientations at differing origin locations permit respective alignment of the same X and Y principal axes to support different touchless user interface control mappings.

9. The method of claim 8, wherein the associating first and second finger movement further comprises:

estimating a virtual coordinate system from the two full range finger movements; and

mapping the virtual coordinate system to a device coordinate system;

wherein the mapping maps at least one location of the finger in the touchless sensory field to at least one location in a device coordinate system.

10. The method of claim 9, further comprising at least one of:

estimating the left-right full range of radial finger movement for identifying an X range along the X principal axis;

estimating the up-down full range of radial finger movement for identifying a Y range along the Y principal axis;

estimating a forward-back range of finger movement for identifying a Z range along a Z principal axis; and

estimating a rotational range of finger movement for identifying the circumferential range.

11. The method of claim 8, further comprising at least one of:

presenting a first indicator when the calibrating is starting;

presenting a second indicator when the associating is complete;

or presenting a third indicator when the calibrating is complete.

12. The method of claim 8, further comprising at least one of:

presenting an image for teaching the at least one finger movement;

or presenting a video clip for teaching the at least one finger movement.

13. The method of claim 8, wherein the associating the at least one finger movement includes at least one of:

learning an up-down jitter movement;

learning a left-right jitter movement;

learning a back-and-forth sweep finger movement;

learning a forward projecting finger movement;

learning a rotational finger movement;

or

learning a finger swipe movement.

14. The method of claim 8, further comprising:

identifying a location of the finger movement; and

associating the location with a control on a display of the mobile device, wherein a variance of the finger movement at the location is associated with a calibration of the control.

15. The method of claim 14, wherein the associating a location includes mapping a three-dimensional finger movement to a control command.

16. The method of claim 14, wherein the associating a location includes mapping a two-dimensional finger movement to a control command.

17. A touchless unit communicatively coupled to a mobile device, comprising:

a plurality of sensors for;

evaluating two full range finger movements within a touchless sensory field each maximally across a full radial range of finger motion; and

associating the two full range finger movements with at least one control of the mobile device for calibrating a touchless control of the mobile device, by

ultrasonic tracking a first and second full radial range of touchless finger tip movement within a virtual coordinate system of the touchless sensory field with respect to a stationary origin of the finger tip movement during the first and second full radial range;

respectively aligning the first full radial range and second full radial range with a X and Y principal axis of the virtual coordinate system;

estimating an elliptical trajectory of the virtual coordinate system from a full rotational range of finger movement with respect to the stationary origin;

translating a circumferential boundary of the elliptical trajectory through an affine transformation to determine the stationary origin;

mapping the X principal and Y principal axes of the virtual coordinate system to a device coordinate system of a display of the mobile device according to the stationary origin and respective first and second full radial range of finger movement along the X and Y principal axes,

wherein the calibrating establishes how a user'"'"'s hand or finger movement is oriented with respect to its stationary origin relative to a touchless sensing unit producing the touchless sensory field and the mapping of the virtual coordinate system to the device coordinate system.

18. The touchless unit of claim 17, further comprising a processor that:

tracks a location and full range movement of the finger based on pulse-echo detection;

measures between two echo returns a phase angle for determining a centering of the stationary origin;

estimates a virtual coordinate system from the first and second full radial range of finger movement with an coordinate system origin corresponding to the stationary origin of the finger; and

non-linearly maps the virtual coordinate system to a device coordinate system, wherein the mapping maps at least one location of the finger in the touchless sensory field to at least one location in a device coordinate system.