Method and apparatus for touchless control of a device

• US 8,316,324 B2
• Filed: 09/05/2007
• Issued: 11/20/2012
• Est. Priority Date: 09/05/2006
• Status: Expired due to Fees

• Current Assignee: Navisense
• Original Assignee: Navisense
• Inventors: Boillot, Marc
• Primary Examiner(s): Tran, TuyetLien
• Assistant Examiner(s): ITURRALDE, ENRIQUE W
• Application Number:
  US11/850,637
• Publication Number:
  US 20080059915A1
• Time in Patent Office: 1,903 Days
• Field of Search: 715/268, 715701-702, 715/716, 715/754, 715/764, 715/773, 715848-852, 715856-866, 345156-158, 345168-170, 345173-177
• US Class Current: 715/863
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

A touchless sensing unit (110) and method (200) for operating a device via touchless control is provided. The method can include detecting (210) a finger (310) within a touchless sensory space (101), and handling (220) a control of the device (100) in accordance with a movement of the finger. A first finger movement can be recognized for acquiring a control, and a second finger movement for selecting a control. The method can include estimating (230) a location of the finger in the touchless sensory space for acquiring the control, and identifying a finger movement (250) of the finger at the location for selecting the control.

Claims

1. A touchless user interface integrated with a portable device, comprising:

a touchless sensing unit arranged external to a display of the device and integrated thereon for

generating an ultrasonic touchless sensory space projected one inch or more above and wider than a dimension of the display, and

tracking a three-dimensional finger movement that projects forward and retracts backward one inch or more above the display in the touchless sensory space by pulse-echo detection of reflected ultrasonic signals; and

a control unit for

calibrating a tracking of the finger or object movement from forward projecting and backward retracting movement along a principal axis of the ultrasonic touchless sensory space;

mapping a virtual coordinate system of the touchless sensory space in three-dimensions to a two-dimensional display coordinate system of the device according to the calibrating along the principal axis;

associating at least one forward projecting and backward retracting finger or object movement in the virtual coordinate system with a control action of a user interface component in the display coordinate system, and

adjusting at least one control of the device in response to the finger movement according to the mapping,

wherein said control action is an acquire, a select, or a release, and the calibrating compensates for a three-dimensional sweeping range of the finger movement in the touchless sensory space to an absolute range of locations in the device coordinate system.

2. The touchless user interface of claim 1, wherein the control unit identifies a movement of the finger in a region of the touchless sensory space, associates the movement with a control on a display of the device, and performs a control action on the control in response to the movement according to a state machine with a selection lock, wherein the state machine

acquires the control upon detecting a first movement;

assigns the lock to the control upon completion of the first movement; and

selects the control upon detecting a second movement if the control has the lock,

where continued touchless control is granted to acquire other controls during the lock but prevent a selection of the other controls until the lock is thereto assigned.

3. The touchless user interface of claim 2, wherein the control unit identifies a left-right jitter movement along a different principal axis and performs left-right calibration there along, and associates the left-right jitter movement with a control,

wherein a second left-right jitter movement to a left region acquires a left control to produce an acquired left control, or the second left-right jitter movement to a right region acquires a right control to produce an acquired right control.

4. The touchless user interface of claim 2, wherein the control unit identifies an up-down jitter movement of the finger along a different principal axis and performs up-down calibration there along, and associates the up-down jitter movement with a control,

wherein a second up-down jitter movement to the left region selects an acquired left control, or the second up-down jitter to the right region selects an acquired right control.

5. The touchless user interface of claim 1, wherein the control unit identifies an up-down jitter finger movement in the touchless sensory space, associates the up-down jitter finger movement with a scrolling of a menu on a display of the device, and performs a scrolling of the menu in response to the up-down jitter finger movement.

6. The touchless user interface of claim 2, wherein the control unit identifies a rotational finger movement in the touchless sensory space, associates the rotational finger movement with a scrolling of a menu on a display of the device, and performs a scrolling of the menu in response to the rotational movement.

7. The touchless user interface of claim 6, wherein a clockwise rotational finger movement corresponds to a down scroll, and a counter-clockwise rotational finger movement corresponds to an up scroll.

8. The touchless user interface of claim 1, wherein the control is one of a voice mail control, an email control, a display control, a game control, a contact list, an audio control, a camera control, a security control, or a telephone control.

9. The touchless user interface of claim 1, wherein the control unit differentiates between the presence of a thumb or a finger in the touchless sensory field for associating the control.

10. The touchless user interface of claim 1, wherein the device is a cell phone, a radio, a portable music player, a personal digital assistant, a portable memory module, a portable personal identifier device, a hand-held game, or a hand-held electronic security device.

11. The touchless user interface of claim 1, further comprising a button communicatively coupled to the controller, wherein the controller

detects a button press;

and

grants touchless control in response to the button press.

12. The touchless user interface of claim 1, wherein the button is associated with a control attribute, and the touchless control is associated with a value of the control attribute.

13. The touchless user interface of claim 1, further comprising at least one button communicatively coupled to the controller, wherein the controller

detects a first button press;

determines if the first button press is held down within a time duration,

if the button press is held down within the time duration,

grants touchless control of the device, else,

if the button press is held down longer than the time duration,

initiates a calibration of the device for touchless sensing responsive to recognizing a touchless finger movement upon release of the button.

14. A method for touchless control suitable for use with a mobile device, comprising:

generating an ultrasonic touchless sensory space projected one inch or more above and wider than a dimension of the display;

calibrating a tracking of a forward projecting and backward retracting finger one inch or more along a principal axis above and approximately perpendicular to the display;

detecting a presence of a finger or object within the touchless sensory space from a mapping of a virtual coordinate system along three-dimensions therein to a display coordinate system along two-dimensions of the device according to the calibrating along the principal axis,

wherein a relative range of locations of the finger movement in the virtual coordinate system is mapped to an absolute range of locations in the device coordinate system;

tracking three-dimensional movement of the finger within the touchless sensory space by pulse-echo detection of reflected ultrasonic signals one inch or more above the display;

associating at least one finger or object movement in the virtual coordinate system with a control of a user interface component in the display coordinate system and handling a control of the device in accordance with the finger movement, by

recognizing a first finger movement for acquiring a control;

locking the control upon completion of the first finger movement; and

recognizing a second finger movement for selecting the control;

where continued touchless control is granted to acquire other controls during the lock but prevent a selection of the other controls until the lock is thereto assigned.

15. The method of claim 14, wherein the tracking a movement further comprises:

estimating a location of the finger in the touchless sensory space for acquiring a control; and

identifying a finger movement of the finger at the location for selecting the control.

16. The method of claim 14, further comprising visually displaying an acknowledgment that the finger has acquired the control.

17. The method of claim 14, wherein the tracking a movement further comprises:

estimating a location of the finger in the touchless sensory space; and

associating a control on a display of the device with the location.

18. The method of claim 14, further comprising visually displaying an acknowledgment that the finger has acquired the control.

19. The method of claim 14, further comprising:

recognizing a finger swipe movement in the touchless sensory space for selecting the control.

20. The method of claim 14, further comprising:

acquiring the control in response to detecting a first finger movement at the location; and

selecting the control in response to a second finger movement,

wherein the acquiring locks the control, such that while the control remains locked, the second finger movement in the touchless sensory space does not select a second control

where during the lock continued touchless control is granted to acquire other controls until the lock is released.