MOBILE DEVICE WITH VIRTUAL KEYPAD

• US 20100214267A1
• Filed: 06/15/2006
• Published: 08/26/2010
• Est. Priority Date: 06/15/2006
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Yanming, Zou, Kongqiao, Wang, Radivojevic, Zoran, Kangas, Jari A., Hamalainen, Matti
• Application Number:
  US12/295,751
• Publication Number:
  US 20100214267A1
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/166 related to integrated arrangements for adjusting the position of the main body with respect to the supporting surface, e.g. legs for adjusting the tilt angle
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/0221 Arrangements for reducing keyboard size for transport or storage, e.g. foldable keyboards, keyboards with collapsible keys G06F3/0216 takes precedence; for keyboards integrated in a laptop computer G06F1/1666
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector
• CPC Class Codes: H04M 2250/70 methods for entering alphabetical characters, e.g. multi-tap or dictionary disambiguation

Abstract

A mobile electronic device with a virtual input device. The virtual input device includes an optical sensor for detecting movement of a user'"'"'s fingers over a work surface and an acoustic or vibration sensor, for detecting an impact of the user'"'"'s fingers on the work surface. A processor, coupled to the optical sensor and to the acoustic sensor, processes the detected signals as input for the electronic device. The user'"'"'s fingers can be shown on the display overlaid by a virtual mask. The virtual mask may show a keyboard of menu items.

Claims

1. A mobile electronic device with a virtual input device, said mobile electronic device comprising:

an optical sensor, for detecting movement of a user'"'"'s fingers over a work surface, and for generating a signal responsive to the detected movement;

an acoustic or vibration sensor, for detecting an impact of the user'"'"'s fingers on the work surface and for generating a signal responsive to the detected impact;

a first processor, coupled to the optical sensor and a second processor coupled to the acoustic sensor, for receiving and processing the detected signals as input for the electronic device.

2. A device according to claim 1, wherein said first processor is configured to determine the position where a finger impacts with the work surface from the signal generated by the optical sensor and to determine that an impact of a finger and the work surface has taken place from the signal generated by the acoustic or vibration sensor.

3. A device according to claim 1, wherein a particular input command is associated with each of a plurality of fingertip positions, and wherein said first processor is configured to accept the function associated with a given fingertip position when a fingertip movement towards the position concerned is detected to substantially coincide with the detection of a fingertip impact.

4. A device according to any of claims 1, wherein said first processor is configured to track the movement of the user'"'"'s fingers.

5. A device according to claim 4, further comprising a display, wherein said first processor is configured to provide real-time fingertip positions projected at the display.

6. A device according to claim 5, wherein said first or second processor is configured to show a virtual navigation mask on the display for providing optical feedback and/or guidance to the user.

7. A device according to claim 6, wherein said virtual navigation mask includes virtual input elements comprising virtual keypads or keyboards, icons and/or menu items.

8. A device according to claim 7, wherein said first or second processor is configured to highlight the virtual input element associated with the fingertip position when a fingertip movement towards said fingertip position and a fingertip impact have been detected.

9. A device according to claim 1, comprising a housing, wherein said housing comprises a front side in which the display is disposed and a rear side in which the optical sensor is disposed, said device further comprising a support member for keeping the housing in a substantially upright position.

10. A device according to claim 1, wherein said optical sensor is a general-purpose digital photo or video camera.

11. A device according to claim 1, wherein said acoustic sensor is a general-purpose microphone.

12. A device according to claim 1, wherein said device comprises a housing with hinged first and second housing parts, the first hinged housing part being configured to serve as a leg or foot for resting on the work surface to allow the second housing part to assume a substantially upright position.

13. A device according to claim 12, wherein the microphone or accelerometer is disposed in said first housing part and the display and the camera are disposed in the second housing part.

14. A device according to claim 1, wherein said second processor is configured to detect an impact of a finger with the work surface by performing a triggering algorithm in which the signal from the acoustic sensor is processed to perform a logical switch operation.

15. A device according to claim 14, wherein the triggering algorithm is based on:

the sounds or vibrations that travel through the solids in the environment of the acoustic or vibration sensor,

the sounds that travel through the air in the environment of the acoustic sensor,

or a combination of the sounds or vibrations that travel to the solids in the environment of the acoustic or vibration sensor and the sounds that travel through the air in the environment of the acoustic sensor.

16. A device according to claim 1, wherein said first processor and said second processor are integrated in one processor unit.

17. A method for generating input in a mobile electronic device comprising:

optically detecting movement of a user'"'"'s fingers over a work surface,

acoustically or vibrationally detecting impact of a user'"'"' fingers on the work surface, and

processing said signals as input.

18. A method according to claim 17, further comprising determining the position where a finger impacts with the work surface from the signal generated by the optical sensor and determining that an impact of a finger and the work surface has taken place from the signal generated by the acoustic or vibration sensor.

19. A method according to claim 17, further comprising associating a particular input command with each of a plurality of fingertip positions, and accepting the input command associated with a given fingertip position when a fingertip movement towards the position concerned is detected substantially simultaneously with the detection of a fingertip impact.

20. A method according to claim 17, further comprising processing the signals from the optical sensor and from the acoustic or vibration sensor to determine whether a fingertip of said user contacted a location defined on said virtual input device, and if contacted to determine what function of said virtual input device is associated with said location.

21. A method according to claim 17, further comprising tracking the movement of the user'"'"'s fingers and/or finger speed.

22. A method according to claim 21, wherein said device comprises a display, comprising providing real-time fingertip positions projected at the display.

23. A method according to claim 17, further comprising detecting an impact of a finger with the work surface by performing a triggering algorithm in which the signal from the acoustic sensor is processed to perform a logical switch operation.

24. A method according to claim 23, further comprising showing a virtual navigation mask on the display for providing optical feedback to the user.

25. A method according to claim 24, wherein said virtual navigation mask includes virtual input elements comprising virtual keypads or keyboards, icons and/or menu items.

26. A method according to claim 25, further comprising highlighting the virtual input element associated with the fingertip position when a fingertip movement towards said fingertip position and a fingertip impact have been detected.

27. A method according to claim 24, wherein the virtual mask only shows the characters or symbols associated with the key of a virtual keypad.

28. A method according to claim 27 wherein the virtual mask further shows key contours of the keys of a virtual keypad or shows partition lines between the characters or symbols associated with the key of a virtual keypad.

29. A mobile electronic device with a virtual input device, said mobile electronic device comprising:

an optical sensor, for detecting movement of a user'"'"'s fingers over a work surface, and for generating a signal responsive to the detected movement;

a processor coupled to said optical sensor for receiving and processing the detected signal as input for the electronic device, and

a display coupled to the optical sensor, wherein

the processor is configured to show a real-time representation of the position user'"'"'s fingers or fingertips as captured by the optical sensor on the display.

30. A device according to claim 29, wherein said processor is configured to provide real-time fingertip positions projected at the display.

31. A device according to claim 29, wherein the representation of the user'"'"'s fingers on the display is in the form of pointers or hand- or finger shadows.

32. A device according to claim 30, wherein the representation of the user'"'"'s fingers on the display is in the form real time images.

33. A device according to claim 29, wherein the representation of the user'"'"'s fingers on the display is projected into another application.

34. A device according to claim 29, wherein said processor is configured to display a virtual navigation mask over the representation of the users fingers for providing optical feedback to the user.

35. A device according to claim 34, wherein said virtual navigation mask includes virtual input elements.

36. A device according to claim 35, wherein said virtual input elements are virtual keypads or keyboards, icons and/or menu items.

37. A device according to claim 29, wherein the processor is configured to determine the position where a finger impacts with the work surface from the signal generated by the optical sensor.

38. A device according to claim 37, wherein a particular input command is associated with each of a plurality of fingertip positions, and wherein said processor is configured to accept the function associated with a given fingertip position when a fingertip movement towards the position concerned is detected.

39. A device according to claim 38, wherein said processor is configured to track the movement of the user'"'"'s fingers or fingertips.

40. A device according to claim 29, wherein said optical sensor is a general-purpose digital photo or video camera.

41. A method for generating input in a mobile electronic device with an optical sensor and a display comprising:

optically detecting movement of a user'"'"'s fingers over a work surface,

showing a real-time representation of the position of a user'"'"'s fingers or fingertips as captured by the optical sensor on the display, and

processing said signals as input.

42. A method according to claim 41, further comprising tracking the movement of the user'"'"'s fingers and/or fingers speed.

43. A method according to claim 41, further comprising determining the position where a finger impacts with the work surface from the signal generated by the optical sensor.

44. A method according to claim 41, further comprising associating a particular input command with each of a plurality of fingertip positions.

45. A method according to claim 41, further comprising processing the signals from the optical sensor to determine whether a fingertip of said user contacted a location defined on said virtual input device, and if contacted to determine what function of said virtual input device is associated with said location.

46. A method according to claim 41, further showing a virtual navigation mask on the display for providing optical feedback to the user.

47. A method according to claim 46, wherein said virtual navigation mask includes virtual input elements comprising virtual keypads or keyboards, icons and/or menu items.

48. A method according to claim 47, further comprising highlighting the virtual input element associated with the fingertip position when a fingertip movement towards said fingertip position and a fingertip impact have been detected.

49. A method according to claim 46, wherein the virtual mask only shows the characters or symbols associated with the key of a virtual keypad.

50. A method according to claim 49, wherein the virtual mask further shows key contours of the keys of a virtual keypad or shows partition lines between the characters or symbols associated with the key of a virtual keypad.