Gesture Based User Interface Supporting Preexisting Symbols

• US 20100328201A1
• Filed: 06/29/2010
• Published: 12/30/2010
• Est. Priority Date: 03/23/2004
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Matsumoto, Hitoshi, Marvit, David L., Reinhardt, Albert H. M.
• Granted Patent: US 8,692,764 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A motion controlled handheld device includes a display having a viewable surface and operable to generate an image and a gesture database maintaining a plurality of gestures. Each gesture is defined by a motion of the device with respect to a first position of the device. The gestures comprise symbol gestures each corresponding to a character from a preexisting character set. The device includes an application database maintaining at least one application and a gesture mapping database comprising a gesture input map for the application. The gesture input map comprises mappings of the symbol gestures to corresponding inputs for the application. The device includes a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface. The device also includes a control module operable to load the application, to track movement of the handheld device using the motion detection module, to compare the tracked movement against the symbol gestures to identify a matching symbol gesture, to identify, using the gesture input map, the corresponding input mapped to the matching symbol gesture, and to provide the corresponding input to the application.

Claims

1-21. -21. (canceled)

22. A handheld motion control system comprising:

a motion detector having a three-axis acceleration sensor;

a device locator configured to identify and select a device to be controlled from among a plurality of devices based on a direction of movement of the motion detector; and

a wireless communication interface configured to communicate information obtained by the motion detector to a processing apparatus of the selected device configured to obtain the information from the wireless communication interface to process the information.

23. The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera along a third axis perpendicular to each of a first axis and a second axis of the three-axis acceleration sensor.

24. The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera in a direction along a longitudinal side of the handheld device.

25. The motion control system according to claim 22, wherein the handheld device further comprises a camera with an axis of the camera of the handheld device along an axis that is parallel to a mounting surface of the three-axis acceleration sensor.

26. The motion control system according to claim 22, wherein the motion detector of the handheld device includes a processor operable to allow a user to repeatedly selectively engage and disengage motion sensitivity.

27. The motion control system according to claim 22, wherein the handheld device is operable to provide feedback information to notify a user that a particular motion of the handheld device is recognized as a particular gesture.

28. The motion control system according to claim 27, wherein the handheld device is operable to provide the feedback information in an audio format.

29. The motion control system according to claim 27, wherein the handheld device is operable to provide the feedback information in a vibratory format.

30. The motion control system according to claim 22, wherein the processing apparatus is a component of a DVD player.

31. The motion control system according to claim 22, wherein the processing apparatus is a component of a television.

32. The motion control system according to claim 22, wherein the handheld device is connected to other computer peripheral equipment.

33. The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies a user of output.

34. The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies a user of a battery level.

35. The motion control system according to claim 22, wherein the handheld device further comprises a unit that notifies via a sound.

36. The motion control system according to claim 22, wherein the handheld device further comprises a user interface.

37. The motion control system according to claim 22, wherein the handheld device further comprises a unit that gives a vibratory feedback.

38. The motion control system according to claim 23, wherein the camera is configured to provide information used for a distance measurement.

39. The motion control system according to claim 24, wherein the camera is configured to provide information used for a distance measurement.

40. The motion control system according to claim 25, wherein the camera is configured to provide information used for a distance measurement.

41. The motion control system according to claim 22, wherein the processing apparatus comprises a calendar application.

42. The motion control system according to claim 22, wherein the motion detector is operable to allow a user to reset a base reference position of the handheld device.

43. The motion control system according to claim 22, wherein the processing apparatus is operable to provide electronic mail functionality.

44. The motion control system according to claim 22, wherein the processing apparatus is operable to provide a navigation menu.

45. The motion control system according to claim 22, wherein the processing apparatus is operable to provide time and date information.

46. The motion control system according to claim 22, wherein the processing apparatus further comprises a gesture mapping database.

47. The motion control system according to claim 22, wherein the processing apparatus further comprises a gesture database.

48. The motion control system according to claim 47, wherein the gesture database is operable to record a gesture made by a user.

49. The motion control system according to claim 22, wherein:

the handheld device further comprises a camera; and

the motion detector is operable to detect a tilt and translation motion based on information obtained by the three-axis acceleration sensor and the camera.

50. A method for controlling a motion control system including a handheld device and a processing apparatus, the method comprising:

detecting movement of the handheld device by using a motion detector having a three-axis acceleration sensor;

identifying and selecting a device to be controlled from among a plurality of devices based on the detected movement of the handheld device;

communicating the movement obtained by the motion detector to the processing apparatus through a wireless communication interface of the identified and selected device for processing of the obtained movement by the processing apparatus.