Dynamic adaptation of gestures for motion controlled handheld devices

• US 7,301,526 B2
• Filed: 03/23/2004
• Issued: 11/27/2007
• Est. Priority Date: 03/23/2004
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Marvit, David L., Reinhardt, Albert H. M.
• Primary Examiner(s): Liang; Regina
• Application Number:
  US10/807,563
• Publication Number:
  US 20050212754A1
• Time in Patent Office: 1,344 Days
• Field of Search: 345156-158, 715856-864
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A motion controlled handheld device includes a display having a viewable surface and operable to generate an image and a gesture database maintaining a plurality of gestures. Each gesture is defined by a motion of the device with respect to a first position of the device. The gesture database comprises a first gesture set and a second gesture set. The device includes a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface. The device also includes a control module operable to track movement of the handheld device using the motion detection module, compare the tracked movement against the gestures in the first gesture set to determine matching gestures, monitor user precision in indicating the matching gestures, determine that the user precision exceeds a precision threshold, prompt the user to enable the second gesture set, and in response to an appropriate user command, enable the second gesture set such that the controller compares subsequent motion of the device against the gestures in the second gesture set.

Claims

1. A motion controlled handheld device comprising:

a display having a viewable surface and operable to generate an image;

a gesture database maintaining a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device, the gesture database comprising a first gesture set and a second gesture set;

a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface; and

a control module operable to;

track movement of the handheld device using the motion detection module;

compare the tracked movement against the gestures in the first gesture set to determine matching gestures;

monitor user precision in indicating the matching gestures;

determine that the user precision exceeds a precision threshold;

prompt the user to enable the second gesture set;

in response to an appropriate user command, enable the second gesture set such that the controller compares subsequent motion of the device against the gestures in the second gesture set;

monitor matches between a user input gesture and a matching one of the gestures in the gesture database;

determine relatively consistent differences between the user input gesture and the matching gesture; and

modify the matching gesture in the gesture database based on the user input gesture.

2. The motion controlled handheld device of claim 1, wherein the second gesture set maintains less distinction between gestures as compared to the first gesture set.

3. The motion controlled handheld device of claim 1, wherein the second gesture set comprises more gestures than the first gesture set.

4. The motion controlled handheld device of claim 1, wherein the gesture database further defines each of the gestures using a sequence of accelerations.

5. The motion controlled handheld device of claim 1, wherein the appropriate user command is a selected one of movement of the device matching to a particular gesture and a non-motion user input.

6. The motion controlled handheld device of claim 1, further comprising:

a first accelerometer operable to detect acceleration along a first axis;

a second accelerometer operable to detect acceleration along a second axis, the second axis perpendicular to the first axis; and

a third accelerometer operable to detect acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis; and

wherein;

the gesture database further defines each of the gestures using a sequence of accelerations;

the motion detection module is further operable to detect motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

the control module is further operable to match the accelerations measured by the motion detection module against gesture definitions in the gesture database to identify particular ones of the gestures.

7. A method for controlling a handheld device comprising:

generating an image on a viewable surface of the handheld device;

maintaining a gesture database comprising a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device, the gesture database comprising a first gesture set and a second gesture set;

tracking movement of the handheld device in relation to the viewable surface;

comparing the tracked movement against the gestures in the first gesture set to determine matching gestures;

monitoring user precision in indicating the matching gestures;

determining that the user precision exceeds a precision threshold;

prompting the user to enable the second gesture set;

in response to an appropriate user command, enabling the second gesture set such that the controller compares subsequent motion of the device against the gestures in the second gesture set;

monitoring matches between a user input gesture and a matching one of the gestures in the gesture database;

determining relatively consistent differences between the user input gesture and the matching gesture; and

modifying the matching gesture in the gesture database based on the user input gesture.

8. The method of claim 7, wherein the second gesture set maintains less distinction between gestures as compared to the first gesture set.

9. The method of claim 7, wherein the second gesture set comprises more gestures than the first gesture set.

10. The method of claim 7, wherein the gesture database further defines each of the gestures using a sequence of accelerations.

11. The method of claim 7, wherein the appropriate user command is a selected one of movement of the device matching to a particular gesture and a non-motion user input.

12. The method of claim 7, wherein the gesture database further defines each of the gestures using a sequence of accelerations;

the method further comprising;

detecting acceleration along a first axis;

detecting acceleration along a second axis, the second axis perpendicular to the first axis; and

detecting acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis;

detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the gestures.

13. Logic for controlling a handheld device, the logic embodied as a computer program stored on a computer readable medium and operable when executed to perform the steps of:

generating an image on a viewable surface of the handheld device;

maintaining a gesture database comprising a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device, the gesture database comprising a first gesture set and a second gesture set;

tracking movement of the handheld device in relation to the viewable surface;

comparing the tracked movement against the gestures in the first gesture set to determine matching gestures;

monitoring user precision in indicating the matching gestures;

determining that the user precision exceeds a precision threshold;

prompting the user to enable the second gesture set;

in response to an appropriate user command, enabling the second gesture set such that the controller compares subsequent motion of the device against the gestures in the second gesture set;

monitoring matches between a user input gesture and a matching one of the gestures in the gesture database;

determining relatively consistent differences between the user input gesture and the matching gesture; and

modifying the matching gesture in the gesture database based on the user input gesture.

14. The logic of claim 13, wherein the second gesture set maintains less distinction between gestures as compared to the first gesture set.

15. The logic of claim 13, wherein the second gesture set comprises more gestures than the first gesture set.

16. The logic of claim 13, wherein the gesture database further defines each of the gestures using a sequence of accelerations.

17. The logic of claim 13, wherein the gesture database further defines each of the gestures using a sequence of accelerations;

the logic further operable when executed to perform the steps of;

detecting acceleration along a first axis;

detecting acceleration along a second axis, the second axis perpendicular to the first axis; and

detecting acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis;

detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the gestures.

18. A motion controlled handheld device comprising:

means for generating an image on a viewable surface of the handheld device;

means for maintaining a gesture database comprising a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device, the gesture database comprising a first gesture set and a second gesture set;

means for tracking movement of the handheld device in relation to the viewable surface;

means for comparing the tracked movement against the gestures in the first gesture set to determine matching gestures;

means for monitoring user precision in indicating the matching gestures;

means for determining that the user precision exceeds a precision threshold;

means for prompting the user to enable the second gesture set;

means for in response to an appropriate user command, enabling the second gesture set such that the controller compares subsequent motion of the device against the gestures in the second gesture set;

means for monitoring matches between a user input gesture and a matching one of the gestures in the gesture database;

means for determining relatively consistent differences between the user input gesture and the matching gesture; and

means for modifying the matching gesture in the gesture database based on the user input gesture.