Selective engagement of motion input modes

• US 7,903,084 B2
• Filed: 03/23/2004
• Issued: 03/08/2011
• Est. Priority Date: 03/23/2004
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Adler, B. Thomas, Reinhardt, Albert H. M., Marvit, David L.
• Primary Examiner(s): Liang; Regina
• Application Number:
  US10/807,561
• Publication Number:
  US 20050212752A1
• Time in Patent Office: 2,541 Days
• Field of Search: 345156-158, 715/856, 715/857, 715/863, 715/864
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A motion controlled handheld device includes a display having a viewable surface and operable to generate a current image and a gesture database maintaining a plurality of gestures. Each gesture is defined by a motion of the device with respect to a first position of the device. The device includes a gesture mapping database comprising a mapping of each of the gestures to an associated command, a motion detection module operable to detect motion of the device within three dimensions and to identify components of the motion in relation to the viewable surface and a display control module having a first mode of operation and a second mode of operation. The display control module is operable in the first mode of operation to monitor the motion of the device, to determine a location of the device resulting from the motion, and to modify the current image based on the resulting location of the device. The display control module is operable in the second mode of operation to monitor the motion of the device, to track movement of the handheld device using the motion detection module, to compare the tracked movement with the gestures to identify a matching gesture, to identify one of the commands associated with the matching gesture, and to modify the current image based on the identified command. The device also includes a mode selection module operable to detect a mode selection trigger and to switch between the first mode of operation and the second mode of operation in response to detecting the mode selection trigger.

Claims

1. A motion controlled handheld device comprising:

a display having a viewable surface and operable to generate a current image;

a gesture database maintaining a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device;

a gesture mapping database comprising a mapping of each of the gestures to an associated command;

a motion detection module operable to detect motion of the device within three dimensions and to identify components of the motion in relation to the viewable surface;

a display control module having a first mode of motion input operation and a second mode of motion input operation;

the display control module operable in the first mode of motion input_operation to monitor the motion of the device, to determine a location of the device resulting from the motion, and to modify the current image based on the resulting location of the device as compared to an initial location of the device prior to the motion of the device;

the display control module operable in the second mode of motion input operation to monitor the motion of the device, to track movement of the handheld device using the motion detection module, the tracked movement identifying a path traveled by the device, to compare the path with the gestures to identify a matching gesture, to identify one of the commands associated with the matching gesture, and to modify the current image based on the identified command;

a mode selection module operable to detect a mode selection trigger and to switch between the first mode of motion input operation and the second mode of motion input operation in response to detecting the mode selection trigger;

wherein the mode selection trigger comprises a change in a state of the device; and

wherein the change in the state of the device occurs when the device switches from a first application to a second application.

2. The motion controlled handheld device of claim 1, wherein the change in the state of the device occurs when the current image switches from a first image to a second image.

3. The motion controlled handheld device of claim 1, wherein the mode selection module switches from the first mode of motion input operation to the second mode of motion input operation in response to detecting a first mode selection trigger, and the mode selection module switches from the second mode of motion input operation to the first mode of motion input operation in response to detecting a second mode selection trigger different than the first mode selection trigger.

4. The motion controlled handheld device of claim 1, wherein the mode selection trigger comprises one of the gestures.

5. The motion controlled handheld device of claim 1, wherein the mode selection trigger comprises non-motion related input received using a user interface of the device.

6. The motion controlled handheld device of claim 1, wherein:

the display control module has a third mode of operation;

the display control module operable in the third mode of operation to disregard the motion of the device.

7. The motion controlled handheld device of claim 1, further comprising:

a first accelerometer operable to detect acceleration along a first axis;

a second accelerometer operable to detect acceleration along a second axis, the second axis perpendicular to the first axis; and

a third accelerometer operable to detect acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis; and

wherein;

the gesture database further defines each of the gestures using a sequence of accelerations;

the motion detection module is further operable to detect motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

the display control module is further operable, in the second mode of motion input operation, to match the accelerations measured by the motion detection module against gesture definitions in the gesture database to identify particular ones of the gestures.

8. A method for controlling a handheld device comprising:

generating a current image on a viewable surface of the handheld device;

maintaining a gesture database maintaining a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device;

maintaining a gesture mapping database comprising a mapping of each of the gestures to an associated command;

detecting motion of the device within three dimensions;

identifying components of the motion in relation to the viewable surface;

in a first mode of motion input operation, monitoring the motion of the device, determining a location of the device resulting from the motion, and modifying the current image based on the resulting location of the device as compared to an initial location of the device prior to the motion of the device;

in a second mode of motion input operation monitoring the motion of the device, tracking movement of the handheld device, comparing the tracked movement with the gestures to identify a matching gesture, identifying one of the commands associated with the matching gesture, and modifying the current image based on the identified command;

detecting a mode selection trigger;

switching between the first mode of motion input_operation and the second mode of motion input_operation in response to detecting the mode selection trigger;

wherein the mode selection trigger comprises a change in a state of the device; and

wherein the change in the state of the device occurs when the device switches from a first application to a second application.

9. The method of claim 8, wherein the change in the state of the device occurs when the current image switches from a first image to a second image.

10. The method of claim 8, further comprising, in a third mode of operation, disregarding the motion of the device.

11. The method of claim 8, wherein the gesture database further defines each of the gestures using a sequence of accelerations;

the method further comprising;

detecting acceleration along a first axis;

detecting acceleration along a second axis, the second axis perpendicular to the first axis; and

detecting acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis;

detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the gestures.

12. Logic for controlling a handheld device, the logic embodied as a non-transitory computer program stored on a computer readable medium and operable when executed to perform the steps of:

generating a current image on a viewable surface of the handheld device;

maintaining a gesture database maintaining a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device;

maintaining a gesture mapping database comprising a mapping of each of the gestures to an associated command;

detecting motion of the device within three dimensions;

identifying components of the motion in relation to the viewable surface;

in a first mode of motion input operation, monitoring the motion of the device, determining a location of the device resulting from the motion, and modifying the current image based on the resulting location of the device as compared to an initial location of the device prior to the motion of the device;

in a second mode of motion input operation monitoring the motion of the device, tracking movement of the handheld device, comparing the tracked movement with the gestures to identify a matching gesture, identifying one of the commands associated with the matching gesture, and modifying the current image based on the identified command;

detecting a mode selection trigger;

switching between the first mode of motion input operation and the second mode of motion input_operation in response to detecting the mode selection trigger;

wherein the mode selection trigger comprises a change in a state of the device; and

wherein the change in the state of the device occurs when the device switches from a first application to a second application.

13. The logic of claim 12, further operable when executed to perform the step of, in a third mode of operation, disregarding the motion of the device.

14. The logic of claim 12, wherein the gesture database further defines each of the gestures using a sequence of accelerations;

the logic further operable when executed to perform the steps of;

detecting acceleration along a first axis;

detecting acceleration along a second axis, the second axis perpendicular to the first axis; and

detecting acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis;

detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and

matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the gestures.

15. A motion controlled handheld device comprising:

means for generating a current image on a viewable surface of the handheld device;

means for maintaining a gesture database maintaining a plurality of gestures, each gesture defined by a motion of the device with respect to a first position of the device;

means for maintaining a gesture mapping database comprising a mapping of each of the gestures to an associated command;

means for detecting motion of the device within three dimensions;

means for identifying components of the motion in relation to the viewable surface;

means for, in a first mode of motion input operation, monitoring the motion of the device, determining a location of the device resulting from the motion, and modifying the current image based on the resulting location of the device as compared to an initial location of the device prior to the motion of the device;

means for, in a second mode of motion input operation monitoring the motion of the device, tracking movement of the handheld device, comparing the tracked movement with the gestures to identify a matching gesture, identifying one of the commands associated with the matching gesture, and modifying the current image based on the identified command;

means for detecting a mode selection trigger;

means for switching between the first mode of motion input operation and the second mode of motion input operation in response to detecting the mode selection trigger; and

wherein the mode selection trigger comprises a change in a state of the device; and

wherein the change in the state of the device occurs when the device switches from a first application to a second application.