Methods and devices for removing unintentional movement in 3D pointing devices

• US 7,535,456 B2
• Filed: 05/02/2005
• Issued: 05/19/2009
• Est. Priority Date: 04/30/2004
• Status: Active Grant

• Current Assignee: DRNC Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Hillcrest Laboratories Incorporated (CEVA Incorporated)
• Inventors: Liberty, Matthew G., Hunleth, Frank A.
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US11/119,987
• Publication Number:
  US 20060028446A1
• Time in Patent Office: 1,478 Days
• Field of Search: 345156-169, 345173-179, 178 1801- 1809, 178 1901- 1906
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device

Abstract

Systems and methods according to the present invention describe 3D pointing devices and methods which detect movement of the 3D pointing device and remove unintentional movement from the output readings.

Claims

1. A handheld device comprising:

a first sensor for determining motion of said handheld device and generating a first output associated therewith;

an accelerometer for determining an acceleration of said handheld device and outputting an acceleration output associated therewith; and

a processing unit for receiving said first output and said acceleration output and for;

(a) determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

(b) if said velocity of said handheld device is below a predetermined velocity threshold, then discarding or attenuating data generated by said first sensor and said accelerometer until either said handheld device has moved more than a predetermined distance or a predetermined time has expired; and

(c) otherwise, if said velocity of said handheld device is above said predetermined velocity threshold, then filtering said data generated by said first sensor, and said accelerometer to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.

2. The handheld device of claim 1, wherein said processing unit also detects an occurrence of an event and filters said data based on said detected event, wherein said handheld device includes at least one button and said event is a button press or button release action.

3. The handheld device of claim 2, wherein said processing unit filters said data using a first filter if said event is of a first type and using a second filter if said event is of a second type.

4. The handheld device of claim 3, wherein said first type of said event is a button press and said second type of said event is a button release.

5. The handheld device of claim 2, wherein said processing unit filters said data to permit some data generated by said first sensor to be processed immediately subsequent to said detected event.

7. The handheld device of claim 1, wherein said at first sensor includes an accelerometer.

8. The handheld device of claim 1, wherein said at first sensor includes a rotational sensor.

9. The handheld device of claim 1, wherein said at first sensor includes a gyroscope.

10. The handheld device of claim 1, wherein said first sensor includes a magnetometer.

11. The handheld device of claim 1, wherein said first sensor includes an optical sensor.

12. The handheld device of claim 1, wherein said first sensor includes a camera.

13. The handheld device of claim 1, wherein said processing unit uses Kalman filtering to remove said tremor from said data.

14. The handheld device of claim 1, wherein said processing unit uses Kalman smoothing to remove said tremor from said data.

15. The handheld device of claim 1, wherein said processing unit uses space-state estimation to remove said tremor from said data.

16. The handheld device of claim 1, wherein said processing unit uses Expectation-Maximization to remove said tremor from said data.

17. The handheld device of claim 1, wherein said processing unit also processes said data to compensate for an orientation of said handheld device.

18. The handheld device of claim 17, wherein said processing unit compensates said data for said orientation of said handheld device by transforming said data from a first frame of reference to a second frame of reference.

19. The handheld device of claim 1, wherein said processing unit uses said removed tremor to identify a user holding said handheld device.

20. The handheld device of claim 1, wherein said processing unit uses extended Kalman filtering to remove said tremor from said data.

6. The handheld device of 1, wherein said first sensor includes at least one of an accelerometer, a rotational sensor, a gyroscope, a magnetometer, an optical sensor and a camera.

21. A method comprising:

sensing motion of a handheld device and generating a first data associated therewith;

sensing an acceleration of said handheld device and outputting an acceleration output associated therewith;

determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

if said velocity of said handheld device is below a predetermined velocity threshold, then discarding or attenuating data associated with at least one of said first output and said acceleration output, until either said handheld device has moved more than a predetermined distance or a predetermined time has expired; and

otherwise, if said velocity of said handheld device is above said predetermined velocity threshold, then filtering said data to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.

22. The method of claim 21, further comprising:

detecting occurrence of an event and discarding or attenuating said output until either a predetermined time has elapsed after said detection of said event or said handheld device has moved a predetermined distance after said detection of said event, whichever comes first;

wherein said handheld device includes at least one button and said event is a button press or button release action.

23. The method of claim 22, wherein said sensing of motion is performed using at least one of an accelerometer, a rotational sensor, a gyroscope, a magnetometer, an optical sensor and a camera.

24. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by an accelerometer.

25. The handheld method of claim 24, wherein said processing performs Kalman filtering to remove said tremor from said data.

26. The handheld method of claim 24, wherein said processing performs Kalman smoothing to remove said tremor from said data.

27. The handheld method of claim 24, wherein said processing performs space-state estimation to remove said tremor from said data.

28. The handheld method of claim 24, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

29. The handheld method of claim 24, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

30. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by a rotational sensor.

31. The handheld method of claim 30, wherein said processing performs Kalman filtering to remove said tremor from said data.

32. The handheld method of claim 30, wherein said processing performs Kalman smoothing to remove said tremor from said data.

33. The handheld method of claim 30, wherein said processing performs space-state estimation to remove said tremor from said data.

34. The handheld method of claim 30, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

35. The handheld method of claim 30, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

36. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by a gyroscope.

37. The handheld method of claim 36, wherein said processing performs Kalman filtering to remove said tremor from said data.

38. The handheld method of claim 36, wherein said processing performs Kalman smoothing to remove said tremor from said data.

39. The handheld method of claim 36, wherein said processing performs space-state estimation to remove said tremor from said data.

40. The handheld method of claim 36, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

41. The handheld method of claim 36, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

42. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by a magnetometer.

43. The handheld method of claim 42, wherein said processing performs Kalman filtering to remove said tremor from said data.

44. The handheld method of claim 42, wherein said processing performs Kalman smoothing to remove said tremor from said data.

45. The handheld method of claim 42, wherein said processing performs space-state estimation to remove said tremor from said data.

46. The handheld method of claim 42, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

47. The handheld method of claim 42, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

48. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by an optical sensor.

49. The handheld method of claim 48, wherein said processing performs Kalman filtering to remove said tremor from said data.

50. The handheld method of claim 48, wherein said processing performs Kalman smoothing to remove said tremor from said data.

51. The handheld method of claim 48, wherein said processing performs space-state estimation to remove said tremor from said data.

52. The handheld method of claim 48, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

53. The handheld method of claim 48, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

54. The handheld method of claim 21, wherein said sensing of movement is performed, at least in part, by a camera.

55. The handheld method of claim 54, wherein said processing performs Kalman filtering to remove said tremor from said data.

56. The handheld method of claim 54, wherein said processing performs Kalman smoothing to remove said tremor from said data.

57. The handheld method of claim 54, wherein said processing performs space-state estimation to remove said tremor from said data.

58. The handheld method of claim 54, wherein said processing performs Expectation-Maximization to remove said tremor from said data.

59. The handheld method of claim 54, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

60. The handheld method of claim 21, wherein said filtering performs Kalman filtering to remove said tremor from said data.

61. The handheld method of claim 21, wherein said filtering performs Kalman smoothing to remove said tremor from said data.

62. The handheld method of claim 21, wherein said filtering performs space-state estimation to remove said tremor from said data.

63. The handheld method of claim 21, wherein said filtering performs Expectation-Maximization to remove said tremor from said data.

64. The handheld method of claim 21, wherein said processing further comprises:

transforming said data to account for an orientation of said handheld device.

65. The handheld method of claim 64, wherein said transforming further comprises:

transforming said data from a first frame of reference to a second frame of reference.

66. The handheld method of claim 21, further comprising:

identifying a user based on said removed tremor.

67. The handheld method of claim 21, wherein said processing performs extended Kalman filtering to remove said tremor from said data.

135. The handheld method of claim 21, wherein said sensing is performed, at least in part, by at least one sensor disposed within said handheld device.

136. The handheld method of claim 21, wherein said discarding or attenuating and filtering is performed, at least in part, by a processor disposed within said handheld device.

68. A handheld device comprising:

means for sensing motion of a handheld device and generating a first output associated therewith;

means for sensing an acceleration of said handheld device and outputting an acceleration output associated therewith;

means for determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

means for, if said velocity of said handheld device is below a predetermined velocity threshold, discarding or attenuating data associated with at least one of said first output and said acceleration output, until either said handheld device has moved more than a predetermined distance or a predetermined time has expired; and

means for, if said velocity of said handheld device is above said predetermined velocity threshold, filtering said data to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.

69. The handheld device of claim 68, wherein said means for discarding or attenuating is also for detecting occurrence of an event and for discarding or attenuating said output until a predetermined time has elapsed after said detection of said event and further wherein said handheld device includes at least one button and said event is a button press or button release action.

70. The handheld device of claim 69, wherein said means for determining motion includes at least one of an accelerometer, a rotational sensor, a gyroscope, a magnetometer, an optical sensor and a camera.

71. The handheld device of claim 68, wherein said means for sensing includes an accelerometer.

72. The handheld device of claim 68, wherein said means for sensing includes a rotational sensor.

73. The handheld device of claim 68, wherein said means for sensing includes a gyroscope.

74. The handheld device of claim 68, wherein said means for sensing includes a magnetometer.

75. The handheld device of claim 68, wherein said means for sensing includes an optical sensor.

76. The handheld device of claim 68, wherein said means for sensing includes a camera.

77. The handheld device of claim 68, wherein said means for processing uses Kalman filtering to remove said tremor from said data.

78. The handheld device of claim 68, wherein said means for processing uses Kalman smoothing to remove said tremor from said data.

79. The handheld device of claim 68, wherein said means for processing uses space-state estimation to remove said tremor from said data.

80. The handheld device of claim 68, wherein said means for processing uses Expectation-Maximization to remove said tremor from said data.

81. The handheld device of claim 68, wherein said means for processing also processes said data to compensate for an orientation of said handheld device.

82. The handheld device of claim 81, wherein said means for processing compensates said data for said orientation of said handheld device by transforming said data from a first frame of reference to a second frame of reference.

83. The handheld device of claim 68, wherein said means for processing uses said removed tremor to identify a user holding said handheld device.

84. The handheld device of claim 68, wherein said means for processing uses extended Kalman filtering to remove said tremor from said data.

85. A system comprising:

a handheld device including;

a first sensor for sensing motion of said handheld device and generating a first output associated therewith; and

an accelerometer for sensing an acceleration of said handheld device and outputting an acceleration output associated therewith; and

a processing unit for receiving said first output and said acceleration output and for;

(a) determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

(b) if said velocity of said handheld device is below a predetermined velocity threshold, then discarding or attenuating data generated by said first sensor and said accelerometer until either said handheld device has moved more than a predetermined distance or a predetermined time has expired; and

(c) otherwise, if said velocity of said handheld device is above said predetermined velocity threshold, then filtering said data generated by said first sensor, and said accelerometer to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.

86. The system of claim 85, wherein said processing unit is also for detecting occurrence of an event and for adjusting said data based on said event, wherein said handheld device includes at least one button and said event is a button press or button release action.

87. The system of claim 85, wherein said first sensor includes at least one of an accelerometer, a rotational sensor, a gyroscope, a magnetometer, an optical sensor and a camera.

88. The system of claim 85, wherein said at least one motion sensor is an accelerometer.

89. The system of claim 88, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

90. The system of claim 88, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

91. The system of claim 88, wherein said processing unit performs space-state estimation to remove said tremor from said data.

92. The system of claim 88, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

93. The system of claim 88, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

94. The system of claim 85, wherein said at least one motion sensor is a rotational sensor.

95. The system of claim 94, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

96. The system of claim 94, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

97. The system of claim 94, wherein said processing unit performs space-state estimation to remove said tremor from said data.

98. The system of claim 94, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

99. The system of claim 94, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

100. The system of claim 94, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

101. The system of claim 94, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

102. The system of claim 94, wherein said processing unit performs space-state estimation to remove said tremor from said data.

103. The system of claim 94, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

104. The system of claim 94, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

105. The system of claim 85, wherein said at least one motion sensor is a gyroscope.

106. The system of claim 105, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

107. The system of claim 105, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

108. The system of claim 105, wherein said processing unit performs space-state estimation to remove said tremor from said data.

109. The system of claim 105, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

110. The system of claim 105, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

111. The system of claim 85, wherein said at least one motion sensor is a magnetometer.

112. The system of claim 111, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

113. The system of claim 111, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

114. The system of claim 111, wherein said processing unit performs space-state estimation to remove said tremor from said data.

115. The system of claim 111, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

116. The system of claim 111, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

117. The system of claim 85, wherein said at least one motion sensor is an optical sensor.

118. The system of claim 117, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

119. The system of claim 117, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

120. The system of claim 117, wherein said processing unit performs space-state estimation to remove said tremor from said data.

121. The system of claim 117, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

122. The system of claim 117, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

123. The system of claim 85, wherein said at least one motion sensor is a camera.

124. The system of claim 123, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

125. The system of claim 123, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

126. The system of claim 123, wherein said processing unit performs space-state estimation to remove said tremor from said data.

127. The system of claim 123, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

128. The system of claim 123, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

129. The system of claim 85, wherein said processing unit performs Kalman filtering to remove said tremor from said data.

130. The system of claim 85, wherein said processing unit performs Kalman smoothing to remove said tremor from said data.

131. The system of claim 85, wherein said processing unit performs space-state estimation to remove said tremor from said data.

132. The system of claim 85, wherein said processing unit performs Expectation-Maximization to remove said tremor from said data.

133. The system of claim 85, wherein said at least one motion sensor is within said handheld device.

134. The system of claim 85, wherein said processing unit is within said handheld device.

137. The system of claim 85, wherein said processing unit also processes said data to compensate for an orientation of said handheld device.

138. The system of claim 137, wherein said processing unit compensates said data for said orientation of said handheld device by transforming said data from a first frame of reference to a second frame of reference.

139. The system of claim 85, wherein said processing unit uses said removed tremor to identify a user holding said handheld device.

140. The system of claim 139, further comprising:

a system component in communication with said handheld device, wherein at least some of said processing of said tremor is performed within said system component.

141. The system of claim 85, wherein said processing unit performs extended Kalman filtering to remove said tremor from said data.

142. The system of claim 85, further comprising:

a system component in communication with said handheld device, wherein at least some of said processing of said tremor is performed within said system component.

143. A computer-readable medium containing program instructions which, when executed on a computer or a processor, perform the steps of:

determining motion of a handheld device and generating a first output associated therewith;

determining an acceleration of said handheld device and outputting an acceleration output associated therewith;

determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

if said velocity of said handheld device is below a predetermined velocity threshold, then discarding or attenuating data associated with at least one of said first output and said acceleration output, until either said handheld device has moved more than a predetermined distance or a predetermined time has expired; and

otherwise, if said velocity of said handheld device is above said predetermined velocity threshold, then filtering said data to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.

144. A handheld device comprising:

a first sensor for determining motion of said handheld device and generating a first output associated therewith;

an accelerometer for determining an acceleration of said handheld device and outputting an acceleration output associated therewith; and

a processing unit for receiving said first output and said acceleration output and for;

(a) determining a velocity of said handheld device based upon at least one of said first output and said acceleration output;

(b) attenuating data generated by at least one of said first sensor and said accelerometer as a function of at least one of said velocity, said first output and said acceleration; and

(c) filtering said data to remove tremor by adaptive, model-based filtering using one of;

linear filtering, Kalman filtering, Kalman smoothing, extended Kalman filtering, state-space estimation, Expectation-Maximization, or another model-based technique.