Pen-shaped handwriting input apparatus using accelerometers and gyroscopes and an associated operational device for determining pen movement

• US 6,229,102 B1
• Filed: 08/28/1998
• Issued: 05/08/2001
• Est. Priority Date: 02/20/1996
• Status: Expired due to Term

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Kitaguchi, Takashi, Furuta, Toshiyuki, Shingyouchi, Mitsuru, Fujisawa, Etsuko, Inoue, Takao, Sato, Yasuhiro, Murata, Norihiko
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US09/141,903
• Time in Patent Office: 984 Days
• Field of Search: 345/156, 345/158, 345/169, 345/179, 178/19.01, 178/19.03, 178/19.04, 178/19.05, 382/119, 382/120, 382/181, 382/187, 382/189
• US Class Current: 178/19.01
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction

Abstract

A small-sized pen-shaped input apparatus precisely detects handwriting input. The apparatus compensates for the effects of the inclination of the pen-shaped input apparatus. An initial inclination angle calculating section calculates the initial value of the inclination angle of a pen shaft in a gravity coordinate system. The inclination angle variation calculating section calculates a variation value of the inclination angle of the pen shaft. A handwriting inclination angle calculating section calculates the inclination angle of the pen shaft when writing. A coordinates conversion calculating section converts the coordinate system of the acceleration from the pen shaft coordinate system to the gravity coordinate system. A movement amount calculating section calculates the movement direction and the movement distance of the pen'"'"'s tip end. Finally, a handwriting detecting section detects a state of handwriting or non-handwriting.

Claims

1. A method for determining the movement of a pen-shaped input device comprising the steps of:

(a) detecting, with three acceleration sensors, the acceleration of said input device in the x-axis, y-axis and z-axis direction in an input device coordinate system where an axis of the input device is the Z axis,

(b) detecting, with three gyroscopes, the respective angular velocity around the x-axis, y-axis and z-axis,

(c) detecting whether said input device is in a writing state;

(d) calculating an initial value of an inclination angle of said input device in a gravity coordinate system, having a Z axis extending in a gravity acceleration direction, when said input device is not in a writing state, using the outputs of said acceleration sensors,

(e) calculating a variation in the inclination angle in the gravity coordinate system using the outputs from the three gyroscopes when the input device is in a writing state,

(f) calculating an inclination angle of the input device in the gravity coordinate system using the calculated initial value of the inclination angle and the calculated inclination angle variation,

(g) converting acceleration in the input device coordinate system to acceleration in the gravity coordinate system using the inclination angle in the gravity coordinate system calculated in step (f),

(h) calculating the movement direction and movement distance of a tip end of said input device using the acceleration converted in step (g).

2. The method of claim 1, further comprising the step of:

(i) compensating the movement distance of said tip end as calculated in step (h) to the movement distance on a writing surface using an inclination formed between said writing surface and said gravity coordinate system.

3. A method as in claim 1, further comprising the steps of:

(i) filtering the output signals from said acceleration sensors and gyroscopes to pass signals having a frequency component associated with a writing operation using said input device; and

(j) determining when said input device begins or ends a writing operation based on the presence or absence of filtered signals having said frequency component.

4. A method as in claim 1, further comprising the step of:

(i) calculating the value of an acceleration variation caused by the inclination angle variation using the variation of the inclination angle in the gravity coordinate system of said input device as detected by the calculation of the inclination angle variation and the mounting locations of said acceleration sensors and

(j) compensating the acceleration output based on the calculated value of acceleration variation,

(k) in the acceleration conversion of step (g). performing acceleration conversion with the compensated acceleration outputs from step (j).

5. A method as in claim 4 wherein step (i) further comprises:

(l) calculating the centrifugal force caused by the inclination angle variation on the basis of the variation velocity of the inclination angle in the gravity coordinate system, said step (i) acceleration variation being calculated on the basis of the calculated centrifugal force.

6. A method as in claim 1, further comprising the step of:

(i) compensating for the difference between the coordinates of said acceleration sensors on said writing surface and the coordinates of the tip end on the basis of the inclination angle in the gravity coordinate system and the mounting position of the acceleration sensors.

7. A method for determining the movement of a tip of a pen-shaped input device comprising the steps of:

(a) detecting an inclination angle of said input device in a coordinate system which has a writing surface as a reference surface;

(b) detecting with three acceleration sensors the acceleration of said input device in an input device coordinate system which has an axis of said input device as the Z axis, the detected acceleration being in the X-axis, Y-axis and Z-axis directions,

(c) converting acceleration in the input device coordinate system detected by the acceleration sensors into acceleration in the coordinate system which has the writing surface as a reference surface, using the inclination angle detected in step (a), and

(d) calculating a movement direction and movement distance at said tip end using the acceleration as converted in step (c).

8. A method for determining the movement of a tip end of a pen-shaped input device comprising the steps of:

(a) detecting, with acceleration sensors, acceleration of said input device in the Xs axis, Ys axis, and Zs axis directions in an input device coordinate system having a longitudinal axis of the input device as the Zs axis,

(b) detecting, with three gyroscopes, the rotational angular velocities of said input device around the Xs, Ys and Zs axes,

(c) calculating an initial value of an inclination angle of said input device in a gravity coordinate system having Xg, Yg and Zg axes, with the Zg axis being oriented in the gravity acceleration direction, using the acceleration detected by said acceleration sensors when said input device is in a non-writing state,

(d) calculating a variation of the inclination angle in the gravity coordinate system using the rotational angle velocities detected by said gyroscopes when said input device is in a writing state,

(e) calculating the inclination angle in the gravity coordinate system of said input device, using the initial value of the inclination angle calculated in step (c) and the inclination angle variation calculated in step (d),

(f) compensating the detected acceleration to more accurately portray acceleration at a tip end of said input device using the mounting position of said acceleration sensors, the rotational angular velocities detected by said three gyroscopes in step (b), the inclination angle variation calculated in step (d), and the inclination angle calculated in step (e),

(g) converting the compensated acceleration obtained in step (f) to acceleration in the gravity coordinate system, using the inclination angle calculated in step (e), and

(h) calculating a movement direction and movement distance using the converted acceleration obtained in step (g).

9. A method as in claim 8, wherein said acceleration sensor for the Xs-axis direction is disposed on the position of Ys=0, said acceleration sensor for the Ys-axis direction is disposed on the position of Xs=0, and said acceleration sensor for the Zs-axis direction is disposed on the Zs axis.

10. A method as in claim 8, wherein said respective acceleration sensors are arranged near the Zs axis.

11. A method as in claim 8, further comprising the steps of:

(i) filtering the output signals from said three acceleration sensors and three gyroscopes to detect, in any of said output signals, a signal which is generated when said input device is in a writing mode and caused by the friction between a tip end of said input device and a writing surface, and

(j) determining that said input device is in a handwriting state when said signal generated by friction is detected, and that said input device is no longer in a handwriting state when said signal generated by friction is not detected for a threshold period of time.

12. A method as in claim 11, further comprising the steps of:

(i) extracting the orbit of the tip end of said input device from the beginning of a handwriting state to the end thereof in accordance with the movement direction and movement distance calculated in step (h), and

(j) transferring an image of the orbit of said tip end extracted in step (i) onto an image writing surface.

13. A method as in claim 12, wherein

step (h) calculates a movement distance in the Xg and Yg directions using the acceleration converted in accordance with step (g), and

step (i) extracts the orbit of the input device'"'"'s tip end from the beginning to the end of a handwriting state in accordance with the movement distance of the input device'"'"'s tip end in the Xg and Yg directions.

14. A method for determining the movement of a pen-shaped input device comprising the steps of:

(a) detecting, with three acceleration sensors, the acceleration of said input device in the x-axis, y-axis and z-axis direction in an input device coordinate system where an axis of the input device is the Z axis,

(b) detecting, with three gyroscopes, the respective angular velocities around the x-axis, y-axis and z-axis,

(c) detecting whether said input device is in a state of handwriting or non-handwriting,

(d) calculating an initial value of a rotational angle of a tip end of said input device in a gravity coordinate system Xg, Yg, Zg having the Zg axis in the direction of gravity acceleration when said tip end contacts a writing surface but said input apparatus is in a state of non-handwriting, said initial value of rotational angle being calculated using the acceleration detected in step (a),

(e) calculating the variation of the rotational angle in the gravity coordinate system of the input device on the basis of the angular velocities detected in step (b) when said input apparatus is detected as being in the state of handwriting,

(f) calculating the rotational angle in the gravity coordinate system of the input device using the initial value of the rotational angle calculated in step (d) and the variation of the rotational angle calculated in step (e),

(g) converting the acceleration in the input device coordinate system detected in step (a) into acceleration in the gravity coordinate system on the basis of the rotational angle calculated in step (f), and

(h) calculating the movement direction and distance of said tip end on the basis of the acceleration as converted in step (g).

15. A method as in claim 14, wherein

the detection in step (c) is made on the basis of the presence or absence of a signal which indicates a writing operation which is contained in one or more of the output signals from said acceleration sensors.

16. A method as in claim 14, wherein the detection in step (c) is made on the basis of whether or not a pressure sensor senses a stress on said tip end caused by contact of said tip end with a writing surface.

17. A method as in any one of claims 14, 15, and 16 wherein step (d) calculates the rotational angle of the input device in the gravity coordinate system several times when said input device is in a state of non-handwriting, and obtains an initial value of the rotational angle of the input device when the tip end is brought into contact with a writing surface by averaging the results of the calculations.

18. A method as in claim 14 further comprising the steps of:

(i) detecting the variation amount of the detected acceleration when said input device is in a state of non-handwriting,

(j) comparing the variation amount detected in step (i) with a predetermined threshold value, and

(k) issuing an alarm signal when said comparison in step (j) indicates that said variation amount exceeds said threshold.

19. A method as in claim 17 further comprising the steps of:

(i) detecting the variation amount of the detected acceleration when said input device is in a state of non-handwriting,

(j) comparing the variation amount detected in step (i) with a predetermined threshold value, and

(k) issuing an alarm signal when said comparison in step (j) indicates that said variation amount exceeds said threshold.

20. A method as in claim 18 wherein in step (d) the rotational angle of the input device in the gravity coordinate system is calculated several times using the detected acceleration when said input device is in the state of non-handwriting, excluding from calculation any acceleration which produces in step (j) an indication that said threshold has been exceeded.

21. A method as in claim 20 wherein in step (d) the rotational angle of the input device in the gravity coordinate system is calculated several times using the detected acceleration when said input device is in the state of non-handwriting, excluding from calculation any acceleration which produces in step (j) an indication that said threshold has been exceeded.

27. A method as in claim 14 further comprising the step of:

(i) calculating the value of an acceleration variation caused by the inclination angle variation on the basis of the variation of the inclination angle in the gravity coordinate system of the input device calculated in step (e) and compensating the acceleration detected by said acceleration sensors, based on the acceleration variation calculated in step (i),

(j) performing step (g) using the compensated acceleration.

28. A method as in claim 27, wherein step (i) is carried out by the steps of:

(k) calculating centrifugal force on the basis of the variation velocity of the inclination angle in the gravity coordinate system,

(l) calculating a value of an acceleration variation caused by the inclination angle variation on the basis of the calculated centrifugal force, and

(m) compensating the acceleration detected by the acceleration sensors based on the calculated acceleration variation.

29. A method as in claim 14, further comprising the step of:

(i) compensating the difference between the coordinates of the acceleration sensor on a writing surface and the coordinates of the input device tip end on the basis of the inclination angle in the gravity coordinate system of the input device and the mounting position of the acceleration sensors.

22. A method for determining the movement of a pen-shaped input device having a tip end comprising the steps of:

(a) detecting, with three acceleration sensors, the acceleration of the input device in an input device coordinate system,

(b) detecting, with three gyroscopes, the angular velocity of said input device about the axes in said input device coordinate system,

(c) determining the movement direction and movement distance of said tip end in a gravity coordinate system based on the accelerations and angular velocities detected in steps (a) and (b).

23. A method as in claim 22, further comprising the step of:

(d) compensating the movement distance of said tip end on the basis of an inclination formed between a writing surface and said gravity coordinate system.

24. A method as in claim 22, further comprising the steps of:

(d) detecting when said input-device is in a state of handwriting and non-handwriting,

(e) calculating an initial inclination angle of said input device relative to said gravity coordinate system when said input device is in a state of non-handwriting,

(f) using said initial inclination angle in the determination of the movement direction and movement distance in step (c).

25. A method as in claim 24, wherein step (d) further comprises the steps of:

(g) filtering the output signals from at least said acceleration sensors for a signal having a characteristic which is associated with the movement of said input device across a writing surface, and

(h) using the presence or absence of said characteristic signal to determine when said input device is in a state of handwriting or non-handwriting.

26. A method as in claim 25, wherein step (g) filters the output signals from said acceleration sensors and gyroscopes.

30. A method as in claim 22 wherein step (c) further comprises the steps of:

(d) converting acceleration obtained in step (a) to acceleration in the gravity coordinate system, and

(e) using the converted acceleration obtained in step (d) in determining the movement direction and movement distance.

31. A method for determining the movement of a pen-shaped input device having a tip end and a longitudinal axis comprising the steps of:

(a) detecting, with three acceleration sensors, the acceleration of the input device in an input device coordinate system,

(b) detecting, with three gyroscopes, the angular velocity of said input device about the axes in said input device coordinate system,

(c) calculating an initial value of an inclination angle of said input device in a gravity coordinate system Xg, Yg, Zg having an axis Zg extending in the gravity acceleration direction using the acceleration detected in step (a),

(d) calculating a variation of the inclination angle in the gravity coordinate system on the basis of the angular velocities detected in step (b),

(e) calculating a handwriting inclination angle in the gravity coordinate system of said input device on the basis of the initial value of the inclination angle calculated in step (c) and the inclination angle variation calculated in step (d),

(f) compensating the acceleration obtained from the acceleration sensors in step (a) tip based on the mounting position of the acceleration sensors relative to the tip end, the rotational angular velocities detected by the gyroscopes in step (b), the inclination angle variation calculated in step (d), and the inclination angle of the input device calculated in step (e),

(g) converting the acceleration in the input device coordinate system, as compensated in step (f), to accelerations in the gravity coordinate system on the basis of the inclination angle of the input device in the gravity coordinate system calculated in step (e), and

(h) calculating the movement direction and movement distance of the tip end of said input device based on the acceleration converted in step (g).

32. A method for-determining the movement of a pen-shaped input device having a tip end and a longitudinal axis comprising the steps of:

(a) detecting acceleration of said input device in an input device coordinate system Xs, Ys, and Zs having the longitudinal axis of said input device as the Zs axis,

(b) detecting the rotational angular velocities of said input device around the Xs, Ys and Zs axes,

(c) detecting whether said input device is in a state of handwriting or non-handwriting by detecting whether the tip end of said input device is brought into contact with a writing surface,

(d) calculating an initial rotation angle of said input device in a gravity coordinate system (Xg, Yg, Zg) having the Zg axis extending in a gravity acceleration direction, when a state of non-handwriting is detected, and on the basis of the acceleration detected in step (a).

33. A method for determining the movement of a pen-shaped input device having a tip end and a longitudinal axis comprising the steps of:

(a) detecting acceleration of said input device in an input device coordinate system (Xs, Ys, Zs) having the longitudinal axis of said input device as the Zs axis,

(b) detecting the rotational angular velocities of said input device around the Xs, Ys and Zs axes,

(c) detecting a state of handwriting or non-handwriting by detecting whether said tip end is brought into contact with a writing surface,

(d) calculating the initial value of the rotation angle of the input device in a gravity coordinate system (Xg, Yg, Zg) having an axis extending in the gravity acceleration direction as the Zg axis when the input device tip end is brought into contact with the writing surface on the basis of the acceleration detected in step (a), and when step (c) detects that said input device is in a state of non-handwriting,

(e) calculating the variation of the rotational angle in the gravity coordinate system (Xg, Yg, Zg) of the input device on the basis of the rotational angles detected in step (b) when step (c) detects a state of handwriting,

(f) calculating the rotational angle in the gravity coordinate system (Xg, Yg, Zg) of the input device on the basis of the initial value of the rotational angle calculated in step (d) and the variation in rotational angle calculated in step (e).

34. A method for determining the movement of a pen-shaped input device having a tip end and a longitudinal axis comprising the steps of:

(a) detecting acceleration of the input device shaft in a coordinate system (X, Y, Z) having the pen shaft as the Z axis,

(b) detecting the angular velocities of the input device around the X, Y and Z axes,

(c) determining the movement of said input device, including determining whether said input device is in a state of handwriting or not, said determination of whether said input device is in a state of handwriting or not being determined by the further steps of;

(d) filtering a characteristic frequency component from output signals from the acceleration sensors used in the step (a) detection, and the gyroscopes used in the step (b) detection, said characteristic frequency component being present in at least one of said output signals when said input device is in a state of handwriting, and

(e) determining that said input device is in the state of handwriting when said characteristic frequency component is present in any of said filtered output signals.

35. A method as in claim 34, wherein said characteristic frequency component is in the neighborhood of 10 Hz.

36. A method as in claim 34, wherein said characteristic frequency component is caused by the friction between said tip end of said input device and a writing surface.

37. A method as in claim 34, wherein step (e) determines the beginning of a handwriting operation when said characteristic frequency component first appears in one of said output signals, and determines the end of a handwriting operation when said characteristic frequency component is absent from all of said output signals.

38. A method for determining the movement of a en-shaped input device having a tip end and a longitudinal axis comprising the steps of:

(a) detecting, with three acceleration sensors, the acceleration of the input device in an input device coordinate system,

(b) detecting, with three gyroscopes, the angular velocity of said input device about the axes in said input device coordinate system,

(c) calculating an initial value of the inclination angle of the input device in a gravity coordinate system having an axis extending in the gravity acceleration direction as the Z axis, on the basis of acceleration detected in step (a), when said input device is in a state of non-handwriting,

(d) calculating a variation of the inclination angle in the gravity coordinate system on the basis of the angular velocity detected in step (b) when the input device is in a state of handwriting,

(e) calculating the inclination angle in the gravity coordinate system of the input device, on the basis of the initial value of the inclination angle calculated in step (c) and the inclination angle variation calculated in step (d),

(f) converting the acceleration detected in step (a) to acceleration in the gravity coordinate system on the basis of the inclination angle in the gravity coordinate system calculated in step (e), and

(g) calculating a movement direction and movement distance at the tip end of the input device on the basis of the acceleration converted in step (f).