Method and apparatus for detecting and interpreting path of designated position

• US 20020036618A1
• Filed: 01/26/2001
• Published: 03/28/2002
• Est. Priority Date: 01/31/2000
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Wakai, Masanori, Kayano, Satomi
• Granted Patent: US 7,138,983 B2
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method of detecting and interpreting a path of designated positions is disclosed. The method concurrently detects a plurality of designated positions on a touch panel, again detects the plurality of designated positions subsequent to a travel of the designated positions, determines the distances between each of the current designated positions and the respective immediately preceding designated positions, treats an immediately preceding designated position, closest to the current designated position of interest, as the immediately preceding designated position of the current designated position of interest, and acquires the path of each designated position, thereby recognizing the paths of the plurality of designated positions that move concurrently. A combination of the paths of the plurality of designated positions is interpreted to identify a designation input by a user, and an operation thus designated is executed.

Claims

1. A position information processing apparatus for processing position information comprising:

a designated position detector means for concurrently detecting a plurality of designated positions;

a designated position storage means for storing the plurality of designated positions detected by the designated position detector means; and

a travel path recognizer means for recognizing the travel paths of the plurality of designated positions based on the plurality of preceding designated positions stored in the designated position storage means and the plurality of current designated positions detected by the designated position detector means.

2. A position information processing apparatus according to claim 1, wherein the travel path recognizer means recognizes the travel paths of the plurality of designated positions by treating one of the plurality of preceding designated positions, closest to each of the plurality of current designated positions, as the preceding designated position of the current designated position.

3. A position information processing apparatus according to claim 1, further comprising a designated-area detector means for detecting an area of a designated position, and

a designated-area storage means for storing an area detected by the designated-area detector means,

wherein the travel path recognizer means recognizes the travel paths of the plurality of designated positions by treating one of the plurality of preceding designated positions having an area, closest to the area of each of the plurality of current designated positions, as the preceding designated position of the current designated position.

4. A position information processing apparatus according to claim 1, wherein the designated position detector means is a touch-panel-type detector means.

5. A position information processing apparatus according to claim 1, wherein the designated position detector means comprises:

an image-pickup means for picking up a scene in which an operator designates a position; and

a designated-position recognizer means for recognizing the designated position from the image of the scene picked up by the image-pickup means.

6. A position information processing apparatus according to claim 1, wherein the designated position detector means detects the position of a finger tip of an operator.

7. An operation apparatus comprising:

a path detector means for detecting paths of a plurality of concurrently moving designated positions;

a designation interpreting means for interpreting a designation represented by a combination of the paths of the plurality of designated positions detected by the path detector means; and

an operation means for performing an operation based on the designation interpreted by the designation interpreting means.

8. An operation apparatus according to claim 7, wherein the designation interpreting means comprises:

a distance measuring means for measuring a distance between a plurality of designated positions;

a distance-change acquisition means for acquiring a change in the distance measured by the distance measuring means; and

a distance-change interpreting means for interpreting the designation based on the distance change acquired by the distance-change acquisition means.

9. An operation apparatus according to claim 8, wherein the distance-change interpreting means interprets the designation as a contraction operation when the acquired distance change decreases.

10. An operation apparatus according to claim 8, wherein the distance-change interpreting means interprets the designation as an expansion operation when the acquired distance change increases.

11. An operation apparatus according to claim 8, wherein the distance-change acquisition means acquires an amount of change in the measured distance, and the distance-change interpreting means interprets the designation as a command for one of a contraction operation and an expansion operation at a magnification corresponding to the acquired amount of change.

12. An operation apparatus according to claim 7, wherein the designation interpreting means comprises:

an angle measuring means for measuring an angle made between a reference line and a line that connects the plurality of designated positions;

an angle-change acquisition means for acquiring a change in the angle measured by the angle measuring means; and

an angle-change interpreting means for interpreting the designation based on the angle change acquired by the angle-change acquisition means.

13. An operation apparatus according to claim 12, wherein the angle-change interpreting means interprets the designation as a command for a clockwise rotation operation when the acquired angle change is in a clockwise direction.

14. An operation apparatus according to claim 12, wherein the angle-change interpreting means interprets the designation as a command for a counterclockwise rotation operation when the acquired angle change is in a counterclockwise direction.

15. An operation apparatus according to claim 12, wherein the angle-change acquisition means acquires a change in the measured angle, and the angle-change interpreting means interprets the designation as a command for a rotation in an amount of rotation corresponding to the acquired amount of change.

16. An operation apparatus according to claim 7, wherein the designation interpreting means comprises:

a designated fixed position detector means for detecting a designed fixed position based on the paths of the plurality of designated positions; and

a designated fixed-position-based designation interpreting means for interpreting the designation based on the designated fixed position detected by the designated fixed-position detector means and the paths of the designated positions other than the designated fixed position.

17. An operation apparatus according to claim 16, wherein the designated fixed-position-based designation interpreting means comprises:

a travel-direction acquisition means for acquiring the direction of travel of the designated position other than the designated fixed position; and

a travel-direction interpreting means for interpreting the designation based on the direction of travel acquired by the travel-direction acquisition means.

18. An operation apparatus according to claim 17, wherein the travel-direction interpreting means interprets the designation as a command for one of a next-item operation, a next-page, a next-screen operation, a last-line operation, a leftward screen shifting operation, an expansion operation in a lateral direction only, and a contraction operation in a lateral direction only, when the acquired direction of travel is leftward.

19. An operation apparatus according to claim 17, wherein the travel-direction interpreting means interprets the designation as a command for one of a next-item operation, a next-page operation, a next-screen operation, a last-line operation, an upward screen shifting operation, an expansion operation in a vertical direction only, and a contraction operation in a vertical direction only, when the acquired direction of travel is upward.

20. An operation apparatus according to claim 17, wherein the travel-direction interpreting means interprets the designation as a command for one of a preceding-item operation, a preceding-page operation, a preceding-screen operation, a first-line operation, a downward screen shifting operation, an expansion operation in a vertical direction only, and a contraction operation in a vertical direction only, when the acquired direction of travel is downward.

21. An operation apparatus according to claim 17, wherein the travel-direction interpreting means interprets the designation as a command for one of a preceding-item operation, a preceding-page operation, a preceding-screen operation, a first-line operation, a rightward screen shifting operation, an expansion operation in a lateral direction only, and a contraction operation in a lateral direction only, when the acquired direction of travel is rightward.

22. An operation apparatus according to claim 17, wherein the travel-direction acquisition means further acquires a distance of travel of the designated position, and interprets the designation as a command for an operation of an operational amount responsive to the acquired distance of travel.

23. An operation apparatus according to claim 16, wherein the designated fixed-position-based designation interpreting means comprises:

a fixed-to-fixed position distance measuring means for measuring the distance between the designated fixed position and the designated positions other than the designated fixed position; and

a distance-change acquisition means for acquiring a change in the distance measured by the fixed-to-fixed-position distance measuring means; and

a distance-change interpreting means for interpreting the designation based on the distance change acquired by the distance-change acquisition means.

24. An operation apparatus according to claim 23, wherein the distance-change interpreting means interprets the designation as a command for one of a contraction operation about the designated fixed position and a screen shifting operation in the direction of travel when the distance change acquired by the fixed-to-fixed-position distance measuring means decreases.

25. An operation apparatus according to claim 23, wherein the distance-change interpreting means interprets the designation as a command for an expansion operation about the designated fixed position or a screen shifting operation in the direction of travel when the distance change acquired by the fixed-to-fixed-position distance measuring means increases.

26. An operation apparatus according to claim 23, wherein there are a plurality of designated positions other than the designated fixed positions, and wherein the distance-change interpreting means interprets the designation based on the change in the distance between the designated fixed position and each of the plurality of designated positions.

27. An operation apparatus according to claim 23, wherein the distance-change acquisition means acquires the amount of change in the measured distance and wherein the distance-change interpreting means interprets the designation as a command for one of a contraction operation and an expansion operation, at a magnification responsive to the acquired amount of change.

28. An operation apparatus according to claim 16, wherein the designated fixed-position-based designation interpreting means comprises:

a moving-to-moving-position distance measuring means for measuring the distance between the designated positions other than the designated fixed position;

a moving-to-moving-position distance change acquisition means for acquiring a change in the distance measured by the moving-to-moving-position measuring means; and

a distance-change interpreting means for interpreting the designation based on the distance change acquired by the moving-to-moving-position change acquisition means.

29. An operation apparatus according to claim 28, wherein the distance-change interpreting means interprets the designation as a command for a contraction operation about the designated moving position when the acquired distance change decreases.

30. An operation apparatus according to claim 28, wherein the distance-change interpreting means interprets the designation as a command for an expansion operation about the designated moving position when the acquired distance change increases.

31. An operation apparatus according to claim 28, wherein the moving-to-moving-position distance change acquisition means acquires the amount of change in the distance between the designated positions, and

wherein the distance-change interpreting means interprets the designation as a command for one of a contraction operation and a expansion operation, at a magnification responsive to the acquired amount of change.

32. An operation apparatus according to claim 28, wherein the distance-change interpreting means interprets the designation based on the change in each of the distances between at least three designated positions.

33. An operation apparatus according to claim 16, wherein the designated fixed-position-based designation interpreting means comprises:

an angle measuring means for measuring an angle made between a reference line and a line that connects the designated fixed position and the designated position other than the designated fixed position;

an angle-change acquisition means for acquiring a change in the angle measured by the angle measuring means; and

an angle-change interpreting means for interpreting the designation based on the angle change acquired by the angle-change acquisition means.

34. An operation apparatus according to claim 33, wherein the angle-change interpreting means interprets the designation as a command for a clockwise rotation operation rotating about the designated fixed position or for a clockwise modification operation with the designated fixed position kept stationary when the acquired angle change is in a clockwise direction.

35. An operation apparatus according to claim 33, wherein the angle-change interpreting means interprets the designation as a command for a counterclockwise rotation operation rotating about the designated fixed position or for a counterclockwise modification operation with the designated fixed position kept stationary when the acquired angle change is in a counterclockwise direction.

36. An operation apparatus according to claim 33, wherein the angle-change acquisition means acquires the amount of change in the measured angle, and wherein the angle-change interpreting means interprets the designation as a command for a rotation operation in a rotation direction about the designated fixed position in an amount of rotation responsive to the amount of change or for a modification operation in the rotation direction with the designated fixed position kept stationary.

37. An operation apparatus according to claim 16, wherein the designated fixed-position-based designation interpreting means comprises:

an angle measuring means for measuring an angle made between a reference line and a line that connects the designated positions other than the designated fixed position;

an angle-change acquisition means for acquiring a change in the angle measured by the angle measuring means; and

an angle-change interpreting means for interpreting the designation based on the angle change acquired by the angle-change acquisition means.

38. An operation apparatus according to claim 37, wherein the angle-change interpreting means interprets the designation as a command for a clockwise rotation operation rotating about the designated fixed position or for a clockwise modification operation about the center of gravity of a designated moving position with the designated moving position kept stationary when the acquired angle change is in a clockwise direction.

39. An operation apparatus according to claim 37, wherein the angle-change interpreting means interprets the designation as a command for a counterclockwise rotation operation rotating about the designated fixed position or for a counterclockwise modification operation about the center of gravity of designated moving positions with the designated moving position kept stationary when the acquired angle change is in a counterclockwise direction.

40. An operation apparatus according to claim 37, wherein the angle-change acquisition means acquires the amount of change in the measured angle, and wherein the angle-change interpreting means interprets the designation as a command for a rotation operation in a rotation direction about the designated fixed position in an amount of rotation responsive to the amount of change or for a modification operation in the rotation direction about the center of gravity of designated moving positions with the designated fixed position kept stationary.

41. An operation apparatus according to claim 7, wherein the designation interpreting means comprises:

a positional-relationship determining means for determining a positional relationship between designated positions; and

a positional-relationship-based designation interpreting means for interpreting the designation based on the positional relationship detected by the positional relationship determined by the positional-relationship determining means.

42. An operation apparatus according to claim 41, wherein the positional-relationship-based designation interpreting means comprises a travel area acquisition means for acquiring the area of travel of the designated position, and interprets the designation as a command for an operation within the acquired area of travel.

43. An operation apparatus according to claim 41, wherein the positional-relationship-based designation interpreting means interprets the designation as a command for a deleting operation or a cutting operation when the positional relationship determined by the positional-relationship determining means is in a vertical relationship.

44. An operation apparatus according to claim 41, wherein the positional-relationship-based designation interpreting means interprets the designation as a command for a copying operation when the positional relationship determined by the positional-relationship determining means is in a lateral relationship.

45. An operation apparatus according to claim 41, wherein the positional-relationship determining means comprises an angle measuring means for measuring the angle made between a reference line and a line that connects a plurality of designated positions, and determines the positional relationship based on the measured angle.

46. An operation apparatus according to claim 41, wherein the positional-relationship determining means determines the positional relationship to be in a vertical relationship when the angle measured by the angle measuring means falls within a predetermined range with respect to a vertical direction.

47. An operation apparatus according to claim 41, wherein the positional-relationship determining means determines the positional relationship to be in a lateral relationship when the angle measured by the angle measuring means falls within a predetermined range with respect to a lateral direction.

48. An operation apparatus according to claim 41, wherein the positional-relationship-based designation interpreting means comprises:

a positional-relationship change determining means for determining a change in the determined positional relationship; and

a positional-relationship change interpreting means for interpreting the designation based on the positional-relationship change determined by the positional-relationship change determining means.

49. An operation apparatus according to claim 48, wherein the positional-relationship change interpreting means interprets the designation as a command for one of an inversion operation, a deleting operation, and a cutting operation when the determined positional-relationship change indicates an inversion.

50. An operation apparatus according to claim 48, wherein the positional-relationship change interpreting means interprets the designation as a command for one of a lateral inversion operation, a deleting operation, and a cutting operation when the determined positional-relationship change indicates a lateral inversion.

51. An operation apparatus according to claim 48, wherein the positional-relationship change interpreting means interprets the designation as a command for one of a vertical inversion operation, a deleting operation, and a cutting operation when the determined positional-relationship change indicates a vertical inversion.

52. An operation apparatus according to claim 48, wherein the positional-relationship change interpreting means interprets the designation as a command for one of an expansion operation and a contraction operation when the determined positional-relationship change indicates no change.

53. An operation apparatus according to claim 48, wherein the positional-relationship change determining means comprises:

an angle measuring means for measuring an angle made between a reference line and a line that connects a plurality of designated positions; and

an angle-change acquisition means for acquiring a change in the angle measured by the angle measuring means,

wherein the positional-relationship change determining means determines the positional-relationship change based on the angle change acquired by the angle-change acquisition means.

54. An operation apparatus according to claim 53, wherein the positional-relationship change determining means determines the positional-relationship change to be in an inversion when the angle change acquired by the angle-change acquisition means falls within a predetermined bilaterally symmetrical range.

55. An operation apparatus according to claim 53, wherein the positional-relationship change determining means determines the positional-relationship change to be no change when the angle change acquired by the angle-change acquisition means falls outside a predetermined bilaterally symmetrical range.

56. An operation apparatus according to claim 48, wherein the positional-relationship change determining means comprises:

a lateral positional-relationship determining means for determining that a plurality of designated positions are horizontally aligned; and

a lateral positional-relationship change determining means for determining a change in the lateral positional relationship determined by the lateral positional-relationship determining means,

wherein the positional-relationship change determining means determines the positional relationship based on the change in the determined lateral positional relationship.

57. An operation apparatus according to claim 56, wherein the positional-relationship change determining means determines the positional-relationship change to be a lateral inversion when the change in the lateral positional relationship determined by the lateral positional-relationship determining means is a negative value.

58. An operation apparatus according to claim 56, wherein the positional-relationship change determining means determines the positional-relationship change to be no change when the change in the lateral positional relationship determined by the lateral positional-relationship determining means is a positive value.

59. An operation apparatus according to claim 48, wherein the positional-relationship change determining means comprises:

a vertical positional-relationship determining means for determining that a plurality of designated positions are vertically aligned; and

a vertical positional-relationship change determining means for determining a change in the lateral positional relationship determined by the vertical positional-relationship determining means,

wherein the positional-relationship change determining means determines the positional relationship based on the change in the lateral positional-relationship determined by the vertical positional-relationship change determining means.

60. An operation apparatus according to claim 59, wherein the positional-relationship change determining means determines the positional-relationship change to be a vertical inversion when the change in the vertical positional relationship determined by the vertical positional-relationship determining means is a negative value.

61. An operation apparatus according to claim 59, wherein the positional-relationship change determining means determines the positional-relationship change to be no change when the change in the vertical positional relationship determined by the vertical positional-relationship determining means is a positive value.

62. An operation apparatus according to claim 41, wherein the positional-relationship determining means comprises:

a designated fixed-position detector means for detecting a designated fixed position based on the paths of the plurality of designated positions; and

a designated fixed-to-moving-position positional-relationship determining means for determining the positional relationship between the designated fixed position detected by the designated fixed-position detector means and the designated position other than the designated fixed position.

63. An operation apparatus according to claim 62, wherein the positional-relationship-based designation interpreting means comprises a travel-direction acquisition means for acquiring the direction of travel of the designated position other than the designated fixed position, and wherein the positional-relationship-based designation interpreting means interprets the designation based on the direction of travel acquired by the travel-direction acquisition means and the positional relationship with respect to the designated fixed position.

64. An operation apparatus according to claim 63, wherein the travel-direction acquisition means comprises a designated position travel distance acquisition means for acquiring the distance of travel of the designated position, and

wherein the positional-relationship-based designation interpreting means interprets the designation as a command for an operation having an operational amount responsive to the distance of travel acquired by the designated position travel distance acquisition means.

65. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated position travel-direction acquisition means is leftward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the right of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of a next-item operation, a next-page operation, a next-screen operation, a last-line operation, and a contraction operation in a lateral direction only.

66. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is leftward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the left of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of a leftward screen shifting operation and an expansion operation in a lateral direction only.

67. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is leftward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is above the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a counterclockwise rotation operation.

68. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is leftward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is below the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a clockwise rotation operation.

69. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated position travel-direction acquisition means is upward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the right of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a counterclockwise rotation operation.

70. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is upward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the left of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a clockwise rotation operation.

71. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is upward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is above the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of an upward screen shifting operation and an expansion operation in a vertical direction only.

72. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is upward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is below the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of a next-item operation, a next-page operation, a next-screen operation, a last-line operation, and a contraction operation in a vertical direction only.

73. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated position travel-direction acquisition means is downward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the right of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a clockwise rotation operation.

74. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is downward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is to the left of the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for a counterclockwise rotation operation.

75. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is downward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is above the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of a preceding-item operation, a preceding-page operation, a preceding-screen operation, a first-line operation, and a contraction operation in a vertical direction only.

76. An operation apparatus according to claim 63, wherein when the direction of travel acquired by the designated travel-direction acquisition means is downward, and when the positional relationship determined by the designated fixed-to-moving-position positional-relationship determining means is below the designated fixed position, the positional-relationship-based designation interpreting means interprets the designation as a command for one of a downward screen shifting operation and an expansion