Method and apparatus for selecting information in multi-dimesional space

• US 20040169636A1
• Filed: 01/22/2004
• Published: 09/02/2004
• Est. Priority Date: 07/24/2001
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Tae-Sik, Lee, Sang-Goog, Lee, Kwang-Woon
• Granted Patent: US 7,259,756 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Method and apparatus for selecting information in a multi-dimensional space are provided. The method includes the steps of sensing the motion of a user intending to select information, and determining whether the sensed motion is an effective motion intended for information selection and returning to the sensing step if it is determined that the motion is not the effective one. Accordingly, difficulty in setting a predetermined threshold to a single value can be overcome. In addition, even a very slight motion can be accurately detected without being influenced by gravity and offsets. Moreover, it can be accurately determined whether a user'"'"'s two- or three-dimensional motion is an effective motion for selecting information and which motion unit among at least two motion units has generated the effective motion, so that a high correct recognition rate for information and high reliability can be achieved, and a user is allowed to easily select information.

Claims

1. A method of selecting information in a multi-dimensional space, the method comprising the steps of:

(a) sensing the motion of a user for information selection;

(b) determining, using the amount of the sensed motion, whether the sensed motion is an effective motion of the intended information selection and returning to step (a) if it is determined that the motion is not the effective motion; and

(c) determining whether the amount of the sensed motion exceeds a predetermined amount after step (a) and going to step (b) if it is determined that the amount of the motion does not exceed the predetermined amount.

3. The method of claim 1, further comprising the step of converting the result of sensing from an analog form into a digital one after step (a) and going to step (b), wherein step (b) comprises determining whether the motion is the effective motion based on the result of conversion.

4. The method of claim 1, further comprising the step of (d) if it is determined that the motion is the effetive motion in step (b), determining the information selected by the effective motion as effective information.

5. The method of claim 1, wherein step (a) comprises visually sensing the motion.

6. The method of claim 1, wherein the motion is a three-dimensional motion.

7. The method of claim 1, wherein the motion is a two-dimensional motion.

8. The method of claim 1, wherein the information selection implies pointing the information.

9. The method of claim 1, wherein the information selection denotes inputting the information.

10. The method of claim 4, wherein step (b) comprises the sub-steps of:

(b1) extracting as a feature amount a portion having a change in the size of the sensing result;

(b2) determining whether the feature amount is at least of a predetermined threshold and turning to step (a) if it is determined that the feature amount is less than the predetermined threshold; and

(b3) if it is determined that the feature amount is at least of the predetermined threshold, determining that the motion is the effective motion and going to step (d).

11. The method of claim 10, wherein step (b1) comprises band-pass filtering the result of sensing and determining the result of band-pass filtering as the feature amount.

12. The method of claim 10, wherein step (b1) comprises the sub-steps of:

(b11) filtering a first frequency component and a second frequency component of the result of sensing; and

(b12) obtaining a difference between the first and second filtered frequency components and determining the difference as the feature amount.

13. The method of claim 12, wherein the first and second frequency components are low-band components.

14. The method of claim 12, wherein the first frequency component is a low-band component, and the second frequency component is a high-band component.

15. The method of claim 12, wherein the first frequency component is a low-band component, and the second frequency component is a band component.

16. The method of claim 12, wherein the first frequency component is a high-band component, and the second frequency component is a band component.

17. The method of claim 12, wherein the first and second frequency components are high-band components.

18. The method of claim 12, wherein the first and second frequency components are band components.

19. The method of claim 10, wherein in step (b1), a positive component of the portion having a change in the size of the sensing result is extracted as the feature amount.

20. The method of claim 10, wherein in step (b1), a negative component of the portion having a change in the size of the sensing result is extracted as the feature amount.

21. The method of claim 10, wherein step (b) further comprises the sub-step of (b4) determining whether the feature amount is maintained to be at least of the predetermined threshold for a first predetermined period if it is determined that the feature amount is at least the predetermined threshold, and turning to step (a) if it is determined that the feature amount is not maintained to be at least of the predetermined threshold for the first predetermined period, and

in step (b3), the motion is determined as the effective motion if it is determined that the feature amount is maintained to be at least of the predetermined threshold for the first predetermined period.

22. The method of claim 21, wherein when whether the motion is the effective motion is determined in step (b), the result of sensing obtained during a period other than the first predetermined period in a second predetermined period is not used, and the second predetermined period is longer than the first predetermined period.

23. The method of claim 10, further comprising the step of regularizing the result of sensing after step (a) and going to step (b), wherein in step (b) whether the motion is the effective motion is determined from the result of regularization.

24. The method of claim 10, wherein step (b) further comprises regularizing the portion having a change in size extracted in step (b1), determining the result of regularization as the feature amount, and going to step (b2).

25. The method of claim 23, wherein the result of regularization is the result of multiplying the result of sensing by a weight which corresponds to a motion degree at which the user can move when selecting the information.

26. The method of claim 24, wherein the result of regularization is the result of multiplying the portion having a change in size by a weight which corresponds to a motion degree at which the user can move when selecting the information.

27. The method of claim 23, further comprising the steps of:

(e1) determining whether there are at least two effective motions; and

(e2) if it is determined that there are at least two effective motions, selecting a largest effective motion from the at least two effective motions and going to step (d),

wherein in step (d), if it is determined that there is only one effective motion in step (e1), information, which is selected by the single effective motion is determined as the effective information, or information, which is selected from information selected by the effective motions by the effective motion selected in step (e2), is determined as the effective information.

28. The method of claim 23, further comprising the steps of:

(f1) determining whether there are at least two effective motions; and

(f2) if it is determined that there are at least two effective motions, selecting a first determined effective motion from the at least two effective motions and going to step (d),

wherein in step (d), if it is determined that there is only one effective motion in step (f1), information selected by the single effective motion is determined as the effective information, or information selected by the effective motion selected from the at least two effective motions in step (f2) is determined as the effective information.

29. The method of claim 23, further comprising the steps of:

(g1) determining whether there are at least two effective motions; and

(g2) if it is determined that there are at least two effective motions, determining whether the effective motions are simultaneously determined;

(g3) if it is determined that the effective motions are simultaneously determined, selecting a largest effective motion from the at least two effective motions and going to step (d); and

(g4) if it is determined that the effective motions are not simultaneously determined, selecting a first determined effective motion from the at least two effective motions and going to step (d),

wherein in step (d), if it is determined that there is only one effective motion in step (g1), information, which is selected by the single effective motion is determined as the effective information, or information, which is selected from information selected by the at least two effective motions by the effective motion selected in step (g3) or (g4), is determined as the effective information.

30. The method of claim 23, further comprising the steps of:

(h1) determining whether there are at least two effective motions; and

(h2) if it is determined that there are at least two effective motions, determining whether the amounts of the effective motions are the same;

(h3) if it is determined that the effective motions are not the same, selecting a largest effective motion from the at least two effective motions and going to step (d); and

(h4) if it is determined that the effective motions are the same, selecting a first determined effective motion from the at least two effective motions and going to step (d),

wherein in step (d), if it is determined that there is only one effective motion in step (h1), information, which is selected by the single effective motion is determined as the effective information, or information, which is selected from information selected by the at least two effective motions by the effective motion selected in step (h3) or (h4), is determined as the effective information.

31. The method of claim 25, wherein the user moves a predetermined body part when selecting the information.

32. The method of claim 31, wherein the predetermined body part is at least one hand.

33. The method of claim 32, wherein the weight varies with the kind of finger.

34. The method of claim 32, wherein the weight varies with an angle at which a finger is bent.

35. The method of claim 32, wherein the motion corresponds to a third joint of a finger.

36. The method of claim 31, wherein step (c) comprises determining whether the amount of the sensed upward, downward, leftward, rightward, frontward, or backward motion of the predetermined body part exceeds the predetermined amount after step (a) and going to step (b) if it is determined that the amount of the sensed motion does not exceed the predetermined amount.

37. The method of claim 31, wherein the predetermined threshold is set according to a motion degree at which the predetermined body part can be moved.

38. The method of claim 22, wherein when the information is selected, the user moves a predetermined body part, and the second predetermined period is set depending on the kind of predetermined body part.

2. (Delete)

39. An apparatus for selecting information in a multi-dimensional space, the apparatus comprising:

at least one motion unit moved by a user intending to select information;

at least one sensing unit for sensing the motion of the motion unit and outputting the result of sensing;

at least one effective signal detection unit for checking whether the motion of the motion unit is an effective motion intending to select the information based on the result of sensing and outputting the result of checking as an effective signal;

a motion amount detection unit for detecting the amount of the motion from the result of sensing; and

a first comparator for comparing the amount of the motion received from the motion amount detection unit with a predetermined amount and outputting the result of comparison,

wherein the effective signal detection unit checks whether the motion of the motion unit is the effective motion in response to the result of comparison performed the first comparator.

41. The apparatus of claim 39, further comprising an analog-to-digital converter for converting the sensing result from an analog form into a digital form and outputting the result of conversion to the effective signal detection unit,

wherein the effective signal detection unit checks whether the motion of the motion unit is the effective motion based on the result of conversion.

42. The apparatus of claim 41, wherein the analog-to-digital converter converts the result of sensing from the analog form into the digital form in response to the result of comparison performed by the first comparator.

43. The apparatus of claim 39, further comprising an effective information determination unit for determining information selected by the motion as effective information in response to the effective signal received from the effective signal detection unit.

44. The apparatus of claim 39, wherein the sensing unit visually senses the motion of the motion unit.

45. The apparatus of claim 39, wherein the motion unit is moved by the user in a three-dimensional space.

46. The apparatus of claim 39, wherein the motion unit is moved by the user in a two-dimensional space.

47. The apparatus of claim 39, wherein the selection of the information implies pointing the information.

48. The apparatus of claim 39, wherein the selection of the information denotes inputting the information.

49. The apparatus of claim 43, wherein the effective signal detection unit comprises:

a feature amount detector for extracting as a feature amount a portion having a change in the size of the sensing result input from the sensing unit and outputting the extracted feature amount;

a second comparator for comparing the feature amount received from the feature amount detector with a predetermined threshold and outputting the result of comparison; and

an effective signal generator for generating the effective signal in response to the result of comparison received from the second comparator, and

wherein the sensing unit senses the motion of the motion unit in response to the result of comparison received from the second comparator.

50. The apparatus of claim 49, wherein the feature amount detector comprises a band-pass filter for band-pass filtering the result of sensing and outputting the result of band-pass filtering as the feature amount.

51. The apparatus of claim 50, wherein the feature amount detector comprises:

a first filter for filtering a first frequency component of the result of sensing and outputting the result of filtering;

a second filter for filtering a second frequency component of the result of sensing and outputting the result of filtering; and

a differential calculator for calculating a difference between the result of filtering received from the first filter and the result of filtering received from the second filter and outputting the difference as the feature amount.

52. The apparatus of claim 51, wherein the first and second frequency components are low-band components.

53. The apparatus of claim 51, wherein the first frequency component is a low-band component, and the second frequency component is a high-band component.

54. The apparatus of claim 51, wherein the first frequency component is a low-band component, and the second frequency component is a band component.

55. The apparatus of claim 51, wherein the first frequency component is a high-band component, and the second frequency component is a band component.

56. The apparatus of claim 51, wherein the first and second frequency components are high-band components.

57. The apparatus of claim 51, wherein the first and second frequency components are band components.

58. The apparatus of claim 51, wherein the feature amount detector further comprises a signal extractor for extracting only a positive component of the difference received from the differential calculator and outputting the extracted positive component as the feature amount.

59. The apparatus of claim 58, wherein the signal extractor comprises a first rectifier for rectifying the difference received from the differential calculator and outputting the result of rectification as the feature amount.

60. The apparatus of claim 51, wherein the feature amount detector further comprises a signal extractor for extracting only a negative component of the difference received from the differential calculator and outputting the extracted negative component as the feature amount.

61. The apparatus of claim 60, wherein the signal extractor comprises:

a first inverter for inverting the difference received form the differential calculator and outputting the result of inversion;

a second rectifier for rectifying the result of inversion received from the first inverter and outputting the result of rectification; and

a second inverter for inverting the result of rectification received from the second rectifier and outputting the result of inversion as the feature amount.

62. The apparatus of claim 49, wherein the effective signal detection unit further comprises:

a first counter for performing a counting operation in response to the result of comparison received from the second comparator; and

a third comparator for comparing the result of counting received from the first counter with a first predetermined period and outputting the result of comparison,

the effective signal generator generates the effective signal in response to the result of comparison received from the third comparator, and

the sensing unit senses the motion in response to the result of comparison received from the third comparator.

63. The apparatus of claim 62, wherein the effective signal detection unit further comprises a fourth comparator for comparing the result of counting received from the first counter with a second predetermined period and outputting the result of comparison, the second predetermined period being longer than the first predetermined period.

64. The apparatus of claim 63, wherein the effective signal generator generates the effective signal in response to the result of comparison received from the fourth comparator.

65. The apparatus of claim 63, wherein the sensing unit senses the motion in response to the result of comparison received from the fourth comparator.

66. The apparatus of claim 49, further comprising a regularizing unit for regularizing the result of sensing received from the sensing unit and outputting the result of regularization to the effective signal detection unit, wherein the effective signal detection unit generates the effective signal based on the result of regularization received from the regularizing unit.

67. The apparatus of claim 66, wherein the regularizing unit comprises a multiplier for multiplying the result of sensing by a weight and outputting the result of multiplication as the result of regularization, the weight being set depending on a motion degree at which the motion unit can be moved.

68. The apparatus of claim 49, wherein the effective signal detection unit further comprises a regularizing unit for regularizing the portion having a change in the size, which is extracted by the feature amount detector, and outputting the result of regularization as the feature amount to the second comparator.

69. The apparatus of claim 68, wherein the regularizing unit comprises a multiplier for multiplying the portion having a change in the size, which is extracted by the feature amount detector, by a weight and outputting the result of multiplication as the result of regularization to the second comparator, the weight being set depending on a motion degree at which the motion unit can be moved.

70. The apparatus of claim 66, further comprising:

a first storage unit for receiving one or more feature amounts corresponding to the one or more effective motions from the feature amount detector and storing the one or more feature amounts;

a second counter for counting the number of one or more effective signal generated from the effective signal detection unit; and

a first feature amount selector for selecting a biggest feature amount from among the one or more feature amounts stored in the first storage unit in response to the result of counting received from the second counter,

wherein in response to the result of counting received from the second counter, the effective information determination unit determines information, which is selected by the sole effective motion, or information, which is selected from a plurality of pieces of information selected by the two or more effective motions by an effective motion corresponding to the selected feature amount, as the effective information.

71. The apparatus of claim 66, further comprising:

a second storage unit for storing one or more instants of time when one or more feature amounts corresponding to the one or more effective motions are generated from the feature amount detector;

a third counter for counting the number of one or more effective signals generated from the effective signal detection unit; and

a first time selector for selecting an earliest time from among the one or more times stored in the second storage unit in response to the result of counting received from the third counter and outputting the selected time,

wherein in response to the result of counting received from the third counter, the effective information determination unit determines information, which is selected by the sole effective motion, or information, which is selected from a plurality of pieces of information selected by the two or more effective motions by an effective motion corresponding to the selected feature amount generated at the selected instant of time, as the effective information.

72. The apparatus of claim 66, further comprising:

a third storage unit for receiving one or more feature amounts corresponding to the one or more effective motions from the effective signal detection unit and storing the one or more feature amounts;

a fourth storage unit for storing of one or more instants of time when one ore more feature amounts corresponding to the one or more effective motions are generated from the effective signal detection unit;

a fourth counter for counting the number of the one or more effective signals generated from the effective signal detection unit;

a fifth comparator for comparing the one or more times stored in the fourth storage unit and outputting the result of comparison;

a second feature amount selector for selecting a largest feature amount from the one or more feature amounts stored in the third storage unit in response to the result of comparison received from the fifth comparator; and

a second time selector for selecting an earliest instant of time from the one or more instants of time stored in the fourth storage unit in response to the result of comparison received from the fifth comparator and outputting the selected time,

wherein the effective information determination unit determines information, which is selected by the sole effective motion, as the effective information in response to the result of counting received from the fourth counter or determines information, which is selected by an effective motion corresponding to the feature amount selected by the second feature amount selector or to the instant of time selected by the second time selector, as the effective information in response to the result of comparison received from the fifth comparator, among a plurality of pieces of information selected by the two or more effective motions.

73. The apparatus of claim 66, further comprising:

a third storage unit for receiving one or more feature amounts corresponding to the one or more effective motions from the effective signal detection unit and storing the one or more feature amounts;

a fourth storage unit for storing one or more instants of time when one or more feature amounts corresponding to the one or more effective motions are generated from the effective signal detection unit;

a fourth counter for counting the number of the one or more effective signals generated from the effective signal detection unit;

a fifth comparator for comparing the one or more feature amounts stored in the third storage unit and outputting the result of comparison;

a second feature amount selector for selecting a biggest feature amount from the one or more feature amounts store in the third storage unit in response to the result of comparison received from the fifth comparator; and

a second time selector for selecting an earliest instant of time from the one or more instants of time stored in the fourth storage unit in response to the result of comparison received from the fifth comparator and outputting the selected time,

wherein the effective information determination unit determines information, which is selected by the sole effective motion, as the effective information in response to the result of counting received from the fourth counter or determines information, which is selected by an effective motion corresponding to the feature amount selected by the second feature amount selector or to the time selected by the second time selector, as the effective information in response to the result of comparison received from the fifth comparator, among a plurality of pieces of information selected by the two or more effective motions.

74. The apparatus of claim 39, wherein the motion unit is attached to a predetermined body part of the user and moved by the user.

75. The apparatus of claim 39, wherein the user points the information by moving the motion unit.

76. The apparatus of claim 39, wherein the user determines input of the information by moving the motion unit.

77. The apparatus of claim 74, wherein the predetermined body part is at least one hand.

78. The apparatus of claim 77, wherein the motion corresponds to a third joint of a finger.

40. (Delete)