Pointing apparatus, pointer control apparatus, pointing method, and pointer control method

• US 8,451,223 B2
• Filed: 03/19/2008
• Issued: 05/28/2013
• Est. Priority Date: 09/06/2007
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Eun-seok, Bang, Won-chul, Choi, Sang-on, Choi, Mun-cheol
• Primary Examiner(s): Lao, Lun-Yi
• Assistant Examiner(s): MERKOULOVA, OLGA VLADIMIROVNA
• Application Number:
  US12/076,553
• Publication Number:
  US 20090066646A1
• Time in Patent Office: 1,896 Days
• Field of Search: 345/158, 345/156, 345/157, 345/159, 345/163, 345/179, 345/180, 345/182, 345/183, 178/19.05, 348/734
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Provided is a pointing apparatus, a pointer control apparatus, a pointing method, and a pointer control method to a pointing apparatus, a pointer control apparatus, a pointing method, and a pointer control method capable of recognizing image codes included in an image frame using an image sensor to determine a pointing direction, and continuously updating the gain between the displacement of the motion of the pointing apparatus and the displacement of the motion of a displayed pointer. The pointing apparatus includes an image receiving unit sensing image patterns that exist in a sensed region, among all of the image patterns arranged in a display region; an inertial sensor sensing an input motion using at least one of the acceleration and angular velocity that are generated due to the motion; and a coordinate determining unit determining moving coordinates that are moved from the central coordinates of the sensed image pattern by coordinate displacement corresponding to the sensed motion.

Claims

1. A pointing device comprising:

an image sensing unit to sense at least one image pattern among a plurality of image patterns arranged in a display region of a display device, to determine first position information;

an inertial sensing unit to sense a motion of the pointing device using at least one of acceleration information and angular velocity information and to determine second position information based on the sensed motion; and

a coordinate determining unit to determine a first set of coordinates on the display device using the first position information and the second position information,

wherein at least one of the image patterns displayed on the display region are stored beforehand and are formed by at least one of a visible ray, an infrared ray, and an ultraviolet ray,

wherein all of the image patterns are arranged in image frames that are displayed at predetermined time intervals, among the image frames displayed on the display region,

wherein at least one of the image patterns displayed on the display region include one or more different image codes.

2. The pointing device of claim 1, further comprising:

a gain calculating unit to compare a coordinate displacement between coordinates of two image patterns that are sensed at a predetermined time interval with an inertial coordinate displacement corresponding to a motion that is sensed by using at least one of the acceleration and the angular velocity information for a predetermined time, and to calculate a gain for synchronizing the inertial coordinate displacement with the coordinate displacement of the first set of coordinates on the display region.

3. The pointing device of claim 2, wherein the coordinate determining unit determines the first set of coordinates after the predetermined time by the inertial coordinate displacement to which the gain is applied.

4. The pointing device of claim 1, wherein the image sensing unit comprises a camera unit to capture at least one of the image patterns displayed on the display region.

5. The pointing device of claim 1, wherein the one or more different image codes include a plurality of different image codes arranged at fixed positions on the display region.

6. The pointing device of claim 1, wherein the first set of coordinates are changed to a second set of coordinates based on the sensed motion.

7. A pointer control device comprising:

a receiving unit to receive at least one sensed image pattern among a plurality of image patterns arranged in a display region of a display device, to extract first position information, and to receive a motion of a pointing device that is sensed by an inertial sensing unit of the pointing device to determine second position information based on the received motion; and

a coordinate determining unit to determine a first set of coordinates on the display device using the first position information and the second position information,

wherein at least one of the image patterns displayed on the display region are stored beforehand and are formed by at least one of a visible ray, an infrared ray, and an ultraviolet ray,

wherein all of the image patterns are arranged in image frames that are displayed at predetermined time intervals, among the image frames displayed on the display region,

wherein at least one of the image patterns displayed on the display region include one or more different image codes.

8. The pointer control device of claim 7, further comprising;

an image generating unit to generate an image frame that includes all of the image patterns.

9. The pointer control device of claim 7, further comprising:

a gain calculating unit to compare a coordinate displacement between coordinates of two image patterns that are sensed at a predetermined time interval with an inertial coordinate displacement corresponding to a motion that is sensed by the inertial sensing unit for a predetermined time, and to calculate a gain for synchronizing the inertial coordinate displacement with the coordinate displacement of the first set of coordinates on the display region.

10. The pointer control device of claim 9, wherein the coordinate determining unit determines the first set of coordinates after the predetermined time by the inertial coordinate displacement to which the gain is applied.

11. The pointer control device of claim 7, further comprising:

a pointer operating unit to move a mouse pointer to absolute coordinates on the display region that correspond to the first set of coordinates.

12. The pointer control device of claim 11, further comprising:

a displaying unit to display the mouse pointer at the absolute coordinates on the display region.

13. The pointer control device of claim 7, wherein the first set of coordinates are changed to a second set of coordinates based on the received motion.

14. The pointer control device of claim 7, further comprising:

an image generating unit to generate a plurality of image frames which are transmitted to the display device to be displayed by the display device; and

a transmitting unit to transmit an image of a pointer to the display device, and to transmit instructions to move the pointer based on the determined first set of coordinates,

wherein the receiving unit receives the at least one sensed image pattern from the pointing device which is disposed externally to the pointer control device.

15. A method for coordinate determining by a pointing device comprising:

sensing, by a pointing device, at least one image pattern that exists in a sensed region, among a plurality of image patterns arranged in a display region;

determining a distance from the pointing device to the at least one sensed image pattern;

adjusting a scale of the sensed region to match a scale of the image patterns arranged in the display region by changing the scale of the sensed region according to the determined distance;

extracting central coordinates of the at least one sensed image pattern using the adjusted scale of the sensed region;

sensing a motion using at least one of the acceleration and angular velocity that are generated due to the motion; and

determining next coordinates of the at least one sensed image pattern that are moved from central coordinates of the at least one sensed image pattern by coordinate displacement corresponding to the sensed motion,

wherein at least one of the image patterns displayed on the display region are stored beforehand and are formed by at least one of a visible ray, an infrared ray, and an ultraviolet ray,

wherein all of the image patterns are arranged in image frames that are displayed at predetermined time intervals, among the image frames displayed on the display region.

16. The method of claim 15, wherein at least one of the image patterns displayed on the display region are composed of one or more different image codes.

17. The method of claim 15, further comprising:

calculating the coordinate displacement between coordinates of two image patterns that are sensed at a predetermined time interval, among the at least one sensed image patterns;

comparing the coordinate displacement between the coordinates of the two image patterns with an inertial coordinate displacement corresponding to a sensed motion using at least one of the acceleration and the angular velocity for a predetermined time; and

calculating a gain for synchronizing the inertial coordinate displacement with the coordinate displacement of the next coordinates on the display region according to the result of the comparison.

18. The method of claim 17, wherein the determining of the next coordinates comprises determining the next coordinates that are moved from the central coordinates of the at least one sensed image pattern, after the predetermined time by the inertial coordinate displacement to which the gain is applied.

19. The method of claim 15, wherein the sensing image patterns comprises capturing at least one of the image patterns displayed on the display region using a camera.

20. The method of claim 15, wherein at least one of the image patterns displayed on the display region are formed by a plurality of different image codes arranged at fixed positions on the display region.

21. A method for coordinate determining by a pointer control device comprising:

receiving at least one image pattern that exists in a sensed region among a plurality of image patterns arranged in a display region and receiving a motion that is sensed by an inertial sensing unit;

extracting central coordinates of the at least one sensed image pattern based on a determined range of the at least one sensed image pattern; and

determining next coordinates that are moved from central coordinates of the at least one received image pattern by coordinate displacement corresponding to the received sensed motion,

wherein at least one of the image patterns displayed on the display region are stored beforehand and are formed by at least one of a visible ray, an infrared ray, and an ultraviolet ray,

wherein all of the image patterns are arranged in image frames that are displayed at predetermined time intervals, among the image frames displayed on the display region,

wherein at least one of the image patterns displayed on the display region include one or more different image codes.

22. The method of claim 21, further comprising;

generating an image frame that includes all of the image patterns.

23. The method of claim 21, further comprising:

calculating the coordinate displacement between coordinates of two image patterns that are sensed at a predetermined time interval among the at least one received sensed image patterns;

comparing the coordinate displacement between the coordinates of the two image patterns with an inertial coordinate displacement corresponding to a motion that is sensed by the inertial sensing unit for a predetermined time; and

calculating a gain for synchronizing the inertial coordinate displacement with the coordinate displacement of the next coordinates on the display region, according to the result of the comparison.

24. The method of claim 23, wherein the determining of the next coordinates comprises determining the next coordinates that are moved from the central coordinates of the at least one received sensed image pattern, after the predetermined time by the inertial coordinate displacement to which the gain is applied.

25. The method of claim 21, further comprising:

moving a mouse pointer to the absolute coordinates on the display region that correspond to the next coordinates.