User interface system based on pointing device

• US 8,537,231 B2
• Filed: 06/11/2009
• Issued: 09/17/2013
• Est. Priority Date: 11/20/2002
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Mekenkamp, Gerhardus Engbertus, Dekker, Tim
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): HSU, AMY R
• Application Number:
  US12/482,864
• Publication Number:
  US 20090251559A1
• Time in Patent Office: 1,559 Days
• Field of Search: 348/222.1, 715/863, 715/856, 715/757
• US Class Current: 348/222.1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/61 Password, biometric
• CPC Class Codes: G08C 2201/71 Directional beams
• CPC Class Codes: G08C 2201/92 Universal remote control
• CPC Class Codes: G08C 2201/94 Smart cards

Abstract

The user interaction system comprises a portable pointing device (101) connected to a camera (102) and sending pictures to a digital signal processor (120), capable of recognizing an object (130) and a command given by the user (100) by moving the pointing device (101) in a specific way, and controlling an electrical apparatus (110) on the basis of this recognition. (Characteristic) pattern generation means (116) can be included in the system for facilitating object recognition.

Claims

1. User interaction system, comprising:

an electrical apparatus;

a portable pointing device operable by a user for pointing to a region in space;

a camera taking a picture, which camera is physically attached to the pointing device so that in operation it always images the region pointed to without imaging the pointing device;

a distributed digital signal processor, the digital signal processor being capable of receiving and processing the picture, and capable of transmitting user interface information derived from the picture to the electrical apparatus;

at least one localization beacon that emit electromagnetic radiation for use by the digital signal processor in order to recognize where the pointing device is pointing;

wherein the digital signal processor estimates a motion or a motion trajectory of the pointing device; and

,

wherein said digital signal processor recognizes where the pointing device is pointing without utilizing any electromagnetic radiation emitted from the pointing device itself.

2. User interaction system as claimed in claim 1, wherein the transmitted user interface information includes at least one feature selected from the group consisting of motion speed, and motion direction of the pointing device.

3. User interaction system as claimed in claim 1, wherein the transmitted user interface information includes at least one feature selected from the group consisting of motion trajectory of the pointing device and a characteristic signature derived from the motion trajectory of the pointing device.

4. User interaction system as in claim 1, wherein the user interface information comprises apparatus control data for controlling operation of the electrical apparatus.

5. Electrical apparatus for use in a user interaction system as in claim 1, characterized in that interface means are comprised which allow the electrical apparatus to send information about supported commands to a pointing device based on an “

identify supported commands”

call of the pointing device to the electrical apparatus.

6. The user interaction system as in claim 1, wherein the motion or the motion trajectory of the pointing device is estimated on basis of motion sensing means.

7. The user interaction system as in claim 1, wherein the motion or the motion trajectory of the pointing device is estimated on basis of successive pictures imaged by the camera at respective instances of time.

8. The user interaction system as in claim 1, wherein the digital signal processor is distributed in a number of parts.

9. The user interaction system as in claim 8, wherein a first part of the digital signal processor is included within the pointing device and a second part of the digital signal processor is included within the electrical apparatus.

10. The user interaction system as in claim 1, wherein the digital signal processor is arranged to analyze gestures made with the pointing device based on said motion trajectory.

11. The user interaction system as in claim 10, wherein the pointing device is arranged to control an operation of the electrical apparatus based on said analyzed gestures.

12. The user interaction system as in claim 1, wherein the motion trajectory is summarized by a signature which is a mathematical representation of said motion. trajectory.

13. The user interaction system as in claim 12, wherein the signature is linked to a command for controlling the electrical apparatus.

14. The user interaction system as in claim 12, wherein the signature is matched by a curve fitting technique.

15. The user interaction system as in claim 1, wherein the pointing device further comprises feedback means for providing a user with feedback of which part of the electrical apparatus the user is pointing to.

16. The user interaction system as in claim 15, wherein said feedback means is selected from the group consisting of light, sound, a display and force feedback means.

17. A method of interacting between an electrical apparatus and a portable pointing device, the device being physically attached to a camera, the method comprising:

pointing the pointing device to a region in space;

taking a picture with the camera, wherein the picture is of the region pointed to, and wherein the picture does not contain any image of the pointing device;

receiving and processing the picture;

receiving electromagnetic radiation from at least one localization beacon;

recognizing where the pointing device is pointing by at least in part on utilizing the received electromagnetic radiation;

transmitting user interface information derived from the picture to the electrical apparatus;

wherein the processing step comprises estimating a motion ion or a motion trajectory of the pointing device; and

,

wherein the recognizing step does not utilize any electromagnetic radiation emitted from pointing device itself.

18. The method of claim 17 further comprising the step of utilizing a motion sensing means in estimating the motion or the motion trajectory of the pointing device.

19. The method of claim 17 further comprising analyzing successive pictures imaged by the camera at respective instances of time in estimating the motion or the motion trajectory of the pointing device.

20. The method of claim 17 further comprising analyzing gestures made with the pointing device based on said motion trajectory.

21. The method of claim 20 further comprising controlling an operation of the electrical apparatus based on said analyzed gestures.

22. The method of claim 17 further comprising summarizing the motion trajectory by a signature which is a mathematical representation of said motion trajectory.

23. The method of claim 22 further comprising linking the signature to a command for controlling an operation of the electrical apparatus.