User interface system based on pointing device

• US 8,971,629 B2
• Filed: 03/31/2011
• Issued: 03/03/2015
• Est. Priority Date: 11/20/2002
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Mekenkamp, Gerhardus Engbertus, Dekker, Tim
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US13/077,334
• Publication Number:
  US 20110187643A1
• Time in Patent Office: 1,433 Days
• Field of Search: 382/181, 345/158, 345/156, 345/1.1
• US Class Current: 382/181
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/61 Password, biometric
• CPC Class Codes: G08C 2201/71 Directional beams
• CPC Class Codes: G08C 2201/92 Universal remote control
• CPC Class Codes: G08C 2201/94 Smart cards

Abstract

The user interaction system comprises a portable pointing device (101) connected to a camera (102) and sending pictures to a digital signal processor (120), capable of recognizing an object (130) and a command given by the user (100) by moving the pointing device (101) in a specific way, and controlling an electrical apparatus (110) on the basis of this recognition.

Claims

1. User interaction system, comprising:

an electrical apparatus;

a portable pointing device operable by a user for pointing to a region in space;

a camera taking a picture;

motion sensing means; and

,

a digital signal processor, capable of receiving and processing the picture, and capable of transmitting user interface information derived from the picture to the electrical apparatus;

wherein the camera is connected to the pointing device so that in operation it images the region pointed to, wherein the system further comprises at least one room localization beacon, in a room wherein the pointing device is used, that can emit electromagnetic radiation, for use by the digital signal processor in order to recognize to which part of the room the pointing device is pointing;

wherein the digital signal processor is further arranged to recognize to which part of a room the pointing device is pointing; and

,

wherein the digital signal processor is arranged to analyze gestures made with the pointing device based on a motion trajectory of the pointing device.

2. The user interaction system as in claim 1, wherein the motion trajectory of the pointing device is estimated on the basis of the motion sensing means.

3. The user interaction system as in claim 1, wherein the motion or the motion trajectory of the pointing device is estimated on basis of successive pictures imaged by the camera.

4. User interaction system as claimed in claim 2, wherein the transmitted user interface information includes at least one feature selected from the group consisting of motion speed, and motion direction of the pointing device.

5. User interaction system as claimed in claim 2, wherein the transmitted user interface information includes at least one feature selected from the group consisting of motion trajectory of the pointing device and a characteristic signature derived from the motion trajectory of the pointing device.

6. User interaction system as claimed in claim 1, wherein the pointing device further comprises feedback means for providing the user with additional information, and wherein said feedback means is arranged to provide feedback of which part of an apparatus the user is pointing to.

7. User interaction system as claimed in claim 6, wherein said feedback means is selected from the group consisting of light, sound production device, a display and force feedback means.

8. User interaction system as claimed in claim 6 wherein said feedback means is force feedback.

9. User interaction system as claimed in claim 8 wherein said force feedback means is arranged to simulate the touching of an apparatus.

10. User interaction system of claim 1, wherein the user interface information comprises apparatus control data for controlling operation of the electrical apparatus.

11. Pointing device for use in a user interaction system of claim 1 wherein the pointing device comprises the digital signal processor and the camera and wherein the pointing device is capable of sending a picture to the digital signal processor.

12. Pointing device of claim 9, further comprising a characteristic projector for optically projecting a characteristic pattern towards a region pointed to.

13. Pointing device of claim 9 further comprising a programmable user interface code generator and a wireless transmitter for transmitting the code to the electrical apparatus.

14. Electrical apparatus for use in a user interaction system as in claim 1, wherein said electrical apparatus comprises an interface means which allow the electrical apparatus to send information about supported commands to the pointing device based on an “

identify supported commands”

call of the pointing device to the electrical apparatus.