REMOTE CONTROL WITH 3D POINTING AND GESTURE RECOGNITION CAPABILITIES

• US 20140092011A1
• Filed: 09/28/2012
• Published: 04/03/2014
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Movea
• Original Assignee: Movea
• Inventors: DE FORAS, Etienne, AUJAY, Guillaune, AUJAY, Grgoire, GOMEZ, David, MICHALLON, Loic, MEUNIER, Nicolas
• Granted Patent: US 9,927,876 B2
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device

Abstract

A remote control, such as a 3D air mouse, includes motion sensors used to measure position/orientation in space to alternatively control a cursor on a display, recognize gestures by the user so that data/commands can be entered into an electronic system being controlled or the like. Timed sequences of pressing a single trigger button and the quantity of motion during said sequences are both timed in relation to thresholds to switch between modes.

Claims

1. A user to machine interface comprising:

a remote control with at least a trigger button,

a communication link between the remote control and a base station having a display,

motion sensing capabilities for generating motion signals at least partially representative of the motion of the remote control,

processing capabilities configured;

for processing said motion signals in at least one of a first mode and a second mode of operation of said interface, and

for switching the operation of the interface between said first and second modes, at least one of which enables operation of said interface based on motion of the remote control,

wherein the switching decisions are based on an evaluation of a combination of i) an analysis of a pattern of at least a sequence of ON/OFF states of said trigger button resulting from the user pressing/releasing said trigger button, and ii) a function Q_motion representative of a quantity of motion imparted to the remote by the user, derived from the motion signals at at least one moment during said sequence of ON/OFF states.

2. The interface of claim 1, wherein the analysis of a pattern of at least a sequence of ON/OFF states is based on a computation of a duration δ

_ON of an ON state and the function Q_motion is calculated at at least one moment during said ON state.

3. The interface of claim 2, wherein said switching decisions are based on an evaluation of the position of a point P defined by the two coordinates values δ

_ON (P) and Q_motion (P), in relation to a set of regions in a 2D plane defined by the δ

_ON and Q_motion axes where said point P is plotted, said set of regions being defined as a function of the first and second modes.

4. The interface of claim 3, wherein each region in said set of regions defines a class of user'"'"'s actions.

5. The interface of claim 4, wherein the switching decision depends on a combination of the result of the evaluation of δ

_ON and Q_motion and of the switching decision defined by the region to which point P belongs.

6. The interface of claim 3, wherein said set of regions includes only one continuous region R.

7. The interface of claim 6, wherein said first mode belongs to a group of modes comprising i) a mode A defined as a passive mode where the interface is not operated based on motion of the remote control and ii) a mode B of control of a cursor on the display, and the result of the evaluation of δ

_ON and Q_motion is that the point P belongs to the region R, the second mode to which the operation of the interface will switch is the other one of modes A and B in which the interface was not operating prior to said switching.

8. The interface of claim 6, wherein said first mode belongs to a group of modes comprising i) a mode A defined as a passive mode where the interface is not operated based on motion of the remote control and ii) a mode B of control of a cursor on the display, and the result of the evaluation of δ

_ON and Q_motion is that the point P corresponding to said first mode does not belong the region R, the second mode to which the operation of the interface will switch is a mode C of user'"'"'s gesture recognition.

9. The interface of claim 8, wherein, at the end of the ON state, the operation of the interface will switch to the first mode in which the interface was operating prior to the beginning of the ON state.

10. The interface of claim 9, wherein the first mode is mode B.

11. The interface of claim 10, wherein, when switching from the first mode to the second mode, the position of the cursor is frozen on the display at the cursor coordinates at the start of the ON state.

12. The interface of claim 11, wherein the cursor is unfrozen at a moment which is delayed by a preset value T after the end of the ON state.

13. The interface of claim 11, wherein a position on the display of a virtual cursor corresponding to the motion of the remote device is computed during execution of the second mode and the cursor is unfrozen after the end of the ON state when the distance of said virtual cursor to said frozen cursor is lower than a preset value.

14. The interface of claim 1, wherein the motion sensing capabilities comprise at least one of a multi-axis accelerometer, a multi-axis gyrometer, a multi-axis magnetometer, radio-localization capabilities and image sensing capabilities.

15. A method of operating a user to machine interface comprising:

acquiring signals from motion sensing capabilities, said signals being at least partially representative of the motion of a remote control held by the user,

processing the signals from the motion sensing capabilities in one of a first mode and a second mode, at least one of which enables operation of said interface based on motion of the remote control,

pressing a trigger button on the remote control by said user to cause the

processing capabilities to switch between said first mode and second modes, said method further comprising determining the switching decisions between said first and second modes based on an evaluation of a combination of;

i) an analysis of a pattern of at least a sequence of ON/OFF states of said trigger button resulting from the user pressing/releasing said trigger button, and ii) a function Q_motion representative of a quantity of motion imparted to the remote by the user, derived from the motion signals at at least one moment during said sequence of ON/OFF states.

16. A computer program to operate a user to machine interface comprising:

at least a module configured for acquiring signals from motion sensing capabilities, said signals being at least partially representative of the motion of a remote control held by the user,

at least a module configured for processing signals from the motion sensing capabilities in at least a first mode and a second mode of operation of the interface,

a module configured to have a trigger button pressed by said user to cause processing the signals from the motion sensing capabilities in one of said first and second modes,

said program further comprising a module configured for determining switching decisions between said first and second modes based on an evaluation of a combination of i) an analysis of a pattern of at least a sequence of ON/OFF states of said trigger button resulting from the user pressing/releasing said trigger button, and ii) a function Q_motion representative of a quantity of motion imparted to the remote by the user, derived from the motion signals at at least one moment during said sequence of ON/OFF states.