Multi-surface controller

• US 10,444,849 B2
• Filed: 08/18/2015
• Issued: 10/15/2019
• Est. Priority Date: 09/01/2014
• Status: Active Grant

• Current Assignee: Yinbo Li
• Original Assignee: Yinbo Li
• Inventors: Li, Yinbo
• Primary Examiner(s): Abdulselam, Abbas I
• Application Number:
  US14/829,512
• Publication Number:
  US 20160062489A1
• Time in Patent Office: 1,519 Days
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: G02B 27/20 for imaging minute objects, e.g. light-pointer
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 23/04 using light waves, e.g. infrared

Abstract

A control device is provided having multiple buttons with touch sensors attached on top of the buttons. The buttons and touch sensors may detect signals of finger movements and/or gestures. The detected signals may be used to control various operations in various operation modes of the controller. The control device may interface with multiple devices.

Claims

1. A device, comprising:

a body configured to be held by a hand in the air;

two or more input interfaces that are connected to the body, the two or more input interfaces comprising;

a first input interface configured to detect a first input comprising a first press action and a first touch gesture from a first finger of the hand when the hand holds the device in the air, wherein the first press action is detected based on comparing a pressure exerted by the first finger on the first input interface against a threshold, and wherein the first touch gesture is detected based on at least one of a direction and a distance of movement of the first finger on the first input interface; and

a second input interface configured to detect a second input from a second finger of the hand when the hand holds the device in the air;

a communication interface for communicating with at least one external device, the at least one external device including a display device to display content and a cursor;

a memory system storing one or more machine instructions; and

a hardware processor configured to execute the one or more machine instructions to;

generate a first control command based on at least one of the first press action or at least a first part of the second input;

generate a second control command based on at least one of the first touch gesture or at least a second part of the second input;

transmit, via the communication interface, the first control command to the at least one external device to control content displayed by the display device; and

transmit, via the communication interface, the second control command to the at least one external device to control a display location of the cursor at the display device.

2. The device of claim 1, wherein the first input interface comprises a first button to detect the first press action and a first touch sensor to detect the first touch gesture;

and

wherein the second input comprises a second press action and a second touch gesture;

wherein the second input interface comprises a second button to detect the second press action and a second touch sensor to detect the second touch gesture;

wherein the second press action is detected based on comparing a pressure exerted by the second finger against the threshold; and

wherein the second touch gesture is detected based on at least one of a direction and a distance of movement of the second finger on the second touch sensor.

3. The device of claim 2, wherein the first button and the first touch sensor form a first movable touch pad;

and

wherein the second button and the second touch sensor form a second movable touch pad.

4. The device of claim 1, wherein the body has a length and a width, the length being longer than the width, the body has a longitudinal axis extending along the length of the body, the body is symmetric about the longitudinal axis of the body.

5. The device of claim 1, wherein the first finger is an index finger, and wherein the second finger is a middle finger.

6. The device of claim 5, wherein the first input interface is located on a surface of the body, the device further comprising a concave area located on another surface of the body, opposite the surface on which the first input interface is located, the concave area facing away from the first input interface and configured to accommodate multiple fingers of the hand to wrap underneath the body to support the body when the hand holds the device in the air.

7. The device of claim 1, further comprising a motion sensor configured to detect a motion of the device;

wherein the hardware processor is configured to execute the one or more machine instructions to;

generate a third control command based on the detection of the motion of the device, the third control command being associated with a coarse control of the display location of the cursor; and

transmit, via the communication interface, the second control command to the at least one external device to control the display location of the cursor.

8. The device of claim 1, wherein the first input interface includes multiple portions, wherein different signals are sensed when different portions of the multiple portions are touched.

9. The device of claim 1, wherein the first input interface and the second input interface are configured to generate, respective, a first type of signal and a second type of signal.

10. The device of claim 1, wherein the first input interface and the second input interface are configured to detect simultaneously a first input and a second input from respecitvely the first finger and from the second finger;

and

wherein the hardware processor is configured to execute the one or more machine instructions to generate a third control command based on combining the first input and the second input.

11. The device of claim 1, wherein each of the first input interface and the second input interface is configured to detect a plurality of types of actions.

12. The device of claim 1, further comprising at least a microphone for detecting sound signals.

13. The device of claim 12, wherein the memory system stores instructions, which when activated cause the hardware processor to translate sound signals generated by a voice into control instructions.

14. The device of claim 1, wherein at least one of the two or more input interfaces has haptic feedback.

15. The device of claim 1, further comprising one or more speakers for broadcasting sound signals.

16. The device of claim 1, further comprising a laser, which when activated, emits a beam of laser light for pointing at an object.

17. The device of claim 1, further comprising a light, which when activated, emits light to indicate a status of the device.

18. The device of claim 1, wherein the communication interface, when activated, connects the device to at least a computing device, wherein the computing device is communicatively connected to multiple display systems, wherein the device controls the computing device to display the content on at least one of the multiple display systems.

19. The device of claim 1, wherein the hardware processor is configured to execute the one or more machine instructions to:

generate a first type of control command based on detecting the first input at the first input interface and no input at the second input interface; and

generate a second type of control command based on detecting the first input at the first input interface and the second input at the second input interface.

20. The device of claim 1, wherein a same signal is interpreted differently depending on a mode in which the device operates.

21. The device of claim 1, wherein the two or more input interfaces includes three buttons, each being connected to sensors.

22. The device of claim 1, wherein at least one of the two or more input interfaces is connected to sensors that independently detect multiple touches on the same input interface.

23. The device of claim 1, wherein

the first input corresponds to the first finger moving towards an upper left side or an upper right side of the device;

the second input corresponds to the second finger moving towards a left side or a right side of the device simultaneously as the first finger moves; and

the control command is a zoom-out control command generated based on a combination of the first input and the second input.

24. The device of claim 1, wherein

the first input corresponds to the first finger moving towards a lower right side or a lower left side of the device;

the second input corresponds to the second finger moving towards a right side or a left side of the device simultaneously as the first finger moves; and

the control command is a zoom-in control command generated based on a combination of the first input and the second input.

25. The device of claim 1, wherein

the first and second inputs correspond to the first finger and the second finger swiping in a same direction simultaneously with respect to, respectively, a first edge of the first input interface and a second edge of the second input interface;

and

the control command is a switch control command generated based on a combination of the first input and the second input.

26. The device of claim 1, wherein

the first and second inputs correspond to the first finger and the second finger swiping in reverse directions simultaneously with respect to, respectively, a first edge of the first input interface and a second edge of the second input interface;

and

the control command is a rotation control command generated based on a combination of the first input and the second input.

27. The device of claim 1, wherein

the first input corresponds to the first finger clicking and holding down a button of the first input interface;

and

the second input corresponds to the second finger moving on a touch pad of the second input interface.

28. The device of claim 27, wherein the control command generated based on the combination of the first and second input is for controlling a movement or an orientation of a 3-dimensional object.

29. The device of claim 1, wherein the body has an ergonomic shape configured to accommodate a palm of the hand.

30. The device of claim 1, wherein the threshold is set based on at least one of:

an application operating on the at least one external device, or an operation mode of the at least one external device.

31. The device of claim 1, wherein the hardware processor is configured to execute the one or more machine instructions to:

for a first application, generate the first control command based on the first input interface detecting a pre-determined number of first press actions within a pre-determined time; and

for a second application, generate the first control command based on the first input interface detecting the first press action for a pre-determined duration.

32. The device of claim 1, wherein the first input interface is configured to detect the first press action and the first touch gesture simultaneously.

33. The device of claim 1, wherein the first input interface is configured to detect the first touch gesture from the first finger, or from the first finger and a third finger.

34. The device of claim 33, wherein the first finger is a thumb, the second finger is a middle finger, and the third finger is an index finger.

35. A method, comprising:

detecting, by a first input interface on a first surface of a device, a first input from a first finger of a hand holding the device in the air, the first input comprising a first press action and a first touch gesture, the first press action being detected based on comparing a pressure exerted by the first finger on the first input interface against a threshold, and the first touch gesture being detected based on at least one of a direction and a distance of movement of the first finger on the first input interface;

detecting, by a second input interface on a surface side of the device different from the first surface, a second input from a second finger of the hand holding the device in the air;

generating, by a hardware processor of the device, a first control command based on at least one of the first press action or at least a first part of the second input;

generating, by the hardware processor of the device, a second control command based on at least one of the first touch gesture or at least a second part of the second input;

sending the first control command to a display device to control content displayed by the display device; and

sending the second control command to the display device to control a display location of a cursor on the display device.

36. A control device, comprising:

a plurality of buttons including at least a first button and a second button, the first button and the second button configured to detect, respectively, a first press action of a first finger and a second press action of a second finger of a hand holding the control device in the air, the first button and the second button being on different surfaces of the control device, the first press action being detected based on comparing a pressure exerted by the first finger on the first button against a threshold, and the second press action being detected based on comparing a pressure exerted by the second finger on the second button against the threshold;

a plurality of sensors including at least a first touch sensor and a second touch sensor, the first touch sensor configured to detect a first touch signal indicating at least one of a direction or a distance of movement of the first finger on the first button, the second touch sensor configured to detect a second touch signal indicating at least one of a direction or a distance of movement of the second finger on the second button;

a processor configured to generate a first control command based on at least one of the first press action or the second press action, and to generate a second control command based on at least one of the first touch signal or the second touch signal; and

a wireless communication interface configured to transmit the first control command to a display device to control content displayed on the display device, and to transmit the second control command to the display device to control a display location of a cursor on the display device.