Haptic controller with touch-sensitive control knob

• US 9,862,099 B1
• Filed: 06/22/2015
• Issued: 01/09/2018
• Est. Priority Date: 06/22/2015
• Status: Active Grant

• Current Assignee: Verily Life Sciences LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Linnell, Jeffrey, Jules, Anthony Sean
• Primary Examiner(s): Kiswanto, Nicholas
• Application Number:
  US14/746,076
• Time in Patent Office: 932 Days
• CPC Class Codes: B25J 13/02 Hand grip control means handles or pedals for crane control B66C13/56; for measuring the force applied to control members G01L5/22; hand-held casings for switching devices, e.g. joy-sticks H01H9/0214
• CPC Class Codes: B25J 13/025 comprising haptic means
• CPC Class Codes: B25J 13/065 comprising joy-sticks
• CPC Class Codes: B25J 13/081 Touching devices, e.g. pressure-sensitive
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B60K 2370/158 Haptic output
• CPC Class Codes: B60K 35/25 using haptic output
• CPC Class Codes: G05B 2219/35438 Joystick
• CPC Class Codes: G05B 2219/35449 Joystick and buttons for menu and function selection, scrolling, +sign and -sign
• CPC Class Codes: G05G 1/08 Controlling members for hand actuation by rotary movement, e.g. hand wheels
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means

Abstract

Example implementations may relate to a haptic hand-holdable controller. In particular, an example device may take the form of a haptic controller, which senses tactile information and provides force feedback for a more intuitive user experience. The force feedback may indicate a state of the device that is being controlled. An example haptic handheld controller may be utilized to manipulate data input to a robot, a tablet computer, and/or any other type of computing device. In an example embodiment, the haptic handheld controller may be such that the controller indicates to the user what manipulation of different types of data feels like, for example, by using operating modes for the haptic handheld controller where a motor varies feedback to the handheld controller.

Claims

1. A hand-holdable controller comprising:

a rotatable knob;

at least one motor that is operable to apply torque to the rotatable knob;

one or more touch sensors comprising at least one touch sensor arranged on a curved surface of the rotatable knob;

a control system configured to;

receive operating mode data indicating an operating mode for the rotatable knob;

operate the at least one motor to apply torque to the rotatable knob in accordance with the operating mode data;

receive touch data generated by the one or more touch sensors;

based on a comparison of the touch data to predetermined grip data that specifies parameters for detection of a plurality of grips on the rotatable knob, detect when each of the plurality of grips occurs on the rotatable knob, wherein the plurality of grips comprises at least a first and second grip;

detect a first rotation of the rotatable knob; and

when the first rotation is detected in association with detection of the first grip, generate first command data for a robotic device; and

when the first rotation is detected in association with detection of the second grip, generate second command data for the robotic device, wherein the second command data is different from the first command data.

2. The hand-holdable controller of claim 1, wherein at least a portion of the touch data is indicative of a touch gesture.

3. The hand-holdable controller of claim 2, wherein the touch gesture comprises a palming or a multi-finger sequence.

4. The hand-holdable controller of claim 1, wherein the touch data is indicative of a grip.

5. A hand-holdable controller configured to operate a robotic system, the hand-holdable controller comprising:

a rotatable knob;

at least one motor that is operable to apply torque to the rotatable knob;

one or more touch sensors comprising at least one touch sensor arranged on a curved surface of the rotatable knob; and

a control system configured to;

receive touch data generated by the one or more touch sensors;

based on a comparison of the touch data to parameters indicated by a predetermined list of anticipated grips on the rotatable knob, select an active grip from the predetermined list of anticipated grips;

detect a first rotation of the knob;

when the first rotation is detected in association with detection of a first grip from the list of anticipated grips, generate first command data for a robotic device; and

when the first rotation is detected in association with detection of a second grip from the list of anticipated grips, generate second command data for the robotic device, wherein the second command data is different from the first command data.

6. The hand-holdable controller of claim 5, wherein the list of anticipated grips includes one or more grips selected from the following types of grips:

(i) a full grip, (ii) a partial grip, (iii) a fingertip grip, and (iv) a palming.

7. The hand-holdable controller of claim 5, wherein the at least one motor is configured to provide feedback comprising (i) vibrational feedback, (ii) resistance to rotation of the rotatable knob, or (iii) rotational feedback by applying a torque to the rotatable knob.

8. The hand-holdable controller of claim 7, wherein the resistance is a minimal resistance corresponding to a free spin function of the rotatable knob.

9. A hand-holdable controller comprising:

a rotatable knob;

at least one motor that is operable to apply torque to the rotatable knob;

one or more touch sensors comprising at least one touch sensor arranged on a curved surface of the rotatable knob; and

a control system configured to;

receive touch data generated by the one or more touch sensors;

receive operating data indicating one or more operating modes for the rotatable knob, wherein the one or more operating modes are associated with at least one grip;

based on a comparison of the touch data to predetermined grip data that specifies parameters for detection of a plurality of grips on the rotatable knob, detect when each of the plurality of grips occurs on the rotatable knob, wherein the plurality of grips comprises at least a first and second grip;

when a first grip from the plurality of grips is detected, set a first operating mode as a current operating mode for the rotatable knob; and

when a second grip from the plurality of grips is detected, set a second operating mode as the current operating mode for the rotatable knob, wherein operation in the second operating mode changes the manner in which the motor applies torque to the rotatable knob, as compared to operation in the first operating mode; and

operate the at least one motor to apply torque to the rotatable knob in accordance with the current operating mode.

10. The hand-holdable controller of claim 9, wherein the control system is further operable to determine surface location data corresponding to two or more surface locations on the one or more surfaces of the rotatable knob;

and

wherein the control system is further configured to;

provide, to a controller application, surface output data indicative of the detected surface location data, wherein the surface output data is used to generate commands for control of a robotic device.

11. The hand-holdable controller of claim 10, wherein the commands vary based on the detected surface location and the current operating mode.

12. The hand-holdable controller of claim 11, further comprising:

a base, wherein the rotatable knob is coupled to the base;

wherein the rotatable knob comprises a proximate end that is proximate to the base and a distal end that is distal from the base;

wherein touch input corresponding to movement from the proximate end is interpreted as a command that decreases magnitude of at least one parameter.

13. The hand-holdable controller of claim 9, wherein the touch data is indicative of a gesture.

14. The hand-holdable controller of claim 13, wherein the gesture comprises a multifinger sequence, a drag, a hold, or a swipe.

15. The hand-holdable controller of claim 9, wherein the touch data is further indicative of a first grip from a plurality of predetermined grips.

16. The hand-holdable controller of claim 15, wherein the plurality of predetermined grips comprise a fingertip grip and a palming.

17. The hand-holdable controller of claim 9, wherein the at least one motor is configured to provide one or more types of feedback selected from the following:

(i) vibrational feedback, (ii) resistance to rotation of the rotatable knob, or (iii) rotational feedback by applying a torque-generating force to the rotatable knob.

18. The hand-holdable controller of claim 17, wherein the at least one motor is configured to provide a return to center function for the rotatable knob.

19. The hand-holdable controller of claim 17, wherein the at least one motor is configured to provide a limited rotation function for the rotatable knob.

20. The hand-holdable controller of claim 17, wherein the at least one motor is configured to provide a maximum speed of rotation for the rotatable knob.

21. The hand-holdable controller of claim 17, wherein the at least one motor is configured to provide a threshold amount of force required to rotate the rotatable knob.

22. A method comprising:

detecting, by a hand-holdable controller, touch data generated by one or more touch sensors, wherein the hand-holdable controller comprises a rotatable knob, and wherein the one or more touch sensors are arranged on the hand-holdable controller to generate touch data that provides a basis for distinguishing between two or more grips on the rotatable knob;

receiving, by the hand-holdable controller, operating mode data for the control knob;

operating at least one motor to affect the rotation of the control knob in accordance with the operating mode data, based at least in part on the detected touch data;

based on a comparison of the touch data to predetermined grip data that specifies parameters for detection of a plurality of grips on the rotatable knob, detect when each of the plurality of grips occurs on the rotatable knob, wherein the plurality of grips comprises at least a first and second grip;

detecting a first rotation of the rotatable knob;

when the first rotation is detected in association with detection of the first grip, generating first command data for a robotic device; and

when the first rotation is detected in association with detection of the second grip, generating second command data for the robotic device, wherein the second command data is different from the first command data.

23. The method of claim 22, further comprising:

detecting surface location data on one or more surfaces of the control knob, wherein the surface location data indicates location information for a touch gesture.

24. The method of claim 22, further comprising:

varying interpretation of a touch gesture received via the control knob, based on detected surface location data, such that a corresponding parameter is greater in magnitude when a detected surface location is closer to a proximate end of the hand-holdable controller.