Embedded grasp sensing devices, systems, and methods

• US 10,289,192 B2
• Filed: 02/29/2016
• Issued: 05/14/2019
• Est. Priority Date: 03/01/2015
• Status: Active Grant

• Current Assignee: Tactical Haptics
• Original Assignee: Tactical Haptics
• Inventors: Provancher, William R., Jones, Michael P.
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US15/554,668
• Publication Number:
  US 20180067545A1
• Time in Patent Office: 1,170 Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 2203/0339 Touch strips, e.g. orthogonal touch strips to control cursor movement or scrolling; single touch strip to adjust parameter or to implement a row of soft keys
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour

Abstract

Implementations of the present invention relate to apparatuses, systems, and methods for measuring the grasp state, position, or hand pose of a user'"'"'s hand relative to a device, which could be used as an input for human-machine input, augmented reality, virtual reality, video games, or tele-manipulation. The device may include a plurality of grasp sensors embedded within the device. A kinematic hand model can be used in combination with grasp sensing data to estimate a user'"'"'s hand pose.

Claims

1. A device for sensing grasp state or position of a user'"'"'s hand, the device comprising:

a housing; and

a plurality of proximity sensing elements housed within or on the device configured to sense grasp state or position of the user'"'"'s hand prior to contact with the device,

wherein;

the plurality of proximity sensing elements are distributed non-uniformly within or on the device,

the plurality of proximity sensing elements comprise a first sensor that is sensing a first distance between a user'"'"'s index finger and the first sensor, wherein the first sensor is positioned on the device in a location that is configured to specifically sense the user'"'"'s index finger, and

the plurality of proximity sensing elements comprise a second sensor that is sensing a second distance between a user'"'"'s middle finger and the second sensor, wherein the second sensor is positioned on the device in a location that is configured to specifically sense the user'"'"'s middle finger.

2. The device of claim 1, wherein the proximity sensing elements are optical emitter-detector pairs.

3. The device of claim 1, wherein the proximity sensing elements are capacitive sensors.

4. The device of claim 3, wherein the capacitive sensors form an array that is placed within the device and are incorporated into a flexible printed circuit board.

5. The device of claim 1, wherein the proximity sensing elements measure a user'"'"'s grasp force on the device.

6. The device of claim 1, wherein:

a first proximity sensing element is positioned within or on the device such that it senses a grasp state of position of a user'"'"'s thumb; and

a second proximity sensing element is positioned within or on the device such that it senses a grasp state of position of a user'"'"'s index finger.

7. The device of claim 6, wherein the proximity sensing elements associated with a user'"'"'s ring finger, and little finger are relatively larger than proximity sensing elements associated with the user'"'"'s thumb and the user'"'"'s index finger.

8. The device of claim 1, wherein the proximity sensing elements measure the distance between the surface and the user'"'"'s hand at each respective sensor location.

9. The device of claim 1, wherein the proximity sensing elements can be used to estimate the grasp force after the user'"'"'s hand contacts the device.

10. The device of claim 9, wherein the surface of the device is deformable to enhance the ability to deduce grasp force after contact.

11. A device for sensing grasp state or position of a user'"'"'s hand, the device comprising:

a housing;

a plurality of proximity sensing elements housed within or on the device configured to sense a distance of a user'"'"'s hand prior to contact with the device at a plurality of locations;

wherein;

each location of the plurality of locations comprises one or more proximity sensing elements, and

a first location comprises one or more proximity sensing elements that are positioned to specifically detect a distance between the device and the user'"'"'s index finger, and

a second location comprises one or more proximity sensing elements that are position to specifically detect a distance between the device and the user'"'"'s middle finger.

12. The device of claim 11, wherein the first location is configured to detect a grasp state or position of a user'"'"'s index finger and the second location is configured to detect a grasp state or position of the user'"'"'s thumb.

13. The device of claim 11, wherein the proximity sensing elements are capacitive sensors.

14. A method for estimating hand pose relative to a sensorized device, the method including:

measuring hand proximity prior to contact with the sensorized device from multiple proximity sensing elements, wherein measuring hand proximity comprises;

receiving a first sensor reading from a first sensor that is sensing a first distance between a user'"'"'s index finger and the first sensor, wherein the first sensor is positioned on a sensorized device in a location that is configured to specifically sense the user'"'"'s index finger; and

receiving a second sensor reading from a second sensor that is sensing a second distance between a user'"'"'s middle finger and the second sensor, wherein the second sensor is positioned on the sensorized device in a location that is configured to specifically sense the user'"'"'s middle finger.

15. The method of claim 14, further comprising incorporating a kinematic hand model to improve the estimation of a user'"'"'s hand pose.

16. The method of claim 14, wherein the sensorized device includes geometric features that are used on the device to locate the user'"'"'s hand in a known location to simplify the prediction model for estimating a user'"'"'s hand pose.