Sensor fusion algorithms for a handheld controller that includes a force sensing resistor (FSR)

• US 10,691,233 B2
• Filed: 05/18/2018
• Issued: 06/23/2020
• Est. Priority Date: 10/11/2016
• Status: Active Grant

• Current Assignee: Valve Corporation
• Original Assignee: Valve Corporation
• Inventors: Dalton, Scott, Bellinghausen, Jeffrey Peter, Nietfeld, Scott Douglas, Leinbaugh, Jeffrey George, Campbell, Ian, Yoo, Cheang Tad, Yang, Lawrence, Mucha, Jeffrey Walter
• Primary Examiner(s): Cleary, Thomas J.
• Application Number:
  US15/984,245
• Publication Number:
  US 20180264357A1
• Time in Patent Office: 767 Days
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: G06F 2203/04103 Manufacturing, i.e. details related to manufacturing processes specially suited for touch sensitive devices
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position

Abstract

Logic of a handheld controller can implement sensor fusion algorithms based on force data provided by a force sensing resistor (FSR) in combination with touch data or proximity data provided by a touch sensor or an array of proximity sensors, respectively. An example sensor fusion algorithm can be used to re-calibrate the FSR when an object contacts an associated control, as detected by the touch sensor. Another example sensor fusion algorithm can be used to ignore spurious inputs detected by the FSR when an object is in contact with an adjacent control. Another example sensor fusion algorithm can be used to detect a hand size of a hand grasping a handle of the controller, as detected by the array of proximity sensors, and to adjust the threshold force to register a FSR input event at the FSR according to the hand size.

Claims

1. A system comprising:

one or more processors;

a handheld controller comprising a controller body, the controller body including;

a handle configured to be grasped by a hand;

a head coupled to the handle at a neck region;

at least one control on the head of the controller body that is configured to be pressed by a thumb of the hand;

a touch sensor mounted within the head of the controller body, positioned underneath the at least one control, and configured to provide, to the one or more processors, touch data indicative of the thumb contacting the at least one control; and

a force sensing resistor (FSR) mounted within the head of the controller body, positioned underneath the at least one control, and configured to provide, to the one or more processors, force data indicative of an amount of force of a press of the at least one control; and

logic configured to;

determine, based at least in part on the touch data provided by the touch sensor, that the thumb has come into contact with the at least one control;

determine, based at least in part on the force data provided by the FSR at a time at which the thumb has come into contact with the at least one control, a resistance value measured by the FSR; and

correlate the resistance value with a digitized FSR input value of zero.

2. The system of claim 1, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after the time at which the thumb has come into contact with the at least one control, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event; and

register the FSR input event based at least in part on the second digitized FSR input value meeting or exceeding the threshold value.

3. The system of claim 2, wherein registering the FSR input event activates a binding associated with the at least one control, and wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value is less than the second digitized FSR input value; and

deactivate the binding based at least in part on the third digitized FSR input value being less than the second digitized FSR input value.

4. The system of claim 1, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after the time at which the thumb has come into contact with the at least one control, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a first threshold value associated with a first FSR input event;

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the first threshold value;

determine that a difference between the second time and the first time is less than a predefined time period;

refrain from registering the first FSR input event; and

register the second FSR input event.

5. The system of claim 1, wherein:

the at least one control is a first control;

the controller body further includes;

a second control on the head of the controller body adjacent to the first control, the second control configured to be pressed by the thumb; and

a second FSR mounted within the head of the controller body, positioned underneath the second control, and configured to provide, to the one or more processors, second force data indicative of an amount of force of a press of the second control; and

the logic is further configured to, after the time at which the thumb has come into contact with the first control;

determine, based at least in part on the second force data provided by the second FSR, a second resistance value measured by the second FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the second control;

determine, based at least in part on the touch data provided by the touch sensor at a time at which the second resistance value is measured by the second FSR, that the thumb is in contact with the first control; and

refrain from registering the FSR input event for the second control based at least in part on determining that the thumb is in contact with the first control.

6. The system of claim 1, wherein:

the at least one control is a first control;

the controller body further includes;

a second control on the head of the controller body adjacent to the first control, the second control configured to be pressed by the thumb; and

a second touch sensor mounted within the head of the controller body, positioned underneath the second control, and configured to provide, to the one or more processors, second touch data indicative of the thumb contacting the second control; and

the logic is further configured to, after the time at which the thumb has come into contact with the first control;

determine, based at least in part on the force data provided by the FSR, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the first control;

determine, based at least in part on the second touch data provided by the second touch sensor at a time at which the second resistance value is measured by the FSR, that the thumb is in contact with the second control; and

refrain from registering the FSR input event for the first control based at least in part on determining that the thumb is in contact with the second control.

7. The system of claim 1, wherein the system comprises a virtual reality (VR) gaming system.

8. A system comprising:

one or more processors;

a handheld controller comprising a controller body, the controller body including;

a handle configured to be grasped by a hand;

a head coupled to the handle at a neck region;

a first control on the head of the controller body that is configured to be pressed by a thumb of the hand;

a second control on the head of the controller body adjacent to the first control, the second control configured to be pressed by the thumb;

a force sensing resistor (FSR) mounted within the head of the controller body, positioned underneath the first control, and configured to provide, to the one or more processors, force data indicative of an amount of force of a press of the first control; and

a touch sensor mounted within the head of the controller body, positioned underneath the second control, and configured to provide, to the one or more processors, touch data indicative of the thumb contacting the second control; and

logic configured to;

determine, based at least in part on the force data provided by the FSR, a resistance value measured by the FSR;

convert the resistance value to a digitized FSR input value;

determine that the digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the first control;

determine, based at least in part on the touch data provided by the touch sensor at a time at which the resistance value is measured by the FSR, that the thumb is in contact with the second control; and

refrain from registering the FSR input event for the first control based at least in part on determining that the thumb is in contact with the second control.

9. The system of claim 8, wherein the first control and the second control are disposed within a threshold distance from each other on the head of the controller body.

10. The system of claim 8, wherein the logic is further configured to:

determine, based at least in part on the touch data provided by the touch sensor at a first time after the time at which the resistance value is measured by the FSR, that the thumb is no longer in contact with the second control;

determine, based at least in part on the force data provided by the FSR at the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds the threshold value; and

register the FSR input event for the first control based at least in part on the second digitized FSR input value meeting or exceeding the threshold value.

11. The system, of claim 10, wherein registering the FSR input event activates a binding associated with the first control, and wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value is less than the second digitized FSR input value; and

deactivate the binding based at least in part on the third digitized FSR input value being less than the second digitized FSR input value.

12. The system of claim 8, wherein the FSR input event is a first FSR input event, the threshold value is a first threshold value, and the logic is further configured to:

determine, based at least in part on the touch data provided by the touch sensor at a first time after the time at which the resistance value is measured by the FSR, that the thumb is no longer in contact with the second control;

determine, based at least in part on the force data provided by the FSR at the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds the first threshold value associated with the first FSR input event;

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the first threshold value;

determine that a difference between the second time and the first time is less than a predefined time period;

refrain from registering the first FSR input event; and

register the second FSR input event.

13. The system of claim 8, wherein:

the controller body further includes a second FSR mounted within the head of the controller body, positioned underneath the second control, and configured to provide, to the one or more processors, second force data indicative of an amount of force of a press of the second control; and

the logic is further configured to, prior to the time at which the resistance value is measured by the FSR;

determine, based at least in part on the touch data provided by the touch sensor, that the thumb has come into contact with the second control;

determine, based at least in part on the second force data provided by the second FSR at a time at which the thumb has come into contact with the second control, a second resistance value measured by the second FSR; and

correlate the second resistance value with a digitized FSR input value of zero.

14. The system of claim 8, wherein:

the controller body further includes a second touch sensor mounted within the head of the controller body, positioned underneath the first control, and configured to provide, to the one or more processors, second touch data indicative of the thumb contacting the first control; and

the logic is further configured to, prior to the time at which the resistance value is measured by the FSR;

determine, based at least in part on the second touch data provided by the second touch sensor, that the thumb has come into contact with the first control;

determine, based at least in part on the force data provided by the FSR at a time at which the thumb has come into contact with the first control, a second resistance value measured by the FSR; and

correlate the second resistance value with a digitized FSR input value of zero.

15. The system of claim 8, wherein the system comprises a virtual reality (VR) gaming system.

16. A method comprising:

determining, based at least in part on touch data provided by a touch sensor, that a thumb of a hand has come into contact with a first control on a head of a controller body of a handheld controller, wherein the head of the controller body is coupled to a handle of the controller body at a neck region, and wherein the touch sensor is mounted within the head of the controller body and is positioned underneath the first control;

determining, based at least in part on force data provided by a force sensing resistor (FSR) at a first time at which the thumb has come into contact with the first control, a resistance value measured by the FSR, wherein the FSR is mounted within the head of the controller body and is positioned underneath the first control; and

correlating the resistance value with a digitized FSR input value of zero.

17. The method of claim 16, further comprising:

determining, based at least in part on second force data provided by a second FSR at a second time after the first time, a second resistance value measured by the second FSR, wherein the second FSR is mounted within the head of the controller body and is positioned underneath a second control on the head of the controller body adjacent to the first control;

converting the second resistance value to a second digitized FSR input value;

determining that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the second control;

determining, based at least in part on the touch data provided by the touch sensor at the second time, that the thumb is in contact with the first control; and

refraining from registering the FSR input event for the second control based at least in part on determining that the thumb is in contact with the first control.

18. The method of claim 16, further comprising:

determining, based at least in part on the force data provided by the FSR at a second time after the first time, a second resistance value measured by the FSR;

converting the second resistance value to a second digitized FSR input value;

determining that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the first control;

determining, based at least in part on second touch data provided by a second touch sensor at the second time, that the thumb is in contact with a second control on the head of the controller body, wherein the second control is adjacent to the first control, and wherein the second touch sensor is mounted within the head of the controller body and is positioned underneath the second control; and

refraining from registering the FSR input event for the first control based at least in part on determining that the thumb is in contact with the second control.

19. The method of claim 16, further comprising:

determining, based at least in part on the force data provided by the FSR at a second time after the first time, a second resistance value measured by the FSR;

converting the second resistance value to a second digitized FSR input value;

determining that the second digitized FSR input value meets or exceeds a first threshold value associated with a first FSR input event;

determining, based at least in part on the force data provided by the FSR at a third time after the second time, a third resistance value measured by the FSR;

converting the third resistance value to a third digitized FSR input value;

determining that the third digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the first threshold value;

determining that a difference between the third time and the second time is less than a predefined time period;

refraining from registering the first FSR input event; and

registering the second FSR input event.

20. The method of claim 16, wherein the handheld controller is usable with a virtual reality (VR) gaming system.