Sensor Fusion Algorithms for a Handheld Controller that Includes a Force Sensing Resistor (FSR)

• US 20180264357A1
• Filed: 05/18/2018
• Published: 09/20/2018
• Est. Priority Date: 10/11/2016
• Status: Active Grant

• Current Assignee: Valve Corporation
• Original Assignee: Valve Corporation
• Inventors: Dalton, Scott, Bellinghausen, Jeffrey Peter, Nietfeld, Scott Douglas, Leinbaugh, Jeffrey George, Campbell, Ian, Yoo, Cheang Tad, Yang, Lawrence, Mucha, Jeffrey Walter
• Granted Patent: US 10,691,233 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: G06F 2203/04103 Manufacturing, i.e. details related to manufacturing processes specially suited for touch sensitive devices
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position

Abstract

Logic of a handheld controller can implement sensor fusion algorithms based on force data provided by a force sensing resistor (FSR) in combination with touch data or proximity data provided by a touch sensor or an array of proximity sensors, respectively. An example sensor fusion algorithm can be used to re-calibrate the FSR when an object contacts an associated control, as detected by the touch sensor. Another example sensor fusion algorithm can be used to ignore spurious inputs detected by the FSR when an object is in contact with an adjacent control. Another example sensor fusion algorithm can be used to detect a hand size of a hand grasping a handle of the controller, as detected by the array of proximity sensors, and to adjust the threshold force to register a FSR input event at the FSR according to the hand size.

Claims

1. A handheld controller comprising:

one or more processors;

a controller body including at least one control that is configured to be pressed;

a touch sensor associated with the at least one control and configured to provide, to the one or more processors, touch data indicative of an object contacting the at least one control;

a force sensing resistor (FSR) associated with the at least one control and configured to provide, to the one or more processors, force data indicative of an amount of force of a press of the at least one control; and

logic configured to;

determine, based at least in part on the touch data provided by the touch sensor, that the object has come into contact with the at least one control;

determine, based at least in part on the force data provided by the FSR at a time at which the object has come into contact with the at least one control, a resistance value measured by the FSR; and

correlate the resistance value with a digitized FSR input value of zero.

2. The handheld controller of claim 1, wherein the object is a finger or a thumb, and wherein the at least one control is disposed on a head of the controller body and is configured to be pressed by the finger or the thumb.

3. The handheld controller of claim 1, wherein the object is a portion of a hand, and wherein the at least one control comprises a handle of the controller body and is configured to be squeezed by the hand.

4. The handheld controller of claim 1, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after the time at which the object has come into contact with the at least one control, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event; and

register the FSR input event based at least in part on the second digitized FSR input value meeting or exceeding the threshold value.

5. The handheld controller of claim 4, wherein registering the FSR input event activates a binding associated with the at least one control, and wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value is less than the second digitized FSR input value; and

deactivate the binding based at least in part on the third digitized FSR input value being less than the second digitized FSR input value.

6. The handheld controller of claim 1, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after the time at which the object has come into contact with the at least one control, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a first threshold value associated with a first FSR input event;

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the first threshold value;

determine that a difference between the second time and the first time is less than a predefined time period;

refrain from registering the first FSR input event; and

register the second FSR input event.

7. A handheld controller comprising:

one or more processors;

a controller body including a first control and a second control adjacent to the first control, the first control and the second control configured to be pressed;

a force sensing resistor (FSR) associated with the first control and configured to provide, to the one or more processors, force data indicative of an amount of force of a press of the first control;

a touch sensor associated with the second control and configured to provide, to the one or more processors, touch data indicative an object contacting the second control; and

logic configured to;

determine, based at least in part on the force data provided by the FSR, a resistance value measured by the FSR;

convert the resistance value to a digitized FSR input value;

determine that the digitized FSR input value meets or exceeds a threshold value that is to be met in order to register a FSR input event for the first control;

determine, based at least in part on the touch data provided by the touch sensor at a time at which the FSR resistance value is measured by the FSR, that the object is in contact with the second control; and

refrain from registering the FSR input event for the first control based at least in part on determining that the object is in contact with the second control.

8. The handheld controller of claim 7, wherein the object is a finger or a thumb, and wherein the first control and the second control are disposed within a threshold distance from each other on a head of the controller body, the first control and the second control being configured to be pressed by the finger or the thumb.

9. The handheld controller of claim 7, wherein the logic is further configured to:

determine, based at least in part on the touch data provided by the touch sensor at a first time after the time at which the FSR resistance value is measured by the FSR, that the object is no longer in contact with the second control;

determine, based at least in part on the force data provided by the FSR at the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds the threshold value; and

register the FSR input event for the first control based at least in part on the second digitized FSR input value meeting or exceeding the threshold value.

10. The handheld controller of claim 9, wherein registering the FSR input event activates a binding associated with the first control, and wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value is less than the second digitized FSR input value; and

deactivate the binding based at least in part on the third digitized FSR input value being less than the second digitized FSR input value.

11. The handheld controller of claim 7, wherein the FSR input event is a first FSR input event, the threshold value is a first threshold value, and the logic is further configured to:

determine, based at least in part on the touch data provided by the touch sensor at a first time after the time at which the FSR resistance value is measured by the FSR, that the object is no longer in contact with the second control;

determine, based at least in part on the force data provided by the FSR at the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds the first threshold value associated with the first FSR input event;

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a third resistance value measured by the FSR;

convert the third resistance value to a third digitized FSR input value;

determine that the third digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the first threshold value;

determine that a difference between the second time and the first time is less than a predefined time period;

refrain from registering the first FSR input event; and

register the second FSR input event.

12. A handheld controller:

one or more processors;

a controller body including a handle;

a force sensing resistor (FSR) associated with the handle and configured to provide, to the one or more processors, force data indicative of an amount of force of a squeeze of the handle;

an array of proximity sensors that are spatially distributed on the handle, the array of proximity sensors configured to provide, to the one or more processors, proximity data indicative of a hand grasping the handle; and

logic configured to;

determine, based at least in part on the proximity data provided by the array of proximity sensors, a size, among a plurality of predefined sizes, of the hand; and

adjust, based at least in part on the size of the hand, a threshold value to an adjusted threshold value that is to be met in order to register a FSR input event for the handle, the adjusted threshold value corresponding to a particular amount of force of the squeeze of the handle.

13. The handheld controller of claim 12, wherein determining the size of the hand comprises:

determining a number of proximity sensors of the array of proximity sensors that provided the proximity data; and

determining the size of the hand based at least in part on the number of proximity sensors that provided the proximity data.

14. The handheld controller of claim 12, wherein:

the plurality of predefined sizes include a first size and a second size that is larger than the first size;

a default threshold value is associated with the first size or the second size; and

adjusting the threshold value comprises at least one of;

increasing the default threshold value to the adjusted threshold value associated with the second size, the adjusted threshold value being greater than the default threshold value;

or

decreasing the default threshold value to the adjusted threshold value associated with the first size, the adjusted threshold value being less than the default threshold value.

15. The handheld controller of claim 12, wherein:

the controller body includes at least one control that is configured to be pressed by a finger or a thumb;

the handheld controller further comprises a second FSR associated with the at least one control and configured to provide, to the one or more processors, second force data indicative of an amount of force of a press of the at least one control; and

the logic is further configured to adjust, based at least in part on the size of the hand, a second threshold value to a second adjusted threshold value that is to be met in order to register a second FSR input event for the at least one control, the second adjusted threshold value corresponding to a particular amount of force of the press of the at least one control.

16. The handheld controller of claim 15, wherein the at least one control is a thumb-operated control that is included on a head of the controller body and is configured to be pressed by the thumb.

17. The handheld controller of claim 12, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after adjusting the threshold value to the adjusted threshold value, a resistance value measured by the FSR;

convert the resistance value to a digitized FSR input value;

determine that the digitized FSR input value meets or exceeds the adjusted threshold value; and

register the FSR input event based at least in part on the digitized FSR input value meeting or exceeding the adjusted threshold value.

18. The handheld controller of claim 17, wherein registering the FSR input event activates a binding associated with the handle, and wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value is less than the digitized FSR input value; and

deactivate the binding based at least in part on the second digitized FSR input value being less than the digitized FSR input value.

19. The handheld controller of claim 12, wherein the logic is further configured to:

determine, based at least in part on the force data provided by the FSR at a first time after adjusting the threshold value to the adjusted threshold value, a resistance value measured by the FSR;

convert the resistance value to a first digitized FSR input value;

determine that the first digitized FSR input value meets or exceeds the adjusted threshold value associated with the FSR input event;

determine, based at least in part on the force data provided by the FSR at a second time after the first time, a second resistance value measured by the FSR;

convert the second resistance value to a second digitized FSR input value;

determine that the second digitized FSR input value meets or exceeds a second threshold value associated with a second FSR input event, the second threshold value being greater than the adjusted threshold value;

determine that a difference between the second time and the first time is less than a predefined time period;

refrain from registering the FSR input event; and

register the second FSR input event.

20. The handheld controller of claim 12, wherein the handheld controller is associated with a virtual reality (VR) gaming system.