Virtual reality / augmented reality handheld controller sensing

• US 10,599,259 B2
• Filed: 08/24/2018
• Issued: 03/24/2020
• Est. Priority Date: 11/20/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Mukherjee, Debanjan, Tome, Basheer, Bhattacharya, Sarnab, Vadathavoor, Bhaskar, Hasan, Shiblee Imtiaz, Li, Chun Yat Frank
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US16/111,702
• Publication Number:
  US 20190155439A1
• Time in Patent Office: 578 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04144 using an array of force sensing means position sensing using the local deformation of sensor cells G06F3/0447
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0443 using a single layer of sensing electrodes
• CPC Class Codes: G06F 3/046 by electromagnetic means

Abstract

A method that includes employing several sensors associated with a handheld controller, where each of the sensors is made of one of a hover, touch, and force/pressure sensor, and generating, by one or more of the sensors, sensor data associated with the position of a user'"'"'s hand and finger in relation to the handheld controller. The method continues with combining the sensor data from several sensors to form aggregate sensor data, sending the aggregate sensor data to a processor, and generating an estimated position of the user'"'"'s hand and fingers based on the aggregate sensor data.

Claims

1. A method comprising:

employing a plurality of sensors on a handheld controller,

wherein each of the plurality of sensors comprises one of a hover, touch, and force/pressure sensor, and

wherein the plurality of sensors including at least one hover sensor, and at least one of a touch sensor and a force/pressure sensor;

generating concurrently, by the plurality of sensors, sensor data associated with the position of a user'"'"'s hand and finger in relation to the handheld controller;

combining the sensor data from the plurality of sensors to form aggregate sensor data;

sending the aggregate sensor data to a processor; and

generating an estimated position of the user'"'"'s hand and fingers in 3 dimensions based on the aggregate sensor data.

2. The method of claim 1, wherein at least one hover and touch sensor uses capacitive sensing to sense the position of the user'"'"'s hand and fingers.

3. The method of claim 1, wherein the force/pressure sensor uses inductive sensing to sense when the user'"'"'s hand and fingers are in contact with the force/pressure sensor.

4. The method of claim 1, wherein the plurality of sensors includes a capacitive sensor, and wherein the processor performs a baseline tracking procedure to establish baseline current and voltage values for the capacitive sensor by:

monitoring the capacitive sensor for a change in one or more of a current and a change in voltage;

compensating for a shift in a baseline capacitance by detecting a change to a baseline value for the current or change to a baseline value for the voltage;

performing a recalibration to reset the baseline current and the baseline voltage values of the capacitive sensor when the processor does not detect the presence of the user'"'"'s finger; and

disabling the recalibration when the user'"'"'s finger is near the capacitive sensor.

5. The method of claim 1, wherein the processor combines the output of force/pressure sensors with output from one of the hover and touch sensors to detect the presence of the user'"'"'s fingers.

6. The method of claim 1, further comprising a force/pressure sensor positioned on a trigger of the handheld controller, wherein the force/pressure sensor provides a continuous stream of sensor data indicative of a force the user'"'"'s hand and fingers applies to the trigger.

7. The method of claim 1, wherein at least one of the plurality of sensors employs haptic feedback to the user'"'"'s hand and fingers.

8. The method of claim 1, wherein the touch pad is partitioned into two or more regions, wherein each region comprises at least one of the plurality of sensors.

9. The method of claim 8, wherein the regions are hexagonal in shape.

10. A method comprising:

employing a plurality of sensors on a handheld controller,

wherein each of the plurality of sensors comprises one of a hover, touch, and force/pressure sensor, and

wherein the plurality of sensors including at least one hover sensor, and at least one of a touch sensor and a force/pressure sensor;

positioning the plurality of sensors in a pattern on a touch pad located on the handheld controller;

generating concurrently, by the plurality of sensors, sensor data associated with the position of a user'"'"'s hand and finger in relation to the handheld controller;

combining the sensor data from the plurality of sensors to form aggregate sensor data;

sensing, by at least three of the plurality of sensors, contact by at least one of the user'"'"'s fingers;

sending the aggregate sensor data to a processor; and

calculating, by the processor, a position of the finger in 3 dimensions, wherein the position of the finger is determined by triangulation using the aggregate sensor data.

11. The method of claim 10, wherein each of the plurality of sensors uses one of capacitive sensing and inductive sensing.

12. The method of claim 10, wherein the algorithm comprises:

obtaining a set of measurements from N number of sensors of the plurality of sensors;

subtracting the median measurement value from each measurement;

identifying a set of (N−

1)/2 measurement values;

calculating the arithmetic mean values of the set of (N−

1)/2 measurement values; and

forming X_Position and Y_Position values from the arithmetic mean values.

13. The method of claim 10, further comprising a force/pressure sensor positioned on a trigger of the handheld controller, wherein the force/pressure sensor provides a continuous stream of sensor data indicative of a force the user'"'"'s hand and fingers applies to the trigger.

14. An apparatus comprising:

a handheld controller;

a touch pad located on the handheld controller;

a plurality of hover, touch, and force/pressure sensors positioned on the touch pad, wherein the sensors sense when a user'"'"'s hand and fingers is in contact with the sensors;

aggregate sensor data formed by the plurality of sensors, wherein the aggregate sensor data are comprised of sensor data generated concurrently from the plurality of sensors; and

a processor, using the aggregate sensor data, configured to;

generate an estimated position of the user'"'"'s hand and fingers in 3 dimensions based on the aggregate sensor data.

15. The apparatus of claim 14, wherein each of the plurality of sensors uses one of capacitive sensing and inductive sensing.

16. The apparatus of claim 14, further comprising a grip of the handheld controller employing at least one force/pressure sensor.

17. The apparatus of claim 14, wherein at least one of the plurality of sensors employs haptic feedback to the user'"'"'s hand and fingers.

18. The apparatus of claim 14, wherein the touch pad enables at least three sensors to be in contact with the user'"'"'s fingers, and wherein the at least three sensors are used to triangulate a position of at least one finger of the user'"'"'s fingers.

19. The apparatus of claim 14,

wherein the plurality of sensors positioned on the touch pad is comprised of one or more capacitive sensors arranged in a double hexagonal pattern, and

wherein the double hexagonal pattern is comprised of an outer hexagon and an inner hexagon in a concentric arrangement, and

wherein one of the plurality of sensors is located at a center of the double hexagonal pattern.

20. The apparatus of claim 19, wherein the aggregate sensor data are formed by combining capacitance sensing measurements from at least one sensors associated with the outer hexagon and one sensor associated with the inner hexagon.