Calibration of grab detection

• US 10,582,038 B2
• Filed: 05/12/2016
• Issued: 03/03/2020
• Est. Priority Date: 07/12/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Hughes, Charles J., Maguire, Yael G., Shirinfar, Shafigh, Toksvig, Michael John McKenzie
• Primary Examiner(s): Rivera Vargas, Manuel A
• Assistant Examiner(s): Perez Bermudez, Yaritza H
• Application Number:
  US15/153,547
• Publication Number:
  US 20160261734A1
• Time in Patent Office: 1,391 Days
• CPC Class Codes: G01D 18/00 Testing or calibrating apparatus or arrangements provided for in groups G01D1/00 - G01D15/00
• CPC Class Codes: G01D 21/00 Measuring or testing not otherwise provided for
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3215 Monitoring of peripheral devices
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 11/3013 where the computing system is an embedded system, i.e. a combination of hardware and software dedicated to perform a certain function in mobile devices, printers, automotive or aircraft systems testing or monitoring of control systems or parts thereof G05B23/02
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/0339 Touch strips, e.g. orthogonal touch strips to control cursor movement or scrolling; single touch strip to adjust parameter or to implement a row of soft keys
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 : using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: H04M 1/724 : User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72454 : according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 : including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

In one embodiment, a method includes receiving sensor data from a number of sensors of the computing device. The sensor data corresponds to a particular state associated with the computing device. The method also includes determining a vector corresponding to the received sensor data; and comparing the vector with a pre-determined hyperplane. The hyperplane corresponds to a number of imminent uses of the computing device. The method also includes determining whether the particular state associated with the computing device corresponds to a particular one of the imminent uses of the computing device based on the comparison of the vector with the pre-determined hyperplane.

Claims

1. A method comprising:

by a computing device, receiving sensor data from a plurality of sensors of the computing device, the sensor data corresponding to a particular state associated with the computing device;

by the computing device, applying a convolution operation to the sensor data from the plurality of sensors to determine whether the sensor data chronologically overlap with one another;

by the computing device, determining a vector corresponding to the received sensor data;

by the computing device, comparing the vector with a pre-determined hyperplane, wherein the hyperplane separates a first set of vectors corresponding to a first state representing physical contact of a user with the computing device and a second set of vectors corresponding to a second state representing no physical contact of the user with the computing device, wherein comparing the vector with the pre-determined hyperplane comprises determining a position of the vector relative to the pre-determined hyperplane;

by the computing device, classifying the vector into the first state or the second state based on the position of the vector relative to the pre-determined hyperplane; and

by the computing device, determining a particular imminent use of the computing device based on the classification of the vector and the sensor data of multiple sensors chronologically overlapping.

2. The method of claim 1, further comprising receiving data defining the hyperplane, wherein the hyperplane is determined by classifying sensor data from a plurality of computing devices.

3. The method of claim 1, further comprising receiving updated data re-defining the hyperplane based at least in part on updated sensor data from a plurality of computing devices.

4. The method of claim 1, further comprising calculating a derivative of the received sensor data from each sensor to detect a transition from the first state to the second state in the sensor data as a function of time.

5. The method of claim 1, wherein the vector comprises a tuple of the received sensor data.

6. The method of claim 1, wherein the classification of the vector into the first state or the second state comprises determining whether the position of the vector is on a same side of the hyperplane as the first set of vectors or the second set of vectors.

7. The method of claim 1, wherein the first set of vectors and the second set of vectors are part of training data, wherein the training data comprises an array of values from sensors of a plurality of computing devices and a state value corresponding to one of imminent uses of each computing device.

8. The method of claim 1, further comprising processing the received sensor data using the convolution operation.

9. The method of claim 1, wherein the hyperplane is generated using a machine-learning algorithm comprising a support-vector machine (SVM), k-means, Bayesian inference, or neural network algorithm.

10. The method of claim 1, wherein the hyperplane is determined by calculating a plurality of support vectors that each represent sensor data corresponding to the particular imminent use.

11. The method of claim 10, wherein the hyperplane is defined by a set of points with a constant dot product with the support vector representing the particular imminent use.

12. The method of claim 1, further comprising determining whether the vector is a non-event.

13. The method of claim 1, further comprising initiating a pre-determined function of the computing device based on the determination of the particular imminent use.

14. The method of claim 13, wherein the pre-determined function comprises powering on the computing device, unlocking the computing device, or bringing the computing device out of a power-saving state.

15. The method of claim 1, wherein:

the computing device is a mobile computing device; and

the particular imminent use corresponds to physical contact between the user and the computing device.

16. The method of claim 1, wherein one or more of the sensors comprises a touch sensor, gyroscope, accelerometer, optical proximity sensor, or ambient light sensor.

17. The method of claim 1, wherein the first set of vectors are determined from a first set of sensor data corresponding to activity associated with physical contact with the computing device and the second set of vectors are determined from a second set of sensor data corresponding to activity not associated with physical contact with the computing device.

18. The method of claim 1, wherein a dimension of the vector corresponds to a number of the plurality of sensors.

19. One or more computer-readable non-transitory storage media embodying software configured when executed to:

receive sensor data from a plurality of sensors of the computing device, the sensor data corresponding to a particular state associated with the computing device;

apply a convolution operation to the sensor data from the plurality of sensors to determine whether the sensor data chronologically overlap with one another;

determine a vector corresponding to the received sensor data;

compare the vector with a pre-determined hyperplane, wherein the hyperplane separates a first set of vectors corresponding to a first state representing physical contact of a user with the computing device and a second set of vectors corresponding to a second state representing no physical contact of the user with the computing device, wherein comparing the vector with the pre-determined hyperplane comprises determining a position of the vector relative to the pre-determined hyperplane

classify the vector into the first state or the second state based on the position of the vector relative to the pre-determined hyperplane; and

determine a particular imminent use of the computing device based on the classification of the vector and the sensor data of multiple sensors chronologically overlapping.

20. A device comprising:

a processor; and

one or more computer-readable non-transitory storage media coupled to the processor and embodying software that;

receive sensor data from a plurality of sensors of the computing device, the sensor data corresponding to a particular state associated with the computing device;

apply a convolution operation to the sensor data from the plurality of sensors to determine whether the sensor data chronologically overlap with one another;

determine a vector corresponding to the received sensor data;

compare the vector with a pre-determined hyperplane, wherein the hyperplane separates a first set of vectors corresponding to a first state representing physical contact of a user with the computing device and a second set of vectors corresponding to a second state representing no physical contact of the user with the computing device, wherein comparing the vector with the pre-determined hyperplane comprises determining a position of the vector relative to the pre-determined hyperplane

classify the vector into the first state or the second state based on the position of the vector relative to the pre-determined hyperplane; and

determine a particular imminent use of the computing device based on the classification of the vector and the sensor data of multiple sensors chronologically overlapping.