Personalized operation of a mobile device using sensor signatures

• US 10,353,495 B2
• Filed: 04/13/2016
• Issued: 07/16/2019
• Est. Priority Date: 08/20/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Nemala, Sridhar Krishna, Vitus, Deborah Kathleen, Klein, David, Medina, Eitan Asher
• Primary Examiner(s): Dragoescu, Claudia
• Application Number:
  US15/098,177
• Publication Number:
  US 20160231830A1
• Time in Patent Office: 1,189 Days
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise

Abstract

Systems and methods for performing personalized operations of a mobile device are provided. An example method includes determining a sensor signature has been received. The sensor signature may be a combination of acoustic and non-acoustic inputs. A classification or recognition or matching score for the received sensor signature may be calculated. A context associated with one or more of the operations of the mobile device may be determined. A confidence level or weight may be determined for the acoustic and non-acoustic inputs based on the context. Based on the confidence level or weight, a determination may be made whether to perform the one or more operations. The determination whether to perform the one or more operations may be further based on a combination of the confidence level or weight and the score. Determining the context can include identifying the type and level of noise associated with the mobile device'"'"'s environment.

Claims

1. A method for operation of a mobile device, the method comprising:

determining that a sensor signature has been received, the sensor signature being a combination of at least one acoustic input and at least one non-acoustic input;

determining a context associated with one or more operations of the mobile device;

determining a confidence level or weight for the at least one acoustic input and the at least one non-acoustic input based on the context;

determining, based on the confidence level or weight, whether to perform the one or more operations of the mobile device; and

initiating performance of the one or more operations of the mobile device associated with the sensor signature, if the confidence level or weight exceeds a predetermined threshold,

wherein the determining that the sensor signature has been received comprises deriving models using machine learning based on actions associated with a particular user, the determination of receipt of the sensor signature for the particular user being based on the machine learning derived models.

2. The method of claim 1, the method further comprising calculating a score for the received sensor signatures;

wherein the determining whether to perform the one or more operations of the mobile device is further based on the score.

3. The method of claim 1, wherein determining the context includes identifying the type and level of noise associated with an environment of the mobile device.

4. The method of claim 1, wherein the sensor signature is based on the at least one acoustic input and the at least one non-acoustic input being received sequentially.

5. The method of claim 1, wherein the sensor signature is based on the at least one acoustic input and the at least one non-acoustic input being received simultaneously.

6. The method of claim 1, wherein the at least one acoustic input includes a voice sound, the voice sound being captured by at least one microphone of the mobile device.

7. The method of claim 1, wherein the at least one non-acoustic input is detected by one or more sensors associated with the mobile device.

8. The method of claim 7, wherein the one or more sensors include one or more of the following:

an accelerometer, a gyroscope, a magnetometer, an image sensor, and a Global Positioning System (GPS).

9. The method of claim 1, wherein the at least one non-acoustic input includes a determination of one or more motions of the mobile device, the determination of the one or more motions of the mobile device comprising a measurement of the one or more motions of the mobile device.

10. The method of claim 9, wherein the one or more motions include one or more of the following:

vibrations of the mobile device due to being held by a hand, vibrations of the mobile device due to tapping on the mobile device, one or more rotations of the mobile device, and movement of the mobile device to draw a figure in a space.

11. The method of claim 2, wherein the score is a recognition or matching score.

17. The method of claim 1, wherein determining whether to perform the one or more operations of the mobile device includes:

comparing the received sensor signature to a user-defined sensor signature associated with the one or more operations of the mobile device;

applying the determined confidence level or weight based on the context to the at least one acoustic input and the at least one non-acoustic input; and

computing a score that indicates how closely the at least one acoustic input and the at least one non-acoustic input of the received sensor signature matches the at least one acoustic input and the at least one non-acoustic input of the user-defined sensor signature in view of the applied determined confidence level or weight.

12. A system for operation of a mobile device, the system comprising:

a processor; and

a memory communicatively coupled with the processor, the memory storing instructions which, when executed by the processor, perform a method comprising;

determining that a sensor signature has been received, the sensor signature being a combination of at least one acoustic input and at least one nonacoustic input;

determining a context associated with one or more operations of the mobile device;

determining a confidence level or weight for the at least one acoustic input and the at least one non-acoustic input based on the context; and

determining, based on the confidence level or weight, whether to perform the one or more operations of the mobile device;

initiating performance of the one or more operations of the mobile device associated with the sensor signature, if the confidence level or weight exceeds a predetermined threshold; and

calculating a classification or recognition or matching score for the received sensor signatures,

wherein the determining whether to perform the one or more operations of the mobile device is further based on the classification or recognition or matching score.

13. The system of claim 12, wherein determining the context includes identifying the type and level of noise associated with an environment of the mobile device.

14. The system of claim 12, wherein the sensor signature is based on the at least one acoustic input and the at least one non-acoustic input being received sequentially or simultaneously.

15. The system of claim 12, wherein the at least one acoustic input includes a voice sound, the voice sound being captured by at least one microphone of the mobile device;

and wherein the at least one non-acoustic input includes a determination of one or more motions of the mobile device being detected by one or more sensors associated with the mobile device, the one or more sensors including one or more of the following;

an accelerometer, a gyroscope, a magnetometer, an image sensor, and a Global Positioning System (GPS).

16. A non-transitory computer-readable storage medium having embodied thereon instructions, which, when executed by one or more processors, perform steps of a method, the method comprising:

determining that a sensor signature has been received, the sensor signature being a combination of at least one acoustic input and at least one non-acoustic input;

determining a context associated with one or more operations of the mobile device;

determining a confidence level or weight for the at least one acoustic input and the at least one non-acoustic input based on the context;

determining, based on the confidence level or weight, whether to perform the one or more operations of the mobile device;

initiating performance of the one or more operations of the mobile device associated with the sensor signature, if the confidence level or weight exceeds a predetermined threshold; and

calculating a score for the received sensor signatures,

wherein the determining whether to perform the one or more operations of the mobile device is further based on the score.