Classifying contacts or associations with a touch sensitive device

• US 10,282,024 B2
• Filed: 02/01/2018
• Issued: 05/07/2019
• Est. Priority Date: 09/25/2014
• Status: Active Grant

• Current Assignee: Qeexo Co. (TDK Corporation)
• Original Assignee: Qeexo Co. (TDK Corporation)
• Inventors: Harrison, Chris, Stone, Joshua Dale
• Primary Examiner(s): Cerullo, Liliana
• Application Number:
  US15/886,562
• Publication Number:
  US 20180157371A1
• Time in Patent Office: 460 Days
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Techniques enabling improved classification of objects against a touch sensitive surface of a touch sensitive device are presented. An object classification component can analyze touch screen data associated with a touch sensitive surface associated with a device and motion information relating to motion along at least one axis associated with the device. The object classification component can determine a contact classification for an object with respect to the touch sensitive surface, based on a result(s) of the analyzing, and in response to determining a contact state of the object with respect to the touch sensitive surface. An event determination component can control a function associated with the touch sensitive surface based on the contact classification for the object, the function comprising enabling or disabling a touch sensitive function of the touch sensitive surface of a display screen of the device or switching the display screen on or off.

Claims

1. A method, comprising:

analyzing, by a system comprising a processor, characteristics of touch screen data associated with a touch sensitive surface that is associated with a device and a portion of motion data of at least one axis associated with the device, wherein a first amount of the motion data and a number of axes of the motion data to be included in the portion of the motion data is determined based at least in part on at least one of a second amount of time available to perform the analyzing, a third amount of resources available to perform the analyzing, or a confidence level that is associated with a preliminary contact classification for an object with respect to the touch sensitive surface and determined based at least in part on a preliminary analysis result of analyzing the touch screen data; and

based at least in part on at least one result of the analyzing, determining, by the system, a contact classification for the object with respect to the touch sensitive surface to determine a contact state of the object in relation to the touch sensitive surface.

2. The method of claim 1, wherein the touch screen data comprises at least one of contact data or capacitive data, wherein, when the touch screen data comprises the contact data, the contact data indicates whether the object is in contact or is not in contact with the touch sensitive surface, and wherein, when the touch screen data comprises the capacitive data, the capacitive data indicates whether the object is in contact with the touch sensitive surface, is not in contact with the touch sensitive surface, is hovering within a defined distance of the touch sensitive surface resulting in a change in a capacitance value associated with the touch sensitive surface, or is not hovering within the defined distance of the touch sensitive surface.

3. The method of claim 1, wherein there is a set of contact classifications, comprising the contact classification, relating to the contact state of the object in relation to the touch sensitive surface, wherein the set of contact classifications comprise a no touch state, a head state, a finger state, a head-hovering state, and a finger-hovering state, wherein the no touch state indicates the object is not in contact with and is not hovering in proximity to the touch sensitive surface, the head state indicates that the object is a face, a head, or an ear of a user and is in contact with the touch sensitive surface, the finger state indicates that the object is a finger or a hand of the user and is in contact with the touch sensitive surface, the head-hovering state indicates that the object is the face, the head, or the ear of the user and is hovering over the touch sensitive surface in proximity to the touch sensitive surface and is not in contact with the touch sensitive surface, and the finger-hovering state indicates that the object is the finger or the hand of the user and is hovering over the touch sensitive surface in proximity to the touch sensitive surface and is not in contact with the touch sensitive surface.

4. The method of claim 1, wherein the motion data comprises at least one of vibro-acoustic data associated with the device, inertial measurement unit data associated with the device, accelerometer data associated with the device, gyroscope data associated with the device, acceleration data indicating an acceleration of the device, velocity data indicating a velocity of the device, angular rate data indicating an angular rate of the device, position data indicating a position or a change in position of the device, or orientation data indicating an orientation or a change in orientation of the device.

5. The method of claim 1, wherein the motion data of the at least one axis associated with the device comprises motion data of two or more axes associated with one or more sources of the motion data.

6. The method of claim 1, further comprising receiving, by the system, the motion data from at least one of an accelerometer, a gyroscope, an inertial measurement unit, or a sensor.

7. The method of claim 1, further comprising determining whether the object is in contact with the touch sensitive surface or whether the object is hovering in proximity to the touch sensitive surface based at least in part on the at least one result of the analyzing.

8. The method of claim 1, further comprising controlling, by the system, a display function associated with the touch sensitive surface based at least in part on the contact classification for the object, wherein the display function comprises enabling a touch sensitive function of the touch sensitive surface of a display screen of the device, disabling the touch sensitive function of the touch sensitive surface, switching the display screen to an on state, or switching the display screen to an off state.

9. The method of claim 1, further comprising determining, by the system, one or more descriptive features that are indicative of the object for at least one subdivision of subdivisions associated with at least one of the touch screen data or the motion data, based at least in part on the at least one result of the analyzing.

10. The method of claim 9, wherein the determining the contact classification comprises determining the contact classification for the object in contact with the touch sensitive surface based at least in part on the one or more descriptive features, at least one location of the at least one subdivision, and the motion data.

11. The method of claim 9, further comprising:

generating, by the system, frame data representative of an image indicating one or more respective locations of contact or hovering of one or more respective portions of the object with the touch sensitive surface based at least in part on at least one of the touch screen data or the motion data; and

dividing, by the system, the frame data into respective portions of the frame data to generate respective subdivisions of the subdivisions, wherein the respective subdivisions comprise the at least one subdivision, wherein the at least one subdivision comprises a first subdivision associated with a first portion of the frame data and a second subdivision associated with a second portion of the frame data, wherein the first subdivision partially overlaps the second subdivision, and wherein the analyzing comprises analyzing the respective subdivisions to determine respective descriptive features of the respective subdivisions.

12. The method of claim 11, further comprising:

converting, by the system, the frame data into a frequency domain representation, wherein the subdivisions are defined in the frequency domain representation;

extracting, by the system, at least one of a spectral centroid, a spectral density, spherical harmonics, a total average spectral energy, or log spectral band ratios based at least in part on the frequency domain representation; and

based at least in part on the extracting, classifying, by the system, one or more respective contacts within one or more respective subdivisions of the subdivisions of the touch sensitive surface.

13. The method of claim 1, further comprising:

comparing, by the system, the contact classification for the object with previous contact classifications; and

determining, by the system, the contact state of the object with the touch sensitive surface based at least in part on a comparison result of the comparing.

14. A system, comprising:

a memory that stores executable components; and

a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising;

an object classification component configured to analyze touch surface information associated with a touch sensitive surface that is associated with a device and a portion of motion information of at least one axis associated with the device, wherein the object classification component is further configured to determine a contact classification for an object with respect to the touch sensitive surface, based at least in part on at least one result of the analyzing, to determine a contact state of the object with respect to the touch sensitive surface, and wherein a number of axes of the motion information to be included in the portion of the motion information is determined based at least in part on at least one of a first amount of time available to perform the analysis, a second amount of processing resources available to perform the analysis, or a confidence value that is associated with a preliminary contact classification for the object with respect to the touch sensitive surface and determined based at least in part on an intermediate analysis result of analyzing the touch surface information.

15. The system of claim 14, wherein the touch surface information comprises at least one of contact information or capacitive information, wherein the contact information indicates whether the object is in contact or is not in contact with the touch sensitive surface, and wherein the capacitive information indicates whether the object is in contact with the touch sensitive surface, is not in contact with the touch sensitive surface, is hovering in proximity to the touch sensitive surface resulting in a change in an amount of capacitance associated with the touch sensitive surface, or is not hovering in proximity to the touch sensitive surface.

16. The system of claim 14, wherein there is a set of contact classifications, comprising the contact classification, relating to the contact state of the object with respect to the touch sensitive surface, wherein the set of contact classifications comprise a no touch state, a head state, a finger state, a head-hovering state, and a finger-hovering state, wherein the no touch state indicates the object is not in contact with and is not hovering in proximity to the touch sensitive surface, the head state indicates that the object is a face, a head, or an ear of a user and is in contact with the touch sensitive surface, the finger state indicates that the object is a finger or a hand of the user and is in contact with the touch sensitive surface, the head-hovering state indicates that the object is the face, the head, or the ear of the user and is hovering over the touch sensitive surface in proximity to the touch sensitive surface and is not in contact with the touch sensitive surface, and the finger-hovering state indicates that the object is the finger or the hand of the user and is hovering over the touch sensitive surface in proximity to the touch sensitive surface and is not in contact with the touch sensitive surface.

17. The system of claim 14, wherein the motion information comprises at least one of vibro-acoustic information associated with the device, inertial measurement unit information associated with the device, accelerometer information associated with the device, gyroscope information associated with the device, acceleration information indicating an acceleration of the device, velocity information indicating a velocity of the device, angular rate information indicating an angular rate of the device, position information indicating a position or a change in position of the device, or orientation information indicating an orientation or a change in orientation of the device.

18. The system of claim 14, wherein the motion information of the at least one axis associated with the device comprises motion information of two or more axes associated with one or more sources of the motion information.

19. The system of claim 14, wherein the object classification component receives the motion information from at least one of an accelerometer, a gyroscope, an inertial measurement unit, or a sensor.

20. The system of claim 14, wherein the executable components further comprise an event determination component configured to control a display function associated with the touch sensitive surface based at least in part on the contact classification for the object, wherein the display function comprises enabling a touch sensitive function of the touch sensitive surface of a display screen of the device, disabling the touch sensitive function of the touch sensitive surface, transitioning the display screen from an off state to an on state, or transitioning the display screen from the on state to the off state.

21. The system of claim 14, wherein the object classification component is further configured to analyze the touch surface information, the motion information, and at least one other type of information, and determine the contact classification for the object with respect to the touch sensitive surface based at least in part on at least one result of the analyzing, and wherein the at least one other type of information comprises at least one of orientation information relating to an orientation of the device or touch information relating to touching of a portion of the device by a user.

22. The system of claim 14, wherein the motion information comprises previous motion information received prior to most recent motion information of the motion information, wherein the touch surface information comprises capacitive information that includes previous capacitive information received prior to most recent capacitive information of the capacitive information, and wherein the object classification component is further configured to determine the contact classification for the object with respect to the touch sensitive surface based at least in part on the previous capacitive information and the previous motion information.

23. The system of claim 14, wherein the object classification component is further configured to determine the contact classification for the object with respect to the touch sensitive surface, based at least in part on a result of analyzing previous classifications associated with the touch sensitive surface, to determine the contact state of the object with respect to the touch sensitive surface.

24. The system of claim 14, wherein the object classification component is further configured to determine one or more descriptive features that are indicative of the object for at least one subdivision of subdivisions of at least one of the touch surface information or the motion information, based at least in part on the at least one result of the analyzing, and wherein the contact classification for the object is determined based at least in part on the one or more descriptive features, at least one location of the at least one subdivision, and the motion information.

25. The system of claim 14, wherein the object classification component is further configured to be trained to distinguish between a first type of contact with the touch sensitive surface and at least one other type of contact with the touch sensitive surface based at least in part on example training classification samples comprising first positive example training classification samples relating to the first type of contact and at least one other example positive training classification samples relating to the at least one other type of contact, and wherein the first type of contact relates to an ear of a user in contact with the touch sensitive surface.

26. The system of claim 14, further comprising a buffer component configured to store at least one of a portion of the touch surface information or the portion of the motion information, in response to the device being determined to be in an in-call state, wherein the object classification component retrieves, from the buffer component, and analyzes at least one of the portion of the touch surface information or the portion of the motion information.

27. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, cause the processor to perform operations, comprising:

examining characteristics of frame data associated with a touch sensitive surface that is associated with a device and a subset of a set of motion data associated with at least one axis that is associated with the device, wherein a number of axes of the motion data to be included in the subset of the motion data is determined based at least in part on at least one of a time period available to perform the examining, an amount of resources available to perform the examining, or a confidence level that is for an initial contact classification for an object with respect to the touch sensitive surface and is determined based at least in part on an intermediate result of examining the frame data; and

based at least in part on at least one result of the examining, determining a contact classification for the object with respect to the touch sensitive surface to determine a contact state of the object with respect to the touch sensitive surface.

28. The non-transitory machine-readable medium of claim 27, wherein the operations further comprise:

receiving the motion data from at least one of an accelerometer, a gyroscope, an inertial measurement device, or a sensor, wherein the motion data is associated with more than one axis, and wherein the motion data is indicative of the contact classification to be associated with the object with respect to the touch sensitive surface.