Detection of and response to extra-device touch events

• US 9,448,637 B2
• Filed: 05/01/2013
• Issued: 09/20/2016
• Est. Priority Date: 05/01/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Avrahami, Daniel
• Primary Examiner(s): Sternbane, Larry
• Application Number:
  US13/874,905
• Publication Number:
  US 20140327655A1
• Time in Patent Office: 1,238 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Technologies for detection of and response to extra-device touch events may include mobile device sensors, such as accelerometers, that detect that a touch event has taken place by detecting device movement in response to the touch event. In some embodiments, a touch event feature vector is generated from sensor readings and passed to a classifier that determines the touch event location. Other touch event information, such as timing, intensity and a profile of the object that caused the touch event can also be determined. The detecting and locating of touch events allows the interaction area of a mobile device to be extended beyond the physical bounds of the device, and allows multiple users to simultaneously use a mobile device in a more convenient manner. Applications executing on a mobile device or the device'"'"'s operating system can take various actions based on the touch event'"'"'s location.

Claims

1. A mobile device comprising:

one or more sensors to detect movement of the mobile device caused by an extra-device touch event on a surface on which the mobile device is placed and within a proximity of the mobile device, wherein the one or more sensors comprise one or more accelerometers or gyroscopes;

one or more classifiers to determine a location of the extra-device touch event based at least in part on readings of the one or more sensors; and

a reporting module to report the location of the extra-device touch event to an application executing on the mobile device.

2. The mobile device is claim 1, wherein the location of the extra-device touch event comprises a direction relative to the mobile device.

3. The mobile device of claim 1, wherein the application is capable of performing an action based at least in part on an extra-device touch event location provided to the application by the reporting module.

4. The mobile device of claim 3, wherein the application is further capable of performing a first action if the extra-device touch event location is within a first region of the proximity of the mobile device and a second action if the extra-device touch event location is within a second region of the proximity of the mobile device, the first action being different than the second action.

5. The mobile device of claim 3, further comprising an extra-device touch event determination module to determine timing information of the extra-device touch event, the reporting module to further report the timing information of the extra-device touch event to the application, the action to be performed by the application being further based at least in part on the timing information of the extra-device touch event.

6. The mobile device of claim 3, at least one of the classifiers to further determine an intensity of the extra-device touch event, the reporting module to further report the intensity of the extra-device touch event to the application, the application being capable of performing an action based at least in part on the extra-device touch event location and the intensity of the extra-device touch event provided to the application by the reporting module.

7. The mobile device of claim 1, further comprising:

a vector generation module to generate an extra-device touch event feature vector based on the readings of the one or more sensors for use by the one or more classifiers to determine the location of the extra-device touch event; and

an extra-device touch event determination module to determine whether the readings of the one or more sensors indicate the occurrence of an extra-device touch event.

8. The mobile device of claim 1, further comprising:

a vector generation module to generate an extra-device touch event feature vector based on the readings of the one or more sensors for use by the one or more classifiers to determine the location of the extra-device touch event and an intensity of the extra-device touch event; and

an extra-device touch event determination module to determine whether the readings of the one or more sensors indicate the occurrence of an extra-device touch event.

9. A method of detecting and responding to an extra-device touch event, the method comprising:

determining a location of an extra-device touch event on a surface on which a mobile device is placed and occurring in the proximity of the mobile device based on movement of the mobile device caused by the extra-device touch event and as detected by one or more mobile device sensors, wherein the one or more mobile device sensors comprise one or more accelerometers or gyroscopes;

determining an action to perform based at least in part on the location; and

performing the action at the mobile device.

10. The method of claim 9, further comprising determining timing information for the extra-device touch event, wherein the determining an action to perform is further based on the timing information.

11. The method of claim 9, further comprising determining timing information for the extra-device touch event and timing information for one or more additional extra-device touch events occurring in the proximity of the mobile device, wherein the determining an action to perform is further based on which of the extra-device touch event and the one or more additional extra-device touch events occurred earliest, according at least in part to the timing information for the extra-device touch event and the timing information for the one or more additional extra-device touch events.

12. The method of claim 9, wherein the determining the location comprises:

reading one or more sensor values from the one or more mobile device sensors;

determining that the one or more sensor values indicate the occurrence of an extra-device touch event; and

generating an extra-device touch event feature vector from the one or more sensor values;

wherein the determining the location comprises providing the extra-device touch event feature vector to a classifier.

13. One or more non-transitory computer-readable media having instructions stored thereon for causing a mobile device to perform a method, the method comprising:

determining a location of an extra-device touch event on a surface on which the mobile device is placed and occurring in the proximity of the mobile device based on movement of the mobile device caused by the extra-device touch event and as detected by one or more mobile device sensors, wherein the one or more mobile device sensors comprise one or more accelerometers or gyroscopes;

determining an action to perform based at least in part on the location; and

performing the action at the mobile device.

14. The one or more non-transitory computer-readable media of claim 13, wherein the location comprises a direction relative to the mobile device.

15. The one or more non-transitory computer-readable media of claim 13, further comprising determining timing information for the extra-device touch event, wherein the determining an action to perform is further based on the timing information.

16. The one or more non-transitory computer-readable media of claim 13, further comprising determining timing information for the extra-device touch event and timing information for one or more additional extra-device touch events occurring in the proximity of the mobile device, wherein the determining an action to perform is further based on which of the extra-device touch event and the one or more additional extra-device touch events occurred earliest, according at least in part to the timing information for the extra-device touch event and the timing information for the one or more additional extra-device touch events.

17. The one or more non-transitory computer-readable media of claim 13, wherein the action to perform is one of a plurality of actions, the plurality of actions comprising a first action to be performed if the location is within a first region of the proximity of the mobile device and a second action to be performed if the extra-device touch event is determined to be within a second region of the proximity to the mobile device, the first event being different than the second event.

18. The one or more non-transitory computer-readable media of claim 13, further comprising determining an intensity of the extra-device touch event, wherein the action to be performed is further based on the intensity of the extra-device touch event.

19. The one or more non-transitory computer-readable media of claim 13, further comprising determining a profile of an object used to create the extra-device touch event, wherein the action to be performed is further based on the profile of the object.

20. The one or more non-transitory computer-readable media of claim 13, wherein the determining the location comprises:

reading one or more sensor values from the one or more mobile device sensors;

determining that the one or more sensor values indicate the occurrence of an extra-device touch event; and

generating an extra-device touch event feature vector from the one or more sensor values;

wherein the determining the location comprises providing the extra-device touch event feature vector to a classifier.

21. The one or more non-transitory computer-readable media of claim 20, wherein the determining that the one or more sensor values indicate the occurrence of an extra-device touch event comprises determining that at least one of the one or more mobile sensor values exceeds an extra-device touch event threshold.

22. The one or more non-transitory computer-readable media of claim 20, wherein the classifier is one of a plurality of classifiers, the method further comprising selecting the classifier from the plurality of classifiers based on a surface type of the proximity of the mobile device.

23. The one or more non-transitory computer-readable media of claim 22, wherein the surface type is determined based at least in part on an image of a surface of the proximity of the mobile device.

24. An apparatus comprising:

a means for determining a location of an extra-device touch event on a surface on which a mobile device is placed and occurring in the proximity of the mobile device based on movement of the mobile device caused by the extra-device touch event and as detected by one or more mobile device sensors, wherein the one or more mobile device sensors comprise one or more accelerometers or gyroscopes;

a means for determining an action to perform based at least in part on the location; and

a means for performing the action at the mobile device.

25. The apparatus of claim 24, further comprising a means for determining timing information for the extra-device touch event and timing information for one or more additional extra-device touch events occurring in the proximity of the mobile device, wherein the means for determining an action to perform is further based on which of the extra-device touch event and the one or more additional extra-device touch events occurred earliest, according at least in part to the timing information for the extra-device touch event and the timing information for the one or more additional extra-device touch events.