Detection of user activities by a portable device

• US 9,870,069 B2
• Filed: 12/30/2011
• Issued: 01/16/2018
• Est. Priority Date: 10/13/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: March, Wendy, Krishnamurthy, Lakshman, Shah, Rahul C., Wilder, Carol A., Schulze, Jack, Jones, David M., Ludlam, Nick, Webb, Matthew W., Malia, Joseph A., Davies, Aaron D., Raffa, Giuseppe, Nachman, Lama
• Primary Examiner(s): GARNER, WERNER G
• Application Number:
  US13/977,599
• Publication Number:
  US 20160328029A1
• Time in Patent Office: 2,209 Days
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G10L 25/81 for discriminating voice from music
• CPC Class Codes: G10L 25/84 for discriminating voice from noise

Abstract

A portable consumer device, such as a mobile phone or a tablet computer, may be configured to collect measurements data associated with at least one of movement of the portable consumer device or orientation of the portable consumer device, as well as data associated with an external environment of the portable consumer device. The collected data may be evaluated in order to determine motion of the portable consumer device over time and contextual information associated with the portable consumer device. A user activity may be determined based upon the determined motion and the determined contextual information. As desired, the user activity may be evaluated in association with a suitable application scenario, such as a gaming application scenario.

Claims

1. A portable consumer device comprising:

at least one inertial sensor that collects measurements data associated with at least one of movement of the portable consumer device or orientation of the portable consumer device;

at least one context sensor that collects environment data associated with an external environment of the portable consumer device;

at least one memory storing;

motion processing instructions, that when executed by at least one sensor processor, causes the at least one sensor processor to;

process the collected measurements data to determine motion of the portable consumer device over time;

context processing instructions, that when executed by the at least one sensor processor, causes the at least one sensor processor to;

process the collected environment data to determine contextual information associated with the portable consumer device, and

compare an amplitude of the collected environment data to an amplitude threshold; and

activity processing instructions, that when executed by the at least one sensor processor, causes the at least one sensor processor to;

determine, based at least in part on the determined motion, the determined contextual information, and the comparison of the amplitude of the collected environment data to the amplitude threshold, a user activity,

compare the determined user activity to a desired user activity, and

display user feedback on a display based at least in part on the comparison.

2. The portable consumer device of claim 1, wherein the at least one inertial sensor comprises at least one of (i) an accelerometer or (ii) a gyroscope.

3. The portable consumer device of claim 1, wherein the determined motion comprises a gesture made with the portable consumer device by a user.

4. The portable consumer device of claim 1, wherein detection of a gesture is initiated based at least in part on the determined contextual information.

5. The portable consumer device of claim 1, wherein the determined motion comprises one of (i) remaining still, (ii) walking, (iii) strolling, (iv) running, or (v) cycling.

6. The portable consumer device of claim 1, wherein the at least one context sensor comprises an audio detection device, and

wherein the at least one context processing instruction further comprise instructions that when executed by the at least one sensor processor, cause the at least one sensor processor to:

compare an amplitude of collected audio data to an audio data amplitude threshold.

7. The portable consumer device of claim 1, wherein the at least one context sensor comprises an audio detection device, and

wherein the at least one context processing instructions further comprise instructions, that when executed by the at least one sensor processor, cause the at least one sensor processor to:

process a collected audio data to identify one of (i) human speech, (ii) music, or (iii) mechanical sound.

8. The portable consumer device of claim 1, wherein the at least one context sensor comprises a light detection device, and

wherein the at least one context processing instructions further comprise instructions, that when executed by the at least one sensor processor, cause the at least one sensor processor to:

compare an amplitude of detected ambient light to an ambient light amplitude threshold.

9. The portable consumer device of claim 8, wherein the at least one activity processing instructions further comprise instructions, that when executed by the at least one sensor processor, cause the at least one sensor processor to:

provide user feedback in association with a gaming application.

10. At least one non-transitory computer-readable medium comprising computer-executable instructions that, when executed by at least one processor associated with a portable consumer device, configure the at least one processor to:

receive, from at least one inertial sensor, measurements data associated with one of (i) movement of the portable consumer device or (ii) orientation of the portable consumer device;

receive, from at least one context sensor, data associated with an external environment of the portable consumer device;

compare an amplitude of the data to an amplitude threshold;

determine, based at least in part on the received measurements data, the received environment data, and the comparison of the amplitude of the data to the amplitude threshold, a user activity;

compare the determined user activity to a desired user activity; and

display user feedback on a display based at least in part on the comparison.

11. The non-transitory computer-readable medium of claim 10, wherein at least a portion of the computer-executable instructions are associated with a gaming application.

12. The non-transitory computer-readable medium of claim 11, wherein at least the portion of the computer-executable instructions are sensor data evaluation instructions, that further configure the at least one processor to:

initiate the gaming application based at least in part on the determined user activity.

13. The non-transitory computer-readable medium of claim 10, wherein the determined user activity comprises a gesture made with the portable consumer device by a user.

14. The non-transitory computer-readable medium of claim 13, wherein detection of the gesture is initiated based at least in part on a contextual information.

15. The non-transitory computer-readable medium of claim 10, wherein the determined user activity comprises one of (i) remaining still, (ii) walking, (iii) strolling, (iv) running, or (v) cycling.

16. The non-transitory computer-readable medium of claim 10, wherein the environment data comprises audio data collected by an audio detection device, and

wherein the computer-executable instructions further configure the at least one processor to:

compare an amplitude of the audio data to an audio amplitude threshold and determine the user activity based at least in part on the comparison of the amplitude of the audio data to the audio amplitude threshold.

17. The non-transitory computer-readable medium of claim 10, wherein the environment data comprises an amplitude of detected ambient light, and

wherein the computer-executable instructions further configure the at least one processor to:

compare the detected amplitude of the ambient light to an ambient light amplitude threshold and determine the user activity based at least in part on the comparison of the detected amplitude of the ambient light to the ambient light amplitude threshold.

18. A computer-implemented method comprising:

executing, by a portable consumer device comprising one or more computer processors, at least one application configured for;

outputting a direction associated with a desired user activity;

receiving, from at least one inertial sensor, measurements data associated with one of (i) movement of the portable consumer device or (ii) orientation of the portable consumer device;

receiving, from at least one context sensor, data associated with an external environment of the portable consumer device;

comparing an amplitude of the data to an amplitude threshold;

determining, based at least in part upon the received measurements data, the comparison of the amplitude of the data to the amplitude threshold, and received environment data, a user activity;

comparing the determined user activity to the desired user activity; and

displaying user feedback on a display based at least in part on the comparison.

19. The method of claim 18, wherein the at least one application comprises a gaming application.

20. The method of claim 19, further comprising:

evaluating, by a sensor data evaluation component, at least one of the measurements data or the environment data; and

initiating, by the sensor data evaluation component, the gaming application based at least in part on the evaluation.

21. The method of claim 18, wherein determining the user activity comprises determining a gesture made with the portable consumer device by a user.

22. The method of claim 21, further comprising initiating the determination of the gesture based at least in part on the contextual information.

23. The method of claim 18, wherein determining the user activity comprises determining one of (i) remaining still, (ii) walking, (iii) strolling, (iv) running, or (v) cycling.

24. The method of claim 18, wherein receiving the environment data comprises receiving audio data collected by an audio detection device, and further comprising:

identify, based at least in part on the audio data, one of (i) human speech, (ii) music, or (iii) mechanical sound utilized to determine the user activity.

25. The method of claim 18, wherein receiving the environment data comprises receiving an amplitude of detected ambient light, and further comprising:

comparing the detected amplitude of the ambient light to an ambient light amplitude threshold; and

determining the user activity based at least in part on the comparison of the detected amplitude of the ambient light to the ambient light amplitude threshold.