System and method for eyes-free interaction with a computing device through environmental awareness

• US 8,130,193 B2
• Filed: 03/31/2005
• Issued: 03/06/2012
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Flynt, David Wayne, Brooke, Timothy Lawrence, Chin, Peter
• Primary Examiner(s): Nguyen, Kevin M
• Application Number:
  US11/097,870
• Publication Number:
  US 20060221051A1
• Time in Patent Office: 2,532 Days
• Field of Search: 345/156, 345/175, 345/690
• US Class Current: 345/156
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

A system and method for eyes-free interaction with a computing device through environmental awareness. A smart computing device includes non-environmental impact settings that indicate what type of user impact will be recognized as an input. The smart computing device monitors environmental conditions to distinguish a user impact from an environmental impact. Upon detecting a user impact, a command is generated to perform a function on an application.

Claims

1. A computer-implemented method for interaction with a mobile device through environmental awareness, the method comprising:

obtaining preferences on a mobile device, wherein the preferences indicate;

a plurality of environmental context settings, wherein each of the plurality of environmental context settings identify a respective application of a plurality of applications of the mobile device to implicate when a respective environmental context external to the mobile device is detected by the mobile device, and

a command of the respective application to execute in response to detecting the respective environmental context and in response to receiving a non-environmental impact;

monitoring, by an environmental awareness sensor of the mobile device, environmental conditions, wherein the environmental conditions monitored comprise a location condition, a movement condition, an audio condition and a light condition;

determining when the monitoring by the environmental awareness sensor indicates a rhythmic pattern that indicates an activity and an occurrence of an environmental condition and when the monitoring by the environmental awareness indicates an occurrence of a non-environmental impact;

in response to determining the occurrence of the non-environmental impact;

interpreting the at least one environmental condition external to the mobile device detected by the environmental awareness sensor of the mobile device to determine a current environmental context external to the mobile device,

matching the determined current environmental context external to the mobile device to an environmental context setting of the plurality of environmental context settings indicated in the preferences,

based on the matching of the determined current environmental context external to the mobile device to the environmental context setting, implicating the respective application of the plurality of applications indicated in the preferences that is matched with the environmental context setting in the preferences, and

executing the command of the respective application.

2. The computer-implemented method of claim 1, wherein the plurality of environmental context settings include at least one member of a group comprising:

a detectable location, a detectable movement, a detectable conductance, a detectable audio, a detectable light level, a detectable temperature, a detectable heat level, a detectable smoke level, and a detectable altitude.

3. The computer-implemented method of claim 1, wherein the environmental awareness sensor includes at least one member of a group comprising:

a location sensor, a movement sensor, a conductance sensor, an audio sensor, a light sensor, a temperature sensor, a heat sensor, a smoke sensor and an altitude sensor.

4. The computer-implemented method of claim 1, wherein the environmental condition external to the mobile device further comprises:

a conductance condition, a temperature condition, a heat condition, a smoke condition, and an altitude condition.

5. The computer-implemented method of claim 1, wherein the non-environmental impact comprises any one of:

a double tap impact, a triple tap impact, a movement, an acceleration, and a touch.

6. The computer-implemented method of claim 1,

wherein the preferences further indicate a plurality of respective application commands, wherein each of the plurality of respective application commands is implicated by a different non-environmental impact type,

wherein receiving a non-environmental impact further includes detecting a non-environmental impact type, and

wherein executing the command of the respective application further includes executing a command of the plurality of respective commands that is implicated by the non-environmental impact type.

7. The computer-implemented method of claim 1,

wherein the preferences further indicate condition settings, wherein the condition settings are internal scheduling events associated with the mobile device,

wherein matching the determined current environmental context external to the mobile device to the environmental context setting indicated in the preferences further includes matching current conditions associated with the internal scheduling events of the mobile device to condition settings associated with the preferences, and

wherein identifying the respective application of the plurality of applications indicated in the preferences further includes identifying the respective application of the plurality of applications indicated in the preferences that matches with the environmental context setting and indicated condition settings.

8. A computer-readable storage medium having computer executable instructions for interaction through environmental awareness, the instructions comprising:

obtaining user preferences on a mobile device, wherein the user preferences indicate;

a plurality of environmental context settings, wherein each of the plurality of environmental context settings identify a respective application of a plurality of applications of the mobile device to implicate when a respective environmental context external to the mobile device is detected by the mobile device, and

a command of the respective application to execute in response to detecting the respective environmental context and in response to receiving a user tap input;

detecting a user tap input, wherein the user tap input is detected by;

monitoring, by an accelerometer, environmental impact patterns to the mobile device, and

distinguishing a user tap input from the environmental impact patterns to the mobile device by determining when the monitoring by the accelerometer indicates a rhythmic pattern that indicates an activity and an occurrence of an environmental condition and when the monitoring by the accelerometer indicates an occurrence of the user tap;

in response to detecting the user tap input;

detecting, by an environmental awareness sensor of the mobile device, at least one environmental condition external to the mobile device,

interpreting the at least one environmental condition external to the mobile device detected by the environmental awareness sensor of the mobile device to determine a current environmental context external to the mobile device,

matching the determined current environmental context external to the mobile device to an environmental context setting of the plurality of environmental context settings indicated in the preferences,

based on the matching of the determined current environmental context external to the mobile device to the environmental context setting, implicating the respective application of the plurality of applications indicated in the preferences that is matched with the environmental context setting in the preferences, and

executing the command of the respective application.

9. The computer-readable storage medium of claim 8, wherein the plurality of environmental context settings include at least one member of a group comprising:

a detectable location, a detectable movement, a detectable conductance, a detectable audio, a detectable light level, a detectable temperature, a detectable heat level, a detectable smoke level, and a detectable altitude.

10. The computer-readable storage medium of claim 8, wherein the environmental awareness sensor includes at least one member of a group comprising:

a location sensor, a movement sensor, a conductance sensor, an audio sensor, a light sensor, a temperature sensor, a heat sensor, a smoke sensor and an altitude sensor.

11. The computer-readable storage medium of claim 8, wherein the environmental condition external to the mobile device includes at least one member of a group comprising:

a location condition, a movement condition, a conductance condition, an audio condition, a light condition, a temperature condition, a heat condition, a smoke condition, and an altitude condition.

12. The computer-readable storage medium of claim 8, wherein the user tap input includes at least one member of a group comprising:

a single tap impact, a double tap impact, and a triple tap impact.

13. The computer-readable storage medium of claim 8,

wherein the user preferences further indicate a plurality of respective application commands, wherein each of the plurality of respective application commands is implicated by a different user tap input,

wherein receiving a user tap input further includes detecting a user tap type, and

wherein executing the command of the respective application further includes executing a command of the plurality of respective commands that is implicated by the user tap type.

14. The computer-readable storage medium of claim 8,

wherein the preferences further indicate condition settings, wherein the condition settings are internal scheduling events associated with the mobile device,

wherein matching the determined current environmental context external to the mobile device to the environmental context setting indicated in the preferences further includes matching current conditions associated with the internal scheduling events of the mobile device to condition settings associated with the preferences, and

wherein identifying the respective application of the plurality of applications indicated in the preferences further includes identifying the respective application of the plurality of applications indicated in the preferences that matches with the environmental context setting and indicated condition settings.

15. A system for interaction through environmental awareness, the system comprising:

a processor; and

a memory having computer executable-instructions stored thereon, wherein the computer-executable instructions are configured for;

obtaining preferences on a mobile device, wherein the preferences include;

a plurality of environmental context settings, wherein each of the plurality of environmental context settings identify a respective application of a plurality of applications of the mobile device to implicate when a respective environmental context external to the mobile device is detected by the mobile device, and

a plurality of respective application commands, wherein each of the plurality of respective application commands is executed in response to detecting the respective environmental context external to the mobile device and in response to receiving a non-environmental impact type;

detecting a non-environmental impact type, wherein the non-environmental impact type is detected by;

monitoring, by an accelerometer, environmental impact patterns to the mobile device,

distinguishing a non-environmental impact from the environmental impact patterns to the mobile device by determining when the monitoring by the accelerometer indicates a rhythmic pattern that indicates an activity and an occurrence of an environmental impact and when the monitoring by the accelerometer indicates an occurrence of the non-environmental impact, and

determining the non-environmental impact type of the non-environmental impact;

in response to determining the non-environmental impact type;

detecting, by an environmental awareness sensor of the mobile device, at least one environmental condition external to the mobile device,

interpreting the at least one environmental condition external to the mobile device detected by the environmental awareness sensor of the mobile device to determine a current environmental context external to the mobile device,

matching the determined current environmental context external to the mobile device to an environmental context setting of the plurality of environmental context settings indicated in the preferences,

based on the matching of the determined current environmental context external to the mobile device to the environmental context setting, implicating the respective application of the plurality of applications indicated in the preferences that is matched with the environmental context setting in the preferences, and

executing a command of the plurality of commands of the respective application that is implicated by the non-environmental impact type.

16. The system of claim 15, wherein the plurality of environmental context settings include at least one member of a group comprising:

a detectable location, a detectable movement, a detectable conductance, a detectable audio, a detectable light level, a detectable temperature, a detectable heat level, a detectable smoke level, and a detectable altitude.

17. The system of claim 15, wherein the environmental awareness sensor includes at least one member of a group comprising:

a location sensor, a movement sensor, a conductance sensor, an audio sensor, a light sensor, a temperature sensor, a heat sensor, a smoke sensor and an altitude sensor.

18. The system of claim 15, wherein the environmental condition external to the mobile device includes at least one member of a group comprising:

a location condition, a movement condition, a conductance condition, an audio condition, a light condition, a temperature condition, a heat condition, a smoke condition, and an altitude condition.

19. The system of claim 15, wherein the non-environmental impact includes at least one member of a group comprising:

a single tap impact, a double tap impact, a triple tap impact, a movement, an acceleration, and a touch.

20. The system of claim 15,

wherein the preferences further indicate condition settings, wherein the condition settings are internal scheduling events associated with the mobile device,

wherein matching the determined current environmental context external to the mobile device to the environmental context setting indicated in the preferences further includes matching current conditions associated with the internal scheduling events of the mobile device to condition settings associated with the preferences, and

wherein identifying the respective application of the plurality of applications indicated in the preferences further includes identifying the respective application of the plurality of applications indicated in the preferences that matches with the environmental context setting and indicated condition settings.