Conduct and context relationships in mobile devices

• US 9,363,361 B2
• Filed: 04/12/2011
• Issued: 06/07/2016
• Est. Priority Date: 04/12/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kimchi, Gur, Davis, Marc Eliot, Dyor, Matthew Graham, Horvitz, Eric J.
• Primary Examiner(s): RAHAMAN, MOHAMMED S
• Application Number:
  US13/084,975
• Publication Number:
  US 20120262582A1
• Time in Patent Office: 1,883 Days
• Field of Search: 348/159, 348/E7.085
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

Described herein is a technique for customizing device behavior based on evaluated relationships between a user and the user'"'"'s environment. User conduct is analyzed based on available sensors, which may include user-facing cameras. The user'"'"'s environmental context is similarly evaluated based on the available sensors. Conduct and context, as well as any identified relationships between such conduct and context, may be reported to application programs, and may form the basis for customized device behavior.

Claims

1. A mobile device comprising:

a user sensor to sense conduct of a user;

an environment sensor to capture images of an environment;

user analysis logic that is responsive to the user sensor to identify the conduct of the user;

environment analysis logic that is responsive to the environment sensor to identify a context of the user based in part on the images of an environment captured by the environment sensor; and

device operational logic that varies device operation based at least in part on the identified conduct and context of the user, wherein the device operation includes;

determining a first object based at least in part on the identified conduct of the user;

determining a second object located in a path being traveled by the user based at least in part on the context of the user and the identified conduct of the user; and

presenting an alert to the user based at least in part on determining that the first object is different than the second object.

2. The mobile device of claim 1, wherein the device operational logic further varies the device operation based on a relationship of the user to the environment as indicated by the identified conduct and the context of the user.

3. The mobile device of claim 1, wherein the environment sensor indicates a position of the mobile device.

4. The mobile device of claim 1, wherein the environment sensor indicates an orientation of the mobile device.

5. The mobile device of claim 1, wherein the conduct of the user is identified relative to the mobile device.

6. The mobile device of claim 1, wherein the identified conduct of the user comprises a position of the user relative to the mobile device.

7. The mobile device of claim 1, wherein the identified conduct of the user comprises an orientation of the user relative to the mobile device.

8. The mobile device of claim 1, further comprising an application interface that is accessible by applications executing on the mobile device to report the identified conduct and context of the user to the applications.

9. The mobile device of claim 1, wherein the user sensor comprises a user-facing camera.

10. The mobile device of claim 1, wherein the identified conduct of the user comprises at least one of:

positions;

orientations;

actions;

activities;

or

states.

11. The mobile device of claim 1, wherein the determining the first object further includes determining the first object of a gaze of the user based at least in part on the conduct of the user.

12. A mobile device comprising:

a user-facing camera;

an environment-facing camera;

logic that is responsive to an analysis of an output of the user-facing camera and an analysis of an output of the environment-facing camera to identify relationships between a user and an environment associated with the user, wherein the analysis of the output of the environment-facing camera comprises;

determining a first object based at least in part on a conduct of the user captured by the user-facing camera;

determining a second object located in a path being traveled by the user based at least in part on a context of the user captured by the environment-facing camera; and

presenting an alert to the user based at least in part on determining that the first object is different than the second object; and

an application interface that is accessible by one or more applications executing on the mobile device to report the identified relationships to the applications.

13. The mobile device of claim 12, further comprising a logger that logs the identified relationships over time.

14. The mobile device of claim 12, wherein the logic analyzes images of the user to determine a conduct of the user.

15. The mobile device of claim 12, wherein the logic analyzes images of the user to determine a direction of the gaze of the user.

16. The mobile device of claim 12, wherein the logic analyzes images of the user to determine an orientation of the user relative to the mobile device.

17. The mobile device of claim 12, wherein at least one of the first object or second object includes at least one of a person, building, or landmark in the environment.