Providing recommendations based upon environmental sensing

• US 9,344,773 B2
• Filed: 02/05/2013
• Issued: 05/17/2016
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hayton, Mike, Johnson, Zach
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): NEWLIN, TIMOTHY R
• Application Number:
  US13/759,906
• Publication Number:
  US 20140223467A1
• Time in Patent Office: 1,197 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06Q 30/0252 based on events or environment, e.g. weather or festivals
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4668 for recommending content, e.g. movies
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts

Abstract

Embodiments are disclosed that relate to providing digital content recommendations based upon environmental sensor data. For example, one embodiment provides a computing device configured to receive sensor data from a sensor system, to detect a user present in the use environment via the sensor data, identify the user, recognize a current state of each of one or more use environment state features associated with the user via the sensor data, and store the current state of each of the one or more use environment state features. The computing device is further configured to detect a triggering condition for a selected use environment state feature, and in response provide information related to the selected use environment state feature based upon one or more of the current state of the use selected use environment state feature and a previously-stored state of the selected use environment state feature.

Claims

1. A computing device, comprising:

a logic subsystem; and

a data storage subsystem comprising instructions stored thereon that are executable by the logic subsystem to;

receive sensor data from a sensor system including one or more sensors configured to acquire the sensor data by monitoring a use environment;

associate the sensor data with a user;

detect via the sensor data a plurality of users present in the use environment;

recognize a current state of each of one or more use environment state features associated with each user via the sensor data;

store the current state of each of the one or more use environment state features;

detect a triggering condition for a selected use environment state feature; and

upon detecting the triggering condition, provide a recommendation of media content for consumption;

prevent presentation of a media content item if the plurality of users in the use environment exceeds a number allowed by an end-user license; and

allow presentation of the media content item of the media content item if the plurality of users does not exceed the number allowed by the end-user license.

2. The computing device of claim 1, where the current state of each of the one or more use environment features is stored via one or more of a local storage machine and a remote storage machine.

3. The computing device of claim 1, where the instructions are further executable to determine whether each user has provided authorization to be monitored by the sensor system, where the current state of each of the one or more use environment state features associated with the user are stored only if the user has provided authorization.

4. The computing device of claim 1, further comprising capturing the sensor data at a first, lower bitrate corresponding to a passive monitoring mode when no activity is detected, and at a second, higher bitrate corresponding to an active monitoring mode when activity is detected.

5. The computing device of claim 1, where the one or more use environment state features associated with each user includes one or more of a physical feature of the user and a device being operated by the user.

6. The computing device of claim 5, where the triggering condition includes a change in the physical feature of the user meeting a triggering threshold.

7. The computing device of claim 6, where the information related to the selected use environment feature includes one or more of information describing the change in the physical feature, a recommendation of goods or services related to the physical feature, and a recommendation of one or more changes to user behavior.

8. The computing device of claim 1, where the one or more use environment state features further includes one or more of an identity of a content item being consumed in the use environment, an identity of a constituent component of the content item being consumed in the use environment, a device accessing the content item, and a current access position of the content item.

9. The computing device of claim 8, where the triggering condition includes a user input requesting provision of the information related to the selected use environment state feature.

10. The computing device of claim 8, where the information related to the selected use environment state feature includes one or more of information regarding a related content item, auxiliary information for the content item, and a link to acquire an instance of the content item.

11. The computing device of claim 1, where the one or more sensors include one or more of an audio sensor and an image sensor.

12. On a computing device, a method for monitoring a use environment, the method comprising:

acquiring sensor data via one or more sensors monitoring the use environment;

detecting via the sensor data a plurality of users present in the use environment;

providing a recommendation to view a media content item based upon the sensor data;

allowing presentation of the media content item if the plurality of users does not exceed the number allowed by an end-user license; and

preventing presentation of a media content item if the plurality of users in the use environment exceeds a number allowed by the end-user license.

13. The method of claim 12, wherein detecting via the sensor data the plurality of users comprises detecting the plurality of users via depth image data.

14. The method of claim 12, further comprising detecting an activity performed by one or more of the plurality of users in the use environment, and providing the recommendation based on the activity.

15. The method of claim 12, further comprising detecting one or more of the plurality of users performing a different activity than another of the plurality of users, and wherein providing information comprises recommending an activity that can be performed by the plurality of users together.

16. The method of claim 12, further comprising

prior to detecting the plurality of users, acquiring first sensor data at a first bitrate via the one or more sensors monitoring the use environment, the first bitrate corresponding to a passive monitoring mode, where detecting activity in the use environment is based on the first sensor data;

and

in response to detecting one or more of the plurality of users in the use environment, acquiring second sensor data at a second bitrate via the one or more sensors, the second bitrate being higher than the first bitrate and corresponding to an active monitoring mode.

17. The method of claim 12, further comprising determining whether each user has provided authorization to be monitored by the sensor system by recognizing one or more of previously-defined authorization information and an authorization gesture, where the current state of each of one or more use environment features is not stored when authorization to be monitored by the sensor system has not been provided.

18. A computing device, comprising:

a logic subsystem; and

a data storage subsystem comprising instructions stored thereon that are executable by the logic subsystem to;

acquire sensor data via one or more sensors monitoring the use environment;

detect via the sensor data a plurality of users present in the use environment;

provide a recommendation to view a media content item based upon the sensor data;

allow presentation of the media content item if the plurality of users does not exceed the number allowed by an end-user license; and

prevent presentation of a media content item if the plurality of users in the use environment exceeds a number allowed by the end-user license.

19. The computing device of claim 18, wherein the instructions are further executable to detect the plurality of users via depth image data.

20. The computing device of claim 18, wherein the instructions are further executable to detect an activity performed by one or more of the plurality of users in the use environment, and provide the recommendation based on the activity.