Environmental control via wearable computing system

• US 10,345,768 B2
• Filed: 09/29/2014
• Issued: 07/09/2019
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fullam, Scott, Corlett, Barry John
• Primary Examiner(s): Mitchell, Jason D
• Application Number:
  US14/500,864
• Publication Number:
  US 20160091877A1
• Time in Patent Office: 1,744 Days
• Field of Search: 700276
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/28 responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light controlling the output signals based on the game progress A63F13/50
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: E06B 9/68 Operating devices or mechanisms, e.g. with electric drive
• CPC Class Codes: F24F 11/30 for purposes related to the operation of the system, e.g. for safety or monitoring
• CPC Class Codes: F24F 11/52 Indication arrangements, e.g. displays
• CPC Class Codes: F24F 11/56 Remote control
• CPC Class Codes: F24F 11/57 using telephone networks
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 11/62 characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
• CPC Class Codes: F24F 11/63 Electronic processing
• CPC Class Codes: F24F 2120/00 Control inputs relating to users or occupants
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 3/00 Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
• CPC Class Codes: H04L 12/28 characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks] wireless communication networks H04W ; arrangements for dividing the transmission path H04W40/00
• CPC Class Codes: H05B 47/175 by remote control

Abstract

Various embodiments relating to controlling ambient environmental conditions to affect one or more human subjects wearing wearable computing systems are disclosed. In one example, a head-mounted computing system includes a see-through display configured to present an augmented reality image in front of an environment when viewed through the see-through display, a logic subsystem, and a storage subsystem. The storage subsystem may hold instructions executable by the logic subsystem to present the augmented reality image via the see-through display, and during presentation of the augmented reality image, send a command to control an environmental control system external to the head-mounted computing system. The command may be configured to trigger adjustment of an environmental parameter of the environment to enhance presentation of the augmented reality image.

Claims

1. A head-mounted computing system comprising:

a see-through display configured to present an augmented reality image in front of a real-world environment when viewed through the see-through display;

an optical sensor system configured to measure one or more environmental lighting parameters of the real-world environment;

a logic subsystem; and

a storage subsystem holding instructions executable by the logic subsystem to;

present the augmented reality image via the see-through display;

determine an image parameter of the image based on content in the augmented reality image;

during presentation of the augmented reality image, receive one or more measured environmental lighting parameters from the optical sensor system, wherein the one or more measured environmental lighting parameters comprise one or more of an ambient light intensity and an ambient light color;

determine that a display parameter of the see-through display has reached an end of an adjustment range; and

in response to determining that the display parameter has reached the end of the adjustment range, during presentation of the augmented reality image, send a command to control an environmental control system external to the head-mounted computing system, wherein the command is configured to change one or more physical lighting parameters of the real-world environment to adjust the ambient light based on the image parameter of the image and the one or more measured environmental lighting parameters to visually enhance presentation of the augmented reality image.

2. The head-mounted computing system of claim 1, wherein the environmental control system comprises a remote-controlled lighting system.

3. The head-mounted computing system of claim 1, wherein the command is a first command, wherein the environmental control system further comprises a heating, ventilation, and air conditioning (HVAC) system, and wherein the instructions are further executable to send a second command configured to trigger an adjustment of an ambient temperature provided by the HVAC system.

4. The head-mounted computing system of claim 1, wherein the command is a first command, and wherein the storage subsystem holds instructions executable by the logic subsystem to:

during presentation of the augmented reality image, detect a content-based trigger; and

in response to the content-based trigger, send a second command to control the environmental control system, wherein the second command is configured to trigger adjustment of an environmental parameter to visually enhance presentation of content corresponding to the content-based trigger.

5. The head-mounted computing system of claim 1, wherein the command is a first command, and further comprising:

a sensor subsystem including one or more biometric sensors configured to provide biometric data of a human subject wearing the head-mounted computing system, wherein the storage subsystem holds instructions executable by the logic subsystem to determine a comfort characteristic of the human subject based on the biometric data, and during presentation of the augmented reality image, send a second command configured to trigger adjustment of an environmental parameter to change the comfort characteristic of the human subject.

6. The head-mounted computing system of claim 5, wherein the comfort characteristic comprises a stress level, and wherein the second command is configured to trigger adjustment of the environmental parameter to increase the stress level of the human subject.

7. The head-mounted computing system of claim 5, wherein the comfort characteristic comprises an estimated eye strain level, and wherein the second command is configured to trigger adjustment of the environmental parameter to decrease the estimated eye strain level of the human subject.

8. A wearable computing system, comprising:

a sensor subsystem including one or more biometric sensors configured to provide biometric data of a human subject wearing the wearable computing system in an environment;

a logic subsystem; and

a storage subsystem holding instructions executable by the logic subsystem to

receive eye imaging data from the one or more biometric sensors of the sensor subsystem;

using the eye imaging data, determine a comfort characteristic of the human subject; and

send a command to control an environmental control system external to the wearable computing system based on the comfort characteristic, wherein the command is configured to trigger adjustment of an environmental parameter to change the comfort characteristic of the human subject.

9. The wearable computing system of claim 8, wherein the environmental control system comprises one or more of a remote-controlled window shade and a remote-controlled lighting system, wherein the environmental parameter comprises an ambient light intensity of the environment, and wherein the command is configured to trigger adjustment of the ambient light intensity by adjusting the one or more of the remote-controlled window shade and the remote-controlled lighting system.

10. The wearable computing system of claim 8, wherein the storage subsystem holds instructions executable by the logic subsystem to

receive biometric data from the one or more biometric sensors of the sensor subsystem;

determine the comfort characteristic of the human subject based on the biometric data; and

send a second command to control the environmental control system, wherein the second command is configured to trigger adjustment of the environmental parameter to change the comfort characteristic of the human subject.

11. The wearable computing system of claim 10, wherein the one or more biometric sensors comprises one or more of a temperature sensor to provide a temperature of the human subject, a galvanic skin response (GSR) sensor to provide a skin conductance of the human subject, an optical sensor to provide an optical characteristic of an eye of the human subject, a heart rate sensor to provide a heart rate of the human subject, and a carbon dioxide (CO₂) sensor to provide a CO₂ level of breath exhaled by the human subject.

12. The wearable computing system of claim 10, wherein the comfort characteristic comprises an estimated eye strain level of the human subject.

13. The wearable computing system of claim 10, wherein the environmental control system comprises an HVAC system, wherein the environmental parameter comprises an ambient temperature of the environment, and wherein the instructions are further executable to send a command configured to trigger adjustment of the ambient temperature provided by the HVAC system to change the comfort characteristic of the human subject.

14. The wearable computing system of claim 13, wherein the comfort characteristic comprises a thermal level of the human subject.

15. On a computing system, a method for controlling ambient environmental conditions in an environment, the method comprising:

receiving biometric data from a plurality of wearable computing systems, each wearable computing system including one or more biometric sensors configured to provide the biometric data of a different human subject wearing the wearable computing system in the environment, and for each wearable computing system the biometric data representing a comfort characteristic of the human subject wearing the wearable computing system;

determining a collective comfort characteristic of a plurality of human subjects in the environment based on the biometric data received from the plurality of different wearable computing systems, wherein the collective comfort characteristic is based on an individual comfort characteristic of each human subject of the plurality of human subjects, and wherein the individual comfort characteristic of each human subject is based on biometric data of the human subject; and

controlling an environmental control system external to the plurality of wearable computing systems to adjust an environmental parameter of the environment to change the collective comfort characteristic of the plurality of human subjects.

16. The method of claim 15, wherein the environmental control system comprises one or more of a remote-controlled window shade and a remote-controlled lighting system, wherein the environmental parameter comprises an ambient light intensity of the environment, and wherein the one or more of the remote-controlled window shade and the remote-controlled lighting system are controlled to adjust the ambient light intensity of the environment to change the collective comfort characteristic of the plurality of human subjects.

17. The method of claim 16, wherein the collective comfort characteristic comprises an estimated eye strain level of the plurality of human subjects.

18. The method of claim 15, wherein the environmental control system comprises an HVAC system, wherein the environmental parameter comprises an ambient temperature of the environment, and wherein the HVAC system is controlled to adjust the ambient temperature of the environment to change the collective comfort characteristic of the plurality of human subjects.

19. The method of claim 18, wherein the collective comfort characteristic comprises an average thermal level of the plurality of human subjects.