BUILDING LIGHTING AND TEMPERATURE CONTROL WITH AN AUGMENTED REALITY SYSTEM

• US 20170108838A1
• Filed: 10/14/2015
• Published: 04/20/2017
• Est. Priority Date: 10/14/2015
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Patel, Mehul, Norton, Kent, Todeschini, Erik
• Granted Patent: US 10,146,194 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H05B 47/125 by using cameras
• CPC Class Codes: Y02B 20/40 Control techniques providing energy savings, e.g. smart controller or presence detection

Abstract

An augmented reality (AR) system, device, and method for reducing energy usage by monitoring and determining optimal control settings for conditions within a building. Sensors embedded in AR devices worn by workers performing job duties are utilized to detect and measure physical conditions such as visible light intensity and air temperature. Location, time/date, and worker occupancy data is also provided to the AR system. A time- and location-based data profile of environmental conditions within the building is created and regularly updated to track changes in environmental conditions over time. Algorithms running on a processor calculate and provide optimal control settings to building lighting and climate control systems. Optimal control settings take into account energy reduction goals, worker comfort, and regulatory compliance. The system utilizes the data profile to determine initial, reduced-energy control settings based on expected physical conditions within the building.

Claims

1. An augmented reality (AR) system for monitoring and controlling environmental conditions within a building to reduce energy usage, the system comprising:

a display for displaying AR content to a user;

one or more sensors for gathering data of one or more environmental conditions of the building;

a location detector for providing location data of the user relative to the building; and

a computing device communicatively coupled to the display, the one or more sensors, and the location detector, the computing device comprising a processor that is configured by software to;

develop a time- and location-based profile of the one or more environmental conditions of the building using the data gathered by the one or more sensors at known locations over a period of time,

measure one or more current environmental conditions using the data gathered by the one or more sensors at a known location,

compare the one or more current environmental conditions at the known location with expected environmental conditions based on the developed profile, and

determine a control setting for the one or more environmental conditions based on the comparison between the one or more current and expected environmental conditions.

2. The AR system according to claim 1, wherein the processor is further configured by the software to:

generate AR content pertaining to the one or more environmental conditions within the building, and

transmit the AR content to the display.

3. The AR system according to claim 1, wherein the processor is further configured by the software to:

update the profile with the one or more current environmental conditions for the known location.

4. The AR system according to claim 2, wherein the processor is further configured by the software to:

generate a notification message when at least one current environmental condition is outside a range of a set-point by a predetermined amount.

5. The AR system according to claim 4, wherein the one or more sensors comprises at least one light sensor for gathering light data from light sources in the building, and wherein the one or more measured environmental conditions comprises at least one of light intensity and frequency.

6. The AR system according to claim 5, wherein the at least one light sensor comprises a charge-coupled device (CCD).

7. The AR system according to claim 5, wherein the profile of the one or more environmental conditions includes lighting intensity and the processor is further configured by the software to:

detect and characterize the light sources in the building using the light data.

8. The AR system according to claim 7, wherein the notification message includes an indication that at least one of the light sources may be burnt out or malfunctioning when the measured lighting intensity deviates from a lighting set-point by an amount indicative of such a condition.

9. The AR system according to claim 8, wherein the AR content includes at least one of:

the measured lighting intensity, lighting set-point, and an indicator of a malfunctioning or burnt-out light source.

10. The AR system according to claim 5, further comprising:

one or more depth sensors for gathering mapping data of physical objects in the building,

wherein the processor is further configured by the software to;

detect and track the number of people in the building using the mapping data, and

adjust a control setting to reduce energy usage for at least one of the environmental conditions when no people are detected within one or more zones within the building.

11. The AR system according to claim 10, wherein the one or more depth sensors comprise at least one optical 3D scanner, and wherein the processor is further configured by the software to:

construct a three-dimensional (3D) model of the building using the mapping data.

12. The AR system according to claim 11, wherein the control setting for the intensity of artificial light sources is calculated to supplement natural light sources to maintain the overall building light intensity within a range of the lighting set-point.

13. The AR system according to claim 11, wherein the one or more sensors comprises at least one temperature sensor for gathering temperature data from the building and wherein the one or more measured environmental conditions further comprises the ambient temperature.

14. The AR system according to claim 13, wherein the control settings of the climate-control system are calculated to maintain the ambient temperature inside the building within a range of a temperature set-point.

15. A method for monitoring and controlling one or more environmental conditions within a building using an augmented reality (AR) system, the method comprising:

providing an AR device configured to be worn by a user and operatively connected to a computing device, the AR device including one or more sensors, and a display system for presenting AR content generated by the computing device;

identifying a location of the AR device relative to the building;

measuring an environmental condition of the building using environmental data gathered by the one or more sensors at the identified location;

determining one of occupancy levels and location of workers therein;

creating and updating a time- and location-based data profile of the environmental condition;

comparing the measured environmental condition with an expected environmental condition based on a time- and location-based data map of the environmental condition within the building; and

adjusting a control setting if the measured environmental condition is outside of a range of a set-point.

16. The method according to claim 15, wherein the generated AR content comprises indicators of the measured environmental condition and set-point.

17. The method according to claim 15, wherein the measured environmental condition is one of lighting intensity and ambient temperature.

18. The method according to claim 15, further comprising:

displaying AR content instructing the user to move to a location different from the identified location.

19. An augmented reality (AR) device, comprising:

a depth sensor to gather mapping data of (i) physical objects in an environment and (ii) light sources in the environment;

a light sensor to gather light data from the light sources in the environment;

a display for displaying AR content overlapping a user'"'"'s perspective view of the environment; and

a processor communicatively coupled to the depth sensor, the light sensor, and the display, the processor configured by software to;

construct a map of the environment using the mapping data,

measure the lighting intensity at a known location using the light data gathered by the light sensor,

determine an optimal control setting for one or more artificial light sources based on a comparison of the measured lighting intensity against a user-selected lighting intensity set-point, and

generate a notification message if the measured light intensity at the known location deviates from an expected light intensity by an amount indicative of a malfunctioning or burnt-out light source.

20. The AR device of claim 19, further comprising:

a temperature sensor to gather temperature data from the environment;

wherein the processor is further configured to;

measure the temperature at the known location using the temperature data gathered by the temperature sensor,

determine an optimal control setting for a climate-control system based on the difference between a comparison of the measured temperature against a temperature set-point, and

create AR content corresponding to at least one of the measured temperature, the temperature set-point, measured lighting intensity, and lighting intensity set-point.