Augmented reality device, system, and method for safety

• US 10,360,728 B2
• Filed: 05/19/2015
• Issued: 07/23/2019
• Est. Priority Date: 05/19/2015
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Venkatesha, Sharath, Todeschini, Erik, Anderson, Donald
• Primary Examiner(s): Pringle-Parker, Jason A
• Application Number:
  US14/715,672
• Publication Number:
  US 20160343163A1
• Time in Patent Office: 1,526 Days
• CPC Class Codes: G01S 19/17 Emergency applications
• CPC Class Codes: G01S 2205/06 Emergency
• CPC Class Codes: G01S 5/02 using radio waves using satellite radio beacon systems for determining position G01S19/00
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G08B 21/14 Toxic gas alarms G08B21/16 takes precedence
• CPC Class Codes: G08B 7/06 using electric transmission , e.g. involving audible and visible signalling through the use of sound and light sources
• CPC Class Codes: G08B 7/066 guiding along a path, e.g. evacuation path lighting strip

Abstract

Disclosed is a device, system, and method utilizing augmented-reality (AR) for safety. The AR safety system generates hazard-response messages in response to sensor data in order to help users understand and response to hazards in work environments in an intuitive and effective way. The AR safety system fuses a plurality of technologies. These technologies include person location/tracking, hazard detection/tracking, and augmented reality (i.e., AR).

Claims

1. An augmented reality (AR) hazard display, comprising:

a user tracker tracking a user'"'"'s position in real-time;

an audio source to produce sounds and/or audio prompts for the user;

a communicator receiving real-time data indicative of a hazard from a plurality of sensors, the plurality of sensors comprising;

a gas sensor for detecting a gas and providing gas data, the gas sensor being positioned remotely from the augmented reality hazard display, the gas sensor being in a fixed geographical position; and

a weather sensor for providing wind data;

a processor configured to;

in response to the hazard, create a geo-fence indicative of virtual barriers surrounding a hazardous area of the hazard based on receiving the real-time data;

combine the user'"'"'s position tracked by the user tracker and the real-time data to provide an augmented reality (AR) message when the user enters the geo-fence, wherein the augmented reality (AR) message comprises two or more of a hazard identification indicative of an extent and/or severity of the hazard, a map of an area contaminated by hazard, a direction of hazard travel, and an estimation of future hazard movement;

in response to the hazard, estimating movement of the gas based on the gas data and the wind data;

combine the user'"'"'s position tracked by the user tracker and the estimated movement of the gas to provide an augmented reality (AR) message comprising visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via the audio source; and

an interface to convey the AR messages to the user.

2. The augmented reality (AR) hazard display according to claim 1, wherein the user tracker comprises a global positioning system (GPS) receiver.

3. The augmented reality (AR) hazard display according to claim 1, wherein the user tracker comprises at least one camera.

4. The augmented reality (AR) hazard display according to claim 1, wherein the user tracker comprises a wireless-network receiver.

5. The augmented reality (AR) hazard display according to claim 1, wherein the audio sources are configured to create three-dimensional (3D) audio.

6. The augmented reality (AR) hazard display according to claim 1, wherein the interface comprises a head mounted display (HMD).

7. The augmented reality (AR) hazard display according to claim 1, wherein the visual images of the map show the user'"'"'s position and a route to a safe place.

8. The augmented reality (AR) hazard display according to claim 1, wherein the augmented reality (AR) message provided when the user enters the geo-fence comprises prompts to guide a user'"'"'s movement.

9. An augmented reality (AR) safety system, comprising:

a hazard tracker comprising;

a gas sensor for detecting a gas and providing gas data, the gas sensor being positioned remotely from an AR hazard display; and

a weather sensor for providing wind data;

an audio source to produce sounds and/or audio prompts;

a user tracker for tracking a user'"'"'s position; and

the AR hazard display communicatively coupled to the hazard tracker, the audio source, and the user tracker, the AR hazard display adapted to;

in response to detected gas;

estimate movement of the gas based on the gas data and the wind data; and

create a geo-fence indicative of virtual barriers surrounding a hazardous area based on the estimated movement of the gas;

combine the tracked user-position and estimated movement of the gas to generate AR messages;

present the AR messages to the user when the user enters the geo-fence, the AR messages comprising two or more of hazard identity indicative of an extent and/or severity of the detected gas, a map of an area contaminated by the gas, a direction of gas travel, and an estimation of future gas movement;

present visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place based on the tracked user-position and the estimated movement of the gas; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via the audio source.

10. The augmented reality (AR) safety system according to claim 9, wherein the AR hazard display is a smartphone.

11. The augmented reality (AR) safety system according to claim 9, wherein the user tracker and/or the AR hazard display is worn by the user.

12. The augmented reality (AR) safety system according to claim 9, wherein the gas sensor detects an invisible gas.

13. The augmented reality (AR) safety system according to claim 9, wherein the AR messages comprise graphics to help a user visualize the gas.

14. The augmented reality (AR) safety system according to claim 9, wherein the AR messages comprise graphics to help a user visualize the direction in which the gas is moving.

15. The augmented reality (AR) safety system according to claim 9, wherein the AR messages comprise an alert that the user is near a hazardous area or has entered into a hazardous area.

16. The augmented reality (AR) safety system according to claim 9, wherein the AR messages comprise navigational prompts to guide a user to a safe area.

17. The augmented reality (AR) safety system according to claim 9, wherein the user tracker comprises at least one camera.

18. A computer-implemented method for generating an augmented-reality (AR) message, the method comprising:

gathering, in real-time, wind data from a weather sensor and gas data from a gas sensor, the gas sensor being in a fixed geographical position;

gathering, in real-time, user-position data from at least one user-tracking sensor;

in response to detected gas;

estimating movement of the gas based on the gas data and the wind data;

creating a geo-fence indicative of virtual barriers surrounding a hazardous area based on the estimated movement of the gas;

generating, based on the gathered user-position data and the gas data, an AR message when the user enters the geo-fence, the AR message comprising two or more of a hazard identity indicative of an extent and/or severity of the detected gas, a map of an area contaminated by the gas, a direction of gas travel, and an estimation of future gas movement;

presenting visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place based on the user-position data and the estimated movement of the gas; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via an audio source.