Cognitive load reducing platform for first responders

• US 10,417,497 B1
• Filed: 04/29/2019
• Issued: 09/17/2019
• Est. Priority Date: 11/09/2018
• Status: Active Grant

• Current Assignee: Qwake Technologies
• Original Assignee: Qwake Technologies
• Inventors: Cossman, Sam J., Long, II, John Davis, Haciomeroglu, Omer, Ralston, Michael E.
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US16/398,102
• Time in Patent Office: 141 Days
• Field of Search: 345633
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 17/125 by using a video camera to detect fire or smoke
• CPC Class Codes: G08B 21/16 Combustible gas alarms
• CPC Class Codes: G08B 29/186 Fuzzy logic; neural networks
• CPC Class Codes: G08B 7/066 guiding along a path, e.g. evacuation path lighting strip
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/38 for collecting sensor information

Abstract

A cognitive load reducing platform comprises one or more sensors that collect information about an environment as sensor data. A processor is in communication with the one or more sensors, wherein the processor executes a cognitive enhancement engines that processes the sensor data into enhanced characterization data having a reduced amount of data compared to the sensor data. An output device electronically communicate the enhanced characterization data to a user such that the enhanced characterization data reduces the cognitive load of the user, wherein the sensor and the output device comprise an assisted perception module worn by the user during an incident. A command interface device remote from the user to enable a person of authority to manage the incident and the user by receiving and displaying the enhanced characterization data, and by transmitting data and commands back to the assisted perception module.

Claims

1. A cognitive load reducing platform, comprising:

one or more sensors that collect information about an environment as sensor data;

a processor complex comprising one or more processors in communication with the one or more sensors, wherein the one or more processors execute one or more software-based cognitive enhancement engines that process the sensor data from the one or more sensors into enhanced characterization data having a reduced amount of data compared to the sensor data;

one or more output devices to electronically communicate the enhanced characterization data to a user such that the enhanced characterization data is integrated into natural senses of the user and optimized for the performance of a specific task of the user to reduce the cognitive load of the user, wherein at least one of the one or more sensors and at least one of the one or more output devices comprise an assisted perception module worn by the user during an incident to enable the user to navigate in dark or smoke-filled environments;

a command and control interface displayed on a display device remote from the user to enable a person of authority to manage the incident and user by receiving and displaying the enhanced characterization data from the assisted perception module, and by transmitting data and commands back to the assisted perception module; and

a cloud service in communication over a network with the command and control interface to perform at least one of data aggregation, predictive analytics, and archiving on telemetry data received from the command and control interface and the display device, the telemetry data comprising at least one of the sensor data and the enhanced characterization data, wherein one of the command and control interface and the cloud service aggregates the telemetry data from the assisted perception modules to construct a map of the interior of a structure in which the incident occurs.

2. The cognitive load reducing platform of claim 1, wherein multiple users are equipped with respective assisted perception modules, the command and control interface further comprising multiple sub-panels to display the enhanced characterization data from each of the assisted perception modules, wherein in a first communication mode, any of the subpanels is selected to engage in audio or visual icon based communication, and wherein in a second communication mode, the audio or visual icon based communication is broadcast to all of the assisted perception modules.

3. The cognitive load reducing platform of claim 1, wherein the map includes a location of hotspots and a virtual path marker showing paths the users took to a current location in the structure to aid in rapid egress.

4. The cognitive load reducing platform of claim 1, wherein the cloud service automatically archives the telemetry data in a repository.

5. The cognitive load reducing platform of claim 4, wherein the cloud service operates as a software as a service (SaaS) wherein statistical data generated by the cloud service is made available to customers of the cloud service.

6. The cognitive load reducing platform of claim 4, wherein, the cloud service aggregates the telemetry data and third party data collected over the network, the third party data including maps, weather conditions, building schematics, time/date, and building control information.

7. The cognitive load reducing platform of claim 6, wherein the cloud service aggregates the telemetry data and third party data according to any combination of:

first responder unit, incident, first responder department, city, county, state, country, continent, and worldwide for crowdsourced data collection.

8. The cognitive load reducing platform of claim 1, wherein the processor complex is one of part of the assisted perception module and located on a remote server.

9. The cognitive load reducing platform of claim 1, wherein the assisted perception module is integrated with a mask or helmet of a self-contained breathing apparatus (SCBA), and wherein the sensor comprise a thermal imaging camera (TIC) and the one or more cognitive enhancement engines executed by the processor comprises one or more edge enhancement engines to processes thermal images from the TIC to enhance the edges of objects and declutter information in the thermal images.

10. A method comprising:

collecting, using one or more sensors, information about an environment as sensor data;

executing, by one or more processors in communication with the one or more sensors, one or more software-based cognitive enhancement engines that process the sensor data from the one or more sensors into enhanced characterization data having a reduced amount of data compared to the sensor data;

electronically communicating, using one or more output devices, the enhanced characterization data to a user such that the enhanced characterization data is integrated into natural senses of the user and optimized for the performance of a specific task of the user to reduce the cognitive load of the user, wherein at least one of the one or more sensors and at least one of the one or more output devices comprise an assisted perception module worn by the user during an incident to enable the user to navigate in dark or smoke-filled environments; and

displaying a command and control interface on a display device remote from the user to enable a person of authority to manage the incident and user by receiving and displaying the enhanced characterization data from the assisted perception module, and by transmitting data and commands back to the assisted perception module; and

transmitting telemetry data from the command and control interface and the display device to a cloud service in communication over a network with the command and control interface to perform at least one of data aggregation, predictive analytics, and archiving on the telemetry data, the telemetry data comprising at least one of the sensor data and the enhanced characterization data, wherein one of the command and control interface and the cloud service aggregates the telemetry data from the assisted perception modules to construct a map of the interior of a structure in which the incident occurs.

11. The method of claim 10, further comprising:

using, by multiple users, respective assisted perception modules, and configuring the command and control interface with multiple sub-panels to display the enhanced characterization data from each of the assisted perception modules, wherein in a first communication mode, any of the subpanels is selected to engage in audio or visual icon based communication, and wherein in a second communication mode, the audio or visual icon based communication is broadcast to all of the assisted perception modules.

12. The method of claim 10, further comprising:

including on the map a location of hotspots and a virtual path marker showing paths the users took to a current location in the structure to aid in rapid egress.

13. The method of claim 10, further comprising:

automatically archiving the telemetry data in a repository by the cloud service.

14. The method of claim 13, further comprising:

operating the cloud service as a software as a service (SaaS) wherein statistical data generated by the cloud service is made available to customers of the cloud service.

15. The cognitive load reducing platform of claim 13, further comprising:

aggregating, by the cloud service, the telemetry data and third party data collected over the network, the third party data including maps, weather conditions, building schematics, time/date, and building control information.

16. The cognitive load reducing platform of claim 15, further comprising aggregating, by the cloud service, the telemetry data and third party data according to any combination of:

first responder unit, incident, first responder department, city, county, state, country, continent, and worldwide for crowdsourced data collection.

17. An executable software product stored on a non-transitory computer-readable medium containing program instructions for reducing cognitive load of a user, the program instructions for:

collecting, using one or more sensors, information about an environment as sensor data;

processing, by one or more cognitive enhancement engines the sensor data from the one or more sensors into enhanced characterization data having a reduced amount of data compared to the sensor data;

electronically communicating, using one or more output devices, the enhanced characterization data to a user such that the enhanced characterization data is integrated into natural senses of the user and optimized for the performance of a specific task of the user to reduce the cognitive load of the user, wherein at least one of the one or more sensors and at least one of the one or more output devices comprise an assisted perception module worn by the user during an incident to enable the user to navigate in dark or smoke-filled environments; and

displaying a command and control interface on a display device remote from the user to enable a person of authority to manage the incident and user by receiving and displaying the enhanced characterization data from the assisted perception module, and by transmitting data and commands back to the assisted perception module; and

transmitting telemetry data from the command and control interface and the display device to a cloud service in communication over a network with the command and control interface to perform at least one of data aggregation, predictive analytics, and archiving on the telemetry data, the telemetry data comprising at least one of the sensor data and the enhanced characterization data, wherein one of the command and control interface and the cloud service aggregates the telemetry data from the assisted perception modules to construct a map of the interior of a structure in which the incident occurs.

18. The non-transitory computer-readable medium of claim 17, further comprising:

a cloud service in communication over a network with the command and control interface to perform data aggregation, predictive analytics, and archiving on one or more of the sensor data and the enhanced characterization images.