Emergency response augmented reality-based notification

• US 10,769,741 B2
• Filed: 05/29/2019
• Issued: 09/08/2020
• Est. Priority Date: 06/07/2013
• Status: Active Grant

• Current Assignee: Emergency University, Inc.
• Original Assignee: Emergency University, Inc.
• Inventors: Braun, Odelia, Braun, Doron, Johnston, Andrew Richard
• Primary Examiner(s): Ouellette, Jonathan P
• Application Number:
  US16/425,433
• Publication Number:
  US 20190279327A1
• Time in Patent Office: 468 Days
• Field of Search: 705 11-912
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/063112 Skill-based matching of a person or a group to a task
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 50/265 Personal security, identity or safety
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/60 Subscription-based services using application servers or record carriers, e.g. SIM application toolkits
• CPC Class Codes: H04W 4/90 Services for handling of emergency or hazardous situations, e.g. earthquake and tsunami warning systems [ETWS]

Abstract

Embodiments of the invention leverage mobile proliferation to enable laypersons to initiate a timely and effective emergency response in case of an emergency, such as a medical emergency, e.g. cardiac event. Mobile apps are made available as part of an organization'"'"'s overall response plan and program, allowing bystanders of emergency events to easily initiate notifications, for example to trained responders, for example in their facility, in a timely manner commensurate with the type of emergency specific to their facility. More particularly, embodiments of the invention use mobile applications to provide augmented reality-based alerts and notifications, for example to certified first trainees to respond to the scene of the emergency.

Claims

1. A computer implemented method for routing communications signals in response to an emergency, comprising:

providing a processor for receiving an input signal comprising an initial notification transmitted by a bystander at a location and indicating that there is an emergency at said location;

independently of a central emergency medical services (EMS) notification system, said processor extracting content from said signal comprising identification of a specific location of said emergency and identification of said emergency type;

said processor using said extracted content to identify laypersons at or near said location who are trained responders for said emergency and who are most qualified to respond to said emergency based upon the type of said emergency;

said processor prioritizing said identified most qualified trained responders based upon their training and qualification; and

independently of said central notification system, said processor transmitting a prioritized alert directly to at least said identified most qualified trained responders comprising said emergency type and said emergency location;

said alert further comprising the display of a dynamic map identifying and displaying any of the location of any of work force personnel, trained responders, emergency equipment, exit doors, and visual dynamic directions to a nearest exit.

2. The method of claim 1, further comprising generating said dynamic map by:

creating a map for said location to augment a display feature;

locating sensors and/or physical objects on said map, and

establishing routes to or away from said emergency location.

3. The method of claim 2, further comprising creating said map for said location by:

procuring an image, file, map, or other representation of all or part of a facility;

when an image or file is procured tracing said image into discernible vectors for extraction of required data;

extracting dimensions at said location of exterior walls, interior walls, and fixed objects;

translating said extracted dimensions to a representative coordinate system; and

inputting said representative coordinate system data into a database.

4. The method of claim 3, further comprising installing or Integrating said sensors or physical objects by:

installing physical sensors or objects in or around said location at designated positions; and

/or

integrating with existing physical sensors and/or physical objects.

5. The method of claim 3, further comprising;

inputting one or more sensor locations into said database;

establishing horizontal and vertical planes attached to floors, walls, and fixed objects at said location; and

placing digital markers at points divergent points, including points that require a change of direction, an elevator, or that are an end-point in a route.

6. The method of claim 5, further comprising:

creating a route between two selected physical sensors, objects, or digital markers;

determining distance between the two selected physical sensors, objects, or digital markers; and

inputting said selected physical sensors, objects, or digital markers s′

coordinates and distance into said database.

7. The method of claim 6, further comprising localizing an indoor person or object by:

using a combination of physical sensors, digital markers, or physical object locations to triangulate a position of a person or object to be routed from or to;

determining distance to the sensors;

using distance to triangulate the position of a person or object;

translating coordinates of the triangulated position to an augmented reality scene coordinate system; and

when the user moves from an indoor location to an outdoor location, switching to outdoor localizing.

8. The method of claim 6, further comprising localizing any of an outdoor person, object, or shape by:

using location data gathered from a user device or physical object location to determine a location of the person or object;

translating said location into an augmented reality scene coordinate system; and

when the user moves from an outdoor location to an indoor location, switching to indoor localizing.

9. The method of claim 6, further comprising:

using a minimum spanning tree algorithm after the user'"'"'s location is obtained to obtain an optimal route.

10. The method of claim 6, further comprising:

representing the route to the user by projecting a path into the augmented reality scene view;

representing movement by projecting arrows onto the path;

representing points of interest using icons;

hiding augmented scene objects behind a physical object;

re-routing the user if an obstruction is detected in the path by;

identifying a type of object;

for small objects, re-routing around the object; and

for large objects, disabling the route;

notifying all participants in the event of a disabled or blocked route; and

using machine learning to determine the type of obstruction.

11. The method of claim 1, further comprising:

displaying said dynamic map using multiple sensing methods to position a user inside or outside of a facility;

based on the user'"'"'s location, superimposing a route into an augmented scene to route the user from a current location to a calculated point of interest; and

based on new information from one or more sensing methods, directing the user towards a different route if their primary route becomes compromised;

wherein the new route information is immediately transmitted to nearby users to be re-routed as necessary.

12. A computer implemented method for routing communications signals in response to an emergency, comprising:

providing a processor for receiving an input signal comprising an initial notification transmitted by a bystander at a location and indicating that there is an emergency at said location;

independently of a central emergency medical services (EMS) notification system, said processor extracting content from said signal comprising identification of a specific location of said emergency and identification of said emergency type;

said processor using said extracted content to identify laypersons at or near said location who are trained responders for said emergency and who are most qualified to respond to said emergency based upon the type of said emergency;

said processor prioritizing said identified most qualified trained responders based upon their training and qualification; and

independently of said central notification system, said processor transmitting a prioritized alert directly to at least said identified most qualified trained responders comprising said emergency type and said emergency location;

said alert further comprising the display of a dynamic map identifying and displaying any of the location of any of work force personnel, trained responders, emergency equipment, exit doors, and visual dynamic directions to a nearest exit.

13. The method of claim 12, further comprising:

said processor identifying location and operational status of emergency equipment based on parameters that include any of most recent successful maintenance inspection, expiration dates that are relevant to critical equipment, components, supplies, and medication that require routine replacement due to their age and/or expiration date; and

said processor providing notifications to trained responders who have indicated that they are responding, said notifications identifying nearest appropriate emergency equipment.

14. The method of claim 12, further comprising:

when GPS is available, said processor using emergency event GPS location and dynamically comparing said emergency event GPS location with a closest location as determined by GPS to identify appropriate trained responders; and

when GPS is not available said processor using pre-configured location data.

15. The method of claim 12, further comprising:

said processor identifying location and operational status of emergency equipment based on parameters that include most recent confirmation of GPS location.

16. The method of claim 12, further comprising:

said processor receiving a confirming response from trained responders who respond to said alert; and

said processor providing a notification to said bystander indicating receipt of said notification from said trained responders who respond to said alert.

17. The method of claim 12, further comprising:

said processor using multiple real-time notification channels to immediately communicate between said bystander and said responder.

18. The method of claim 12, further comprising:

responsive to any of said initial notification and said content, said processor providing said bystander with any of specific text-based, graphic, video, and voice instructions on actions to perform in response to said emergency.

19. The method of claim 12, further comprising:

said processor providing communication and activity tracking of all communications and times at which actions are taken.

20. The method of claim 12, further comprising:

said processor receiving GPS location information for emergency equipment and confirmation that said emergency equipment is operational.

21. The method of claim 12, further comprising:

said emergency comprising a sudden cardiac arrest (SCA); and

any of said processor using geo-location to identify said trained responders based on their location when providing said confirming response;

said processor providing identification of AEDs based upon proximity and operational readiness;

said processor providing notifications to said trained responders identifying a nearest AEDs prioritized by proximity; and

said processor providing communication and activity tracking of any of time a responder arrived, if and when an AEC is attached, and when an EMS arrived.

22. The method of claim 12, further comprising:

said processor communicating instructions to individuals based on any of a type of emergency, organizational structure, pre-defined roles, equipment availability, training and experience, and an organization'"'"'s emergency policies, procedures, protocols, and emergency response plans.

23. The method of claim 12, further comprising:

using AR, nodes or sensors to track individual personnel'"'"'s location automatically indoors and outdoors, including exiting floor, exiting building, and arriving at assembly points.

24. The method of claim 12, further comprising:

defining one or more digital markers, each of said digital markers comprising a set of coordinates that define an interim point of interest; and

using a relational algorithm and coordinate system to create one or more routes with said digital markers.

25. The method of claim 24, further comprising:

said digital markers comprising a set of coordinates captured by an imaging device for an augmented reality scene.

26. The method of claim 12, further comprising:

using machine learning to automatically detect obstructions and capture, track, and report dangers and occluded routes.

27. The method of claim 12, further comprising:

providing redundant communication mechanisms to communicate data between personnel when cellular, Internet connectivity, and/or Wi-Fi is unavailable.

42. The method of claim 12, further comprising:

said dynamic map providing an augmented reality display for any of;

directing workforce personnel during evacuations and leading them to pre-designated routes, including a designated floor exit, building exit, and/or assembly point;

directing workforce personnel to designated shelters proximate to their location;

directing responders to ill or injured personnel who have indicated that they are ill or in distress;

directing responders to nearest functioning emergency equipment;

directing personnel away from hazards or danger zones resulting from an ongoing emergency;

directing personnel away from dynamic danger zones;

drawing danger zones corresponding with unsafe areas that are unavailable routes and optimizing the user'"'"'s route to safety; and

attaching photos of suspicious activity and persons.

28. A computer implemented method for routing communications signals in response to an emergency, comprising:

providing a processor for receiving an input signal comprising an initial notification transmitted by a bystander at a location and indicating that there is an emergency at said location;

independently of a central emergency medical services (EMS) notification system, said processor extracting content from said signal comprising identification of a specific location of said emergency and identification of said emergency type;

said processor exchanging information with individual workforce members comprising any of notification to evacuate, where to evacuate to, confirmation of receipt of notification by workforce, evacuation from building confirmation, arrival at assembly point confirmation, notification of an emergency, and notification of individual workforce members who have issued a distress notification;

independently of said central notification system, said processor transmitting an alert directly to said individual workforce members

comprising the display of a dynamic map identifying and displaying any of the location of any of work force personnel, trained responders, emergency equipment, exit doors, and visual dynamic directions to a nearest exit.

29. The method of claim 28, further comprising:

said processor providing an incident commander with aggregate, continually updated reports of individuals who have successfully evacuated by points of evacuation;

receipt of notification, including GPS location;

evacuation from building, with new GPS location; and

arrival at an assembly point by GPS location.

30. The method of claim 28, further comprising:

said processor providing an incident commander with distress notifications from individual workforce members; and

said processor triaging said distress notifications to security and emergency response team members.

31. The method of claim 28, further comprising:

said processor providing an incident commander with aggregate, continually updated reports of distress notification by degree of triage assigned.

32. The method of claim 28, further comprising:

said processor providing any of coded or time-limited coded drills for designated responders, or classes and non-coded, facility wide drills to simulate actual emergencies.

33. The method of claim 28, further comprising:

said processor incorporating time-based prompts to ensure necessary individuals are notified, and necessary actions are timely performed.

34. The method of claim 28, further comprising:

said processor ascertaining whether a workforce member whose cell phone is turned on is still in a facility if said workforce member has not confirmed receipt of a notification.

35. The method of claim 28, further comprising:

said processor informing said bystander how many responders have been notified.

36. The method of claim 28, further comprising:

said processor identifying an individual having known disabilities, as specified during said individual'"'"'s registration; and

said processor issuing a pre-configured distress signal during an evacuation with specific instructions for a trained responder, and location of nearest appropriate equipment based upon the disability, to assist said individual with equipment necessary to ensure they are able to evacuate said individual.

37. The method of claim 28, further comprising:

using AR, nodes or sensors to track individual personnel'"'"'s location automatically indoors and outdoors, including exiting floor, exiting building, and arriving at assembly points.

38. The method of claim 28, further comprising:

defining one or more digital markers, each of said digital markers comprising a set of coordinates that define an interim point of interest; and

using a relational algorithm and coordinate system to create one or more routes with said digital markers.

39. The method of claim 38, further comprising:

said digital markers comprising a set of coordinates captured by an imaging device for an augmented reality scene.

40. The method of claim 28, further comprising:

using machine learning to automatically detect obstructions and capture, track, and report dangers and occluded routes.

41. The method of claim 28, further comprising:

providing redundant communication mechanisms to communicate data between personnel when cellular, Internet connectivity, and/or Wi-Fi is unavailable.

43. The method of claim 28, further comprising:

said dynamic map providing an augmented reality display for any of;

directing workforce personnel during evacuations and leading them to pre-designated routes, including a designated floor exit, building exit, and/or assembly point;

directing workforce personnel to designated shelters proximate to their location;

directing responders to ill or injured personnel who have indicated that they are ill or in distress;

directing responders to nearest functioning emergency equipment;

directing personnel away from hazards or danger zones resulting from an ongoing emergency;

directing personnel away from dynamic danger zones;

drawing danger zones corresponding with unsafe areas that are unavailable routes and optimizing the user'"'"'s route to safety; and

attaching photos of suspicious activity and persons.