Method and system for locating and monitoring first responders

• US 8,688,375 B2
• Filed: 05/31/2007
• Issued: 04/01/2014
• Est. Priority Date: 05/31/2006
• Status: Active Grant

• Current Assignee: TRX Systems, Inc.
• Original Assignee: TRX Systems, Inc.
• Inventors: Funk, Benjamin E., Bandyopadhyay, Amrit, Kohn, Eric A., Goldsman, Neil, Teolis, Carole A., Blankenship, Gilmer L.
• Primary Examiner(s): Shafi, Muhammad
• Application Number:
  US11/756,412
• Publication Number:
  US 20080077326A1
• Time in Patent Office: 2,497 Days
• Field of Search: 701/220, 701/221, 701/300, 701400-541, 340/539.13, 340/568.1, 340/573.1, 340988-996, 345/633, 345/156
• US Class Current: 701/495
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled
• CPC Class Codes: G08B 25/016 Personal emergency signalling and security systems emergency non-personal manually actuated alarm activators G08B25/12
• CPC Class Codes: G08G 1/20 Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles

Abstract

The present invention is directed to methods and systems for locating and monitoring the status of people and moveable assets, such as first responders, including firefighters and other public service personnel, and their equipment both indoors and out. The invention can provide for locating and monitoring the status of people and assets in environments where GPS systems do not operate, or where operation is impaired or otherwise limited. The system and method uses inertial navigation to determine the location, motion and orientation of the personnel or assets and communicates with an external monitoring station to receive requests for location, motion orientation and status information and to transmit the location, motion orientation and status information to the monitoring station. The system and method can include storing the location, motion and orientation data as well as status data, in the event that the communication system is unable to communicate with and transmit information to the monitoring station, the system will wait until communication is restored and transmit the status information to the monitoring station to update the location, motion orientation and status information for the person or asset.

Claims

1. A system for tracking a mobile person or asset, comprising:

a portable system associated with a person or asset being tracked, the portable system comprising at least an inertial navigation unit including at least one accelerometer, at least one gyroscope, at least one magnetic field sensor, and a processor that is configured to generate information for the person or asset based on sensor data received from one or more of the at least one accelerometer, the at least one gyroscope, or the at least one magnetic field sensor, wherein the information includes at least three-dimensional location coordinates, and data about one or more of posture of the person or asset, movements of the person or asset, or features associated with the environment in which the person or asset is being tracked; and

a computer configured to receive, from the portable system, the information generated by the processor and to calculate a path from the information that tracks the person or asset, wherein the computer is further configured to develop one or more location-related sensor profiles from the information and to infer one or more building features from the information, and wherein the computer is further configured to execute one or more processing algorithms to make corrections to the three-dimensional location coordinates based on the one or more location-related sensor profiles and the one or more building features and to recalculate the path thereby reducing tracking errors in the path.

2. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit further includes data identifying a segment of the path with which the three-dimensional location coordinates are associated.

3. The system of claim 1, wherein the portable system includes a transceiver in operative communication with the inertial navigation unit.

4. The system of claim 1, wherein the portable system is worn by the person to be tracked.

5. The system of claim 1, wherein the portable system includes a communications module in operative communication with the inertial navigation unit, and wherein the inertial navigation unit and the communications module comprise a single integrated device.

6. The system of claim 1, wherein the portable system includes a communications module in operative communication with the inertial navigation unit, and wherein the inertial navigation unit and communications module establish a wireless personal area network (WPAN) on the person to enable the addition of one or more wireless sensors.

7. The system of claim 1, wherein the portable system further comprises one or more environmental sensors.

8. The system of claim 1, wherein the processor of the portable system compares sensor data received from one or more of the at least one accelerometer, at least one gyroscope, or at least one magnetic field sensor to predefined gait motion information to recognize the nature of the movement of the person.

9. The system of claim 1, wherein sensor data received from one or more of the at least one accelerometer, at least one gyroscope, or at least one magnetic field sensor is used to generate a library of human movement sensor data that can be used to characterize human locomotion.

10. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is turning when an angle change measured over a stride of the person by the at least one gyroscope exceeds a threshold value.

11. The system of claim 10, wherein the end of the turn is determined when an angle change measured over a stride of the person by the at least one gyroscope falls below the threshold value.

12. The system of claim 11, wherein a portion of the path of the person that starts at the end of a turn and ends when the end of another turn is detected is defined as a segment.

13. The system of claim 12, wherein each segment is assigned a segment number, and wherein the segment number increments by one every time the end of another turn is detected.

14. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is moving in a hallway when a number of steps taken by the person, as determined from sensor data received from the at least one accelerometer, exceeds a threshold value, and when a gyroscope angle measured by the at least one gyroscope has not exceeded a threshold value.

15. The system of claim 14, wherein the information generated by the processor of the inertial navigation unit includes data identifying a segment of the path with which the three-dimensional location coordinates are associated, and wherein each set of three-dimensional location coordinates determined to be in the hallway is associated with the same segment.

16. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is stopped when the three-dimensional location coordinates generated by the processor have not changed for a predetermined period of time.

17. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is still when a variance of total acceleration, as determined from sensor data received from the at least one accelerometer, falls below a predetermined threshold value.

18. The system of claim 17, wherein an alarm is triggered at the computer when the information received from the portable system includes data indicating that the person is still.

19. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is standing upright when sensor data received from the at least one accelerometer indicates that the person is not tilting past a threshold value.

20. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is crawling when sensor data received from the at least one accelerometer indicates that the person is tilting forward past a threshold value.

21. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is lying on his or her back when sensor data received from the at least one accelerometer indicates that the person is tilting backward past a threshold value.

22. The system of claim 1, wherein the processor of the inertial navigation unit utilizes a neural network as a pattern recognition tool for detecting different types of movement of the person based on sensor data received from one or more of the at least one accelerometer, the at least one gyroscope, or the at least one magnetic field sensor.

23. The system of claim 22, wherein the neural network includes, as input, training data obtained by having a user walk a path that includes level ground, up stairs, and down stairs.

24. The system of claim 22, wherein the neural network classifies each individual step taken by the person as either being on level ground, up a stair, or down a stair.

25. The system of claim 24, wherein the neural network classifies each individual step based on a shape of the signal received from the at least one accelerometer for each individual step.

26. The system of claim 25, wherein the shape of the signal is measured between two consecutive heel strikes.

27. The system of claim 24, wherein a stair count is obtained of each individual step taken by the person if the step is up a stair or down a stair.

28. The system of claim 24, wherein the information generated by the processor of the inertial navigation unit includes data indicating whether the person has taken a step up a stair or down a stair.

29. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes a heading angle change determined, based on sensor data received from the at least one gyroscope, by taking an average heading angle change over a single stride of the person.

30. The system of claim 29, wherein the heading angle change for the single stride is determined to be caused by sensor drift and removed from an accumulated gyro angle if the heading angle change is lower than a threshold value.

31. The system of claim 1, wherein the at least one magnetic field sensor comprises a digital compass, and wherein an angle from the at least one gyroscope and an angle from the digital compass are fused by the processor of the inertial navigation unit to estimate a more accurate azimuth.

32. The system of claim 31, wherein data from the at least one gyroscope is used to provide high frequency angle control and data from the digital compass is used to provide low frequency angle control when data from the digital compass is determined to be good.

33. The system of claim 31, wherein the sensor fusion is implemented as a feedback control.

34. The system of claim 1, wherein the computer is configured to receive and process information transmitted from a plurality of portable systems associated with a plurality of people or assets being tracked, and wherein received signal strength indication (RSSI) levels of incoming receptions are used to estimate range for portable systems close to one another, the estimate further providing a homing capability.

35. The system of claim 1, wherein the computer includes a graphical user interface, the graphical user interface displaying position estimates of the person based on the information received from the portable system.

36. The system of claim 35, wherein the graphical user interface further displays the status of one or more batteries used to power the portable system.

37. The system of claim 35, wherein the graphical user interface further displays an indication as to whether the person is moving or not moving.

38. The system of claim 35, wherein the graphical user interface further displays a posture indicator for the person, the posture indicator providing information regarding whether the person is one of walking, crawling, or lying on his or her back.

39. The system of claim 35, wherein position estimates of the person are shown differently on the graphical user interface depending on whether the person is indoors or outdoors.

40. The system of claim 1, wherein the computer includes a graphical user interface, the graphical user interface displaying position estimates of the person, based on the information received from the portable system, relative to one or more surrounding landmarks when the person is outdoors.

41. The system of claim 1, wherein the computer includes a graphical user interface, the graphical user interface displaying position estimates of the person overlayed on an image of a floor plan based on the information received from the portable system.

42. The system of claim 1, wherein the computer is in operative communication with a building database, the building database including, for each building stored therein, one or more of global positioning system (GPS) data, satellite images, pre-plan notes about the building, a geographic information systems (GIS) polygon representation, or building maps or floor plans.

43. The system of claim 1, wherein the computer is configured to execute a program to improve the accuracy of the information received from the portable system.

44. The system of claim 1, wherein the computer is configured to execute a map pre-processing program that enables a user to record, via a graphical user interface, one or more floor plan features from one or more floor plan images, and wherein a floor plan feature includes one or more of a corridor intersection, a stairwell location, an elevator location, or an exit location.

45. The system of claim 44, wherein corridor intersections are represented by nodes, and wherein two or more nodes are determined to be a part of the same corridor if they are not separated by any obstructions.

46. The system of claim 45, wherein information is stored for a corridor that identifies at least a corridor start node, a corridor end node, a slope of the corridor, and a list of any corridor nodes encountered between the start node and the end node along the corridor.

47. The system of claim 1, wherein the computer is configured to execute a map matching program that displays position estimates of the person, based on the information received from the portable system, on a pre-processed floor plan and makes corrections to the position estimates based on recorded floor plan features.

48. The system of claim 1, wherein the computer is configured to execute a map matching program that correlates position estimates of the person, based on the information received from the portable system, to one or more features on a map, and wherein a feature includes one or more of a corridor or stairway.

49. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data identifying a segment of the path with which the three-dimensional location coordinates are associated, and wherein the computer is configured to execute a map matching program that determines the segment to be along a corridor if a best-fit line over the segment has a deviation less than a threshold value and a length greater than another threshold value.

50. The system of claim 49, wherein the map matching program determines, for each corridor on a map, a displacement of a first point on the segment from the corridor, a required rotation to correct the first point to the corridor, and a closest point on the corridor to make the correction to.

51. The system of claim 50, wherein the first point on the segment is corrected to a particular corridor along with all other points on the segment if the first point is within the bounds of the corridor, and the correction distance and rotation are below threshold values.

52. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating an elevation change, as determined from sensor data received from the at least one accelerometer, and wherein the computer is configured to execute a program that utilizes this data to identify stairwells and to correct a position estimate of the person to the closest stairwell on a map.

53. The system of claim 52, wherein data indicating an elevation change comprises incrementing or decrementing “

z”

position values as determined from sensor data received from the at least one accelerometer, and wherein each “

z”

position value change corresponds to a stair encountered by the person.

54. The system of claim 53, wherein a change in the “

z”

position value indicates that the person has started moving on a possible stairwell, and wherein movement on the possible stairwell has ended when the change in the “

z”

position value is zero over a threshold period.

55. The system of claim 54, wherein a total change in “

z”

position over a possible stairwell is ignored if the number of stairs encountered is less than a threshold.

56. The system of claim 53, wherein the program resolves elevation change data into floor numbers of a building by comparing the number of stairs counted to the expected number of stairs between floors in the building.

57. The system of claim 52, wherein the program records the winding of a stairwell when discovered, the winding being either clockwise, counter clockwise, or straight, and further establishes the need to wind in the opposite direction if an opposite elevation change occurs.

58. The system of claim 1, wherein the computer is configured to execute a map building program that can generate a floor plan of a building based on the information received from the portable system.

59. The system of claim 58, wherein a new corridor is recorded when the information generated by the processor of the inertial navigation unit indicates that a number of steps taken by the person, as determined from data received from the at least one accelerometer, exceeds a threshold value, and when a change in gyroscope angle measured by the at least one gyroscope has not exceeded a threshold value.

60. The system of claim 59, wherein the corridor is assumed to lie on a cardinal direction of the building if an accumulated gyroscope angle remains small for a long period of time.

61. The system of claim 58, wherein a general shape of the building is obtained from geographic information systems (GIS) vectors of the building.

62. The system of claim 58, wherein the map building program generates a floor plan of the building by combining data received from a plurality of portable systems associated with a plurality of people or assets being tracked in the building.

63. The system of claim 62, where the reliability of features recorded on a generated floor plan increases when confirmed by data received from a plurality of portable systems associated with a plurality of people or assets being tracked in the building.

64. The system of claim 58, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is moving in a corridor, and wherein the map building program corrects a position estimate of the person to a previously discovered corridor on the floor plan if the previously discovered corridor is close by, or else records the corridor as a new corridor.

65. The system of claim 58, wherein the location, winding, start point, and end point of discovered stairwells are recorded, and used to match floor changes of other people or assets being tracked in the building.

66. The system of claim 1, wherein the computer is configured to execute a map matching program to improve the accuracy of the information received from the portable system when the person is indoors, and to execute an outdoor algorithm to improve the accuracy of the information received from the portable system when the person is outdoors.

67. The system of claim 66, wherein, as the person moves from outdoors to indoors, a last position estimate and rotation determined by the outdoor algorithm are used by the map-matching program as a starting position estimate and rotation.

68. The system of claim 66, wherein, as the person moves from outdoors to indoors, the map-matching program determines which entrance or exit on a building floor plan is closest to a latest outdoor position estimate, and uses the determined entrance or exit as a start point for indoor tracking.

69. The system of claim 66, wherein, as the person moves from indoors to outdoors, a last position estimate and rotation determined by the map-matching program are used by the outdoor algorithm as a starting position estimate and rotation.

70. The system of claim 66, wherein as the person moves from indoors to outdoors, the map-matching program determines which entrance or exit on a building floor plan is closest to a latest indoor position estimate, and uses the determined entrance or exit as a start point for outdoor tracking.

71. The system of claim 1, wherein the portable system further comprises a GPS receiver, and wherein the computer executes an outdoor algorithm to fuse data from the inertial navigation unit and the GPS receiver when the person is outdoors.

72. The system of claim 71, wherein one or more GPS parameters are used to validate GPS data, and wherein the one or more GPS parameters include one or more of dilution of precision, number of satellites used, location of satellites used, or signal strength of satellites used.

73. The system of claim 71, wherein GPS position data and inertial navigation unit position data are synched, and wherein incremental distances between the GPS position data are compared to incremental distances between the inertial navigation unit position data to validate the GPS position data.

74. The system of claim 71, wherein a feedback algorithm utilizes validated GPS position data to correct a track of the person by converging a fused estimate of GPS position data and inertial navigation unit position data toward the validated GPS position data.

75. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is moving in a corridor, and wherein the computer is configured to execute one or more processing algorithms to correct a current direction of the person to that of a stored building grid angle for a building if the current direction is within an error of margin of the stored building grid angle.

76. The system of claim 1, wherein the information generated by the processor of the inertial navigation unit includes data indicating that the person is moving in a corridor of a building, and wherein a current direction of the person is stored as a building grid angle for the building to be used for future position estimate corrections.

77. The system of claim 1, wherein the at least one magnetic field sensor measures a magnetic field as a function of position, and wherein a location-related sensor data profile is a magnetic sensor data profile comprising a specific magnetic field distribution associated with a specific location.

78. A system for tracking a mobile person or asset, comprising:

a portable system associated with a person or asset being tracked, the portable system comprising at least an inertial navigation unit including at least one accelerometer, at least one gyroscope, and a processor that is configured to generate information for the person or asset based on sensor data received from one or more of the at least one accelerometer or the at least one gyroscope, wherein the information includes at least three-dimensional location coordinates, and data identifying a segment of a path with which the three-dimensional location coordinates are associated; and

a computer configured to receive, from the portable system, the information generated by the processor and to calculate a path from the information that tracks the person or asset, wherein the computer is configured to develop one or more location-related sensor profiles from the information and to infer one or more building features from the information, wherein the computer is configured to execute one or more processing algorithms to make corrections to the three-dimensional location coordinates based on the one or more location-related sensor data profiles and the one or more building features and to recalculate the path thereby reducing tracking errors in the path, and wherein the computer is further configured to execute a map matching program that determines the segment to be along a building corridor in response to a best-fit line over the segment having a deviation less than a threshold value and a length greater than another threshold value.

79. The system of claim 78, wherein the map matching program determines, for each building corridor on a map, a displacement of a first point on the segment from the building corridor, a required rotation to correct the first point to the building corridor, and a closest point on the building corridor to make the correction to.

80. The system of claim 78, wherein the first point on the segment is corrected to a particular building corridor along with all other points on the segment if the first point is within the bounds of the building corridor, and the correction distance and rotation are below threshold values.

81. A system for tracking a mobile person or asset, comprising:

a portable system associated with a person or asset being tracked, the portable system comprising at least an inertial navigation unit including at least one accelerometer, at least one gyroscope, and a processor that is configured to generate information for the person or asset based on sensor data received from one or more of the at least one accelerometer and the at least one gyroscope, wherein the information includes at least three-dimensional location coordinates, and data about one or more of posture of the person or asset, movements of the person or asset, or the environment in which the person or asset is being tracked; and

a computer configured to receive, from the portable system, the information generated by the processor and to calculate a path from the information that tracks the person or asset, wherein the computer is configured to develop one or more location-related sensor profiles from the information and to infer one or more building features from the information, wherein the computer is configured to execute one or more processing algorithms to make corrections to the three-dimensional location coordinates based on the one or more location-related sensor data profiles and the one or more building features and to recalculate the path thereby reducing tracking errors in the path, and wherein the computer is further configured to execute one or more processing algorithms to correct a current direction of the person or asset to that of a stored building grid angle for a building if the current direction is within an error of margin of the stored building grid angle.

82. A system for tracking a mobile person or asset, comprising:

a portable system associated with a person or asset being tracked, the portable system comprising at least an inertial navigation unit including at least one accelerometer, at least one gyroscope, and a processor that is configured to generate information for the person or asset based on sensor data received from one or more of the at least one accelerometer and the at least one gyroscope, wherein the information includes at least three-dimensional location coordinates, and data about one or more of posture of the person or asset, movements of the person or asset, or the environment in which the person or asset is being tracked; and

a computer configured to receive, from the portable system, the information generated by the processor to calculate a path from the information that tracks the person or asset, wherein the computer is configured to calculate a path from the information that tracks the person or asset, wherein the computer is configured to develop one or more location-related sensor profiles from the information and to infer one or more building features from the information, and wherein the computer is further configured to execute one or more processing algorithms to make corrections to the three-dimensional location coordinates based on the one or more location-related sensor profiles and the one or more building features to recalculate the path thereby reducing tracking errors in the path;

wherein a current direction of the person is stored as a building grid angle for the building to be used for future position estimate corrections.