COLLISION AVOIDANCE SYSTEM AND METHOD FOR AN UNDERGROUND MINE ENVIRONMENT

• US 20150254985A1
• Filed: 03/10/2014
• Published: 09/10/2015
• Est. Priority Date: 09/19/2011
• Status: Active Grant

• Current Assignee: Innovative Wireless Technologies Inc.
• Original Assignee: Innovative Wireless Technologies Inc.
• Inventors: Carpenter, Paul R., Fisher, Matthew A., Silverstrim, James E.
• Granted Patent: US 9,747,802 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 28/04 responsive to presence or absence of the driver, e.g. to weight or lack thereof
• CPC Class Codes: B60K 28/10 responsive to conditions relating to the vehicle
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/085 Taking automatic action to adjust vehicle attitude in preparation for collision, e.g. braking for nose dropping
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Described are methods and systems for collision avoidance in an underground mine environment that use one or more of a computer vision component, an asset tracking system, and a motion detection component for the purpose of determining and responding to potential collision threats. Imaging is captured and processed in real time, so that assets of interest can be identified and used in evaluating potential for collision with other assets. Location data from an asset tracking system is likewise evaluated and used to determine proximity of assets in relation to the host. A final input is provided by the motion detection component that intelligently determines movement patterns and direction of travel. Once these components'"'"' inputs are collectively evaluated, a proximity or a threat value is generated which determine an audible or visual signal or action to prevent collision and increase safety in unfavorable conditions.

Claims

1. A collision avoidance system for vehicles and humans, comprising:

a thermal imaging module for imaging one or more areas of interest and for providing object recognition for recognizing one or more of humans, vehicles, and electrical infrastructure;

an asset tracking module with communications and tracking software;

a motion module for providing direction and speed of travel of an asset;

collision avoidance software configured to provide for;

proximity detection of one or more of assets, vehicles, and humans;

threat determination based on a rating algorithm and multiple inputs; and

collision avoidance based on the threat determination.

2. The system of claim 1, wherein the thermal imaging module comprises long wave infrared cameras and the imaging is capable of being captured at a rate of 10 to 30 frames per second.

3. The system of claim 1, wherein the object recognition module comprises:

a computer vision system capable of performing one or more of;

object recognition on thermal image frames received as a video feed;

object recognition of humans;

object recognition of human faces;

object recognition of heat emitting infrastructure.

4. The system of claim 1, wherein the asset tracking module comprises a communications and tracking system capable of determining tracking locations based on RSSI calculations and known fixed node locations.

5. The system of claim 1, wherein the asset tracking module is configured to receive data from an accelerometer for determining direction and speed of travel.

6. The system of claim 1, wherein the proximity detection is based on inputs from the asset tracking module, the motion module, and the thermal imaging module, in which heat emitting objects are capable of being located in proximity to a host of the collision avoidance system.

7. The system of claim 1, wherein the threat determination is achieved by a rating algorithm with inputs from the thermal imaging module, the asset tracking module, and the motion module.

8. The system of claim 7, wherein the collision avoidance software further provides instructions for avoiding collisions between objects.

9. A collision avoidance system, the system comprising:

a computer vision component;

an asset tracking component;

a motion detection component for calculating a directional velocity component for the asset tracking component; and

a collision avoidance component configured to receive inputs from the computer vision component, asset tracking component, and motion detection component and to combine the inputs into an algorithm to produce an output that determines a warning or action for avoidance of a collision between a first asset and a second asset.

10. The collision avoidance system of claim 9, wherein the first asset is a vehicle or human and the second asset is a vehicle or human.

11. The collision avoidance system of claim 9, wherein the computer vision component comprises:

a video imaging component comprising one or more thermal infrared cameras configured to provide a thermal image capture of an asset;

a computer processor; and

a memory;

wherein the computer processor is configured to receive video image frames from the video imaging component and analyze them to identify assets according to a set of computer-executable instructions stored in the memory.

12. The collision avoidance system of claim 11, wherein the computer-executable instructions are configured to calculate the proximity of a human form by the size and heat intensity of the thermal image capture whereby a brighter image pixel indicates warmer areas and closer and larger humans span a greater number of pixels than do farther and smaller humans.

13. The collision avoidance system of claim 9, wherein the asset tracking component is based on fixed mesh radio nodes and mobile mesh radio nodes, wherein a mobile mesh radio node is placed on an asset and the asset tracking component is configured such that a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes determines the location of the mobile mesh radio node.

14. The collision avoidance system of claim 9, wherein:

the motion detection component comprises an accelerometer-based motion sensor device placed on the first asset and the motion detection component is configured to determine a directional velocity component comprising a speed and direction of travel of the first asset based on the accelerometer-based motion sensor;

the motion detection component comprises an accelerometer-based motion sensor device placed on the second asset and the asset tracking component is configured to determine a directional velocity component comprising a speed and direction of travel of the second asset based on the accelerometer-based motion sensor; and

the asset tracking component and the motion detection component are configured to determine an intersection point and time to intersection point of the first asset and second asset.

15. A collision avoidance system, the system comprising:

a computer vision component comprising;

an imaging modality comprising one or more thermal infrared cameras configured to provide a thermal image capture of a region of interest;

a computer processor; and

a memory comprising a set of computer-executable instructions configured for instructing the computer processor to analyze the thermal image capture to identify assets present in the region of interest;

an asset tracking component based on fixed mesh radio nodes and mobile mesh radio nodes, wherein a mobile mesh radio node is placed on a first asset and the asset tracking component is configured to determine the location of the mobile mesh radio node based on a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes;

a motion detection component configured to determine a directional velocity component for the asset tracking component based on an accelerometer-based motion sensor device placed on the first asset, wherein the directional velocity component comprises a speed and direction of travel; and

a collision avoidance component which is configured to receive inputs from the computer vision component, the asset tracking component, and the motion detection component and combine the inputs into an algorithm programmed in a set of computer-executable instructions which instruct a computer processor to calculate a Threat Rating Value that determines a warning or action for the first asset to avoid collision with a second asset.

16. The collision avoidance system of claim 15, wherein the first asset is a vehicle and the second asset is a vehicle or the first asset is a vehicle and the second asset is a human.

17. The collision avoidance system of claim 15, wherein the computer-executable instructions are configured to instruct the computer processor to calculate the Threat Rating Value as:

TRV=(K_VH·

A_VH·

V_VH)+(K_VO·

A_VO·

V_VO)+(K_TS·

max[TRV_TS1 . . . TRV_TSn])

wherein;

TRV_TS1=C_TS1·

D_TS1·

V_TS1

TRV_TSn=C_TSn·

D_TSn·

V_TSn

K_VH=Weight constant for the host computer vision component input

A_VH=Amplitude level for the host computer vision component input

V_VH=Value of the host computer vision component input

K_VO=Weight constant for the object computer vision component input

A_VO=Amplitude level for the object computer vision component input

V_VO=Value of the object computer vision component input

V_TS1=Value of the first asset tracking component input

TRV_TS1=Threat rating value for the n^th asset tracking component input

C_TSn=Confidence level for the n^th asset tracking component input

D_TSn=Directional velocity component for the n^th asset tracking component input

V_TSn=Value of the n^th asset tracking component input

TRV_TSn=Threat rating value for the n^th asset tracking system input

TRV=Threat rating value for the Collision Avoidance Component

18. A method for avoiding asset collisions, the method comprising:

providing a thermal image of a first asset with a computer vision component and identifying the type of asset with the computer vision component, the computer vision component comprising;

a video imaging component comprising one or more thermal infrared cameras which provide video image frames comprising a thermal image capture of an object;

a computer processor; and

a memory comprising a set of computer-executable instructions which instruct the computer processor to analyze video image frames received from the video imaging component to identify assets present in the thermal image capture, wherein the set of computer-executable instructions employ object classification algorithms to identify the asset;

tracking the location of the first asset with an asset tracking component comprising fixed mesh radio nodes and mobile mesh radio nodes, wherein a mobile mesh radio node is placed on a first vehicle or a human and the asset tracking component determines the location of the mobile mesh radio node based on a Received Signal Strength Indication (RSSI) between the mobile mesh radio node and surrounding fixed mesh radio nodes;

tracking the speed and direction of travel of the first asset with a motion detection component which determines a directional velocity component for the asset tracking component based on an accelerometer-based motion sensor device placed on the first asset, wherein the directional velocity component comprises a speed and direction of travel;

determining a Threat Rating Value through a collision avoidance component which receives inputs from the computer vision component, asset tracking component, and motion detection component and combines the inputs into an algorithm programmed in a set of computer-executable instructions which instruct a processor to calculate the Threat Rating Value; and

issuing a warning or instruction for action for the first asset to avoid collision with a second asset based on the Threat Rating Value.

19. The method of claim 18, wherein the first asset is a vehicle and the second asset is a vehicle or the first asset is a vehicle and the second asset is a human.

20. The method of claim 18, wherein the Threat Rating Value is calculated as:

TRV=(K_VH·

A_VH·

V_VH)+(K_VO·

A_VO·

V_VO)+(K_TS·

max[TRV_TS1 . . . TRV_TSn])

wherein;

TRV_TS1=C_TS1·

D_TS1·

V_TS1

TRV_TSn=C_TSn·

D_TSn·

V_TSn

K_VH=Weight constant for the host computer vision component input

A_VH=Amplitude level for the host computer vision component input

V_VH=Value of the host computer vision component input

K_VO=Weight constant for the object computer vision component input

A_VO=Amplitude level for the object computer vision component input

V_VO=Value of the object computer vision component input

V_TS1=Value of the first asset tracking component input

TRV_TS1=Threat rating value for the n^th asset tracking component input

C_TSn=Confidence level for the n^th asset tracking component input

D_TSn=Directional velocity component for the n^th asset tracking component input

V_TSn=Value of the n^th asset tracking component input

TRV_TSn=Threat rating value for the n^th asset tracking system input

TRV=Threat rating value for the Collision Avoidance Component.