Situational awareness system and method

• US 9,269,239 B1
• Filed: 09/22/2014
• Issued: 02/23/2016
• Est. Priority Date: 09/22/2014
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Original Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Inventors: Jensen, Dana J., Koenck, Steven E.
• Primary Examiner(s): Pham, Toan N
• Application Number:
  US14/492,235
• Time in Patent Office: 519 Days
• Field of Search: 340/691.6, 340/691.1, 340/691.2, 340/5.1, 340/8.1, 345 7- 9, 345418-427
• US Class Current: 1/1
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled
• CPC Class Codes: G01S 5/0009 Transmission of position information to remote stations involving assistance data G01S5/0236
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G08B 5/22 using electric transmission; using electromagnetic transmission
• CPC Class Codes: H04W 4/043 using ambient awareness, e.g. involving buildings using floor or room numbers
• CPC Class Codes: H04W 84/18 Self-organising networks, e.g. ad-hoc networks or sensor networks

Abstract

A situational awareness system includes a network radio module configured to be coupled with an ad-hoc network. A portable situational awareness unit includes a sensor associated with a user for sensing data indicative of a location of a current field of view of the user in an external scene and to output a signal indicative of the location of the current field of view of the user. A receiver is configured to receive a signal indicative of a location of an object in the external scene and/or identifying information for the object. A display is configured to display a visual indicator in the current field of view of the user. A processing module is configured to generate a visual indicator indicative of identifying information for the object and a location for the object, and to provide the visual indicator to the display for display to the user.

Claims

1. A situational awareness system, comprising:

at least one network radio module configured to be coupled with an ad-hoc network;

at least one portable situational awareness unit, including;

at least one sensor configured to be associated with a user and to sense data indicative of a location of a current field of view of the user in an external scene and to output at least one first signal indicative of the location of the current field of view of the user in the external scene;

a receiver configured to receive at least one second signal from a remote data source via the at least one network radio module, the at least one second signal indicative of at least one of;

a location of at least one object in the external scene and identifying information for at least one object in the external scene;

a display configured to be associated with the user and to display at least one visual indicator in the current field of view of the user; and

a processing module coupled with the at least one sensor, the at least one network radio module, and the display, the processing module having at least one processor coupled with a non-transitory processor-readable medium storing processor-executable instructions for causing the at least one processor to;

receive the at least one first and second signals;

generate, at least partially based on the at least one first and second signals, at least one visual indicator indicative of identifying information for the at least one object located in the external scene and a location for the at least one object relative to the current field of view of the user; and

provide the at least one visual indicator to the display for display in the current field of view of the user.

2. The system of claim 1, wherein the display includes at least one transparent portion configured to be positioned in the current field of view of the user such that the current field of view of the user is directed through the at least one transparent portion of the display.

3. The system of claim 2, wherein the at least one object is visible in the current field of view of the user, and wherein the at least one visual indicator is displayed at a location associated with the at least one object visible in the current field of view of the user.

4. The system of claim 1, wherein the at least one object is obscured in the current field of view of the user by an obscuring object, and wherein the at least one visual indicator is formatted to be displayed in the current field of view of the user over the obscuring object so as to indicate at least one of:

a relative location of the at least one object behind the obscuring object, and a path of the at least one object behind the obscuring object.

5. The system of claim 1, wherein the at least one object is located outside the current field of view of the user, and wherein the at least one visual indicator is further indicative of at least one of a direction and a distance to move the current field of view of the user so as to bring the at least one object within the current field of view of the user.

6. The system of claim 5, wherein the at least one object is located above the current field of view of the user, and wherein the at least one visual indicator is further indicative of at least one of:

a ground projection of the at least one object and a path of the at least one object.

7. The system of claim 1, wherein the at least one sensor is further configured to capture at least one image of the external scene and to transmit at least one third signal indicative of the at least one image of the external scene, and wherein the non-transitory processor-readable medium further stores processor-executable code for causing the least one processor to:

receive the at least one third signal indicative of the at least one image of the external scene;

overlay the at least one visual indicator onto the at least one image of the external scene; and

provide the at least one image of the external scene and the at least one visual indicator to the display for display in the current field of view of the user.

8. The system of claim 7, wherein the non-transitory processor-readable medium further stores processor-executable code for causing the least one processor to transmit at least one fourth signal to the at least one network radio module, the at least one fourth signal indicative of the at least one image of the external scene captured by the at least one sensor.

9. The system of claim 1, wherein the identifying information for the at least one object includes at least one of:

type of the at least one object, friendly identification of the at least one object, enemy identification of the at least one object, distance to the at least one object, speed of the at least one object, past track of the at least one object, predicted future track of the at least one object, predicted time until the at least one object moves into the current field of view of the user, predicted location where the at least one object enters the current field of view of the user, predicted time until the at least one object moves outside the current field of view of the user, predicted time until the at least one object enters the current field of view of the user, RF frequency emitted by the at least one object, and RF power emitted by the at least one object.

10. The system of claim 1, wherein the at least one object is configured to capture images of the external scene and wherein the non-transitory processor-readable medium further stores processor-executable instructions for causing the at least one processor to:

receive at least one third signal indicative of at least one image of the external scene captured by the at least one object;

overlay the at least one visual indicator onto the at least one image of the external scene; and

provide the at least one image of the external scene captured by the at least one object and the at least one visual indicator to the display for display in the current field of view of the user.

11. The system of claim 10, wherein the current field of view of the user has a first vantage point, and wherein the at least one image of the external scene captured by the at least one object has a second vantage point different from the first vantage point.

12. The system of claim 10, wherein the at least one third signal is received from at least one of:

the at least one object and the at least one network radio module.

13. The system of claim 1, wherein the at least one portable situational awareness unit is at least one first portable situational awareness unit, further comprising at least one second situational awareness unit coupled with the at least one network radio module, and wherein the at least one second situational awareness unit includes at least one sensor configured to capture at least one image of the external scene and to output at least one signal indicative of the at least one image of the external scene, and a transmitter configured to transmit the at least one signal to the network radio module, and wherein the non-transitory processor-readable medium of the at least one first portable situational awareness unit further stores processor-executable instructions for causing the at least one processor to:

receive at least one third signal indicative of the at least one image of the external scene captured by the at least one second portable situational awareness unit;

overlay at least one visual indicator onto the at least one image of the external scene captures by the at least one second portable situational awareness unit; and

provide the visual indicator and the at least one image of the external scene captures by the at least one second portable situational awareness unit to the display for display in the current field of view of the user.

14. The system of claim 13, wherein the at least one first portable situational awareness unit has a current field of view of the user from a first vantage point, and wherein the at least one second portable situational awareness unit has a current field of view of the user from a second vantage point different from the first vantage point.

15. A method of providing situational awareness to a user, comprising:

receiving, by at least one processor via an input port, a first signal from at least one sensor associated with a user, the first signal indicative of a location of a current field of view of the user in an external scene;

accessing, by the at least one processor, data indicative of at least one of a location and identifying information for at least one object in the external scene;

generating, by the at least one processor, at least one visual indicator indicative of the location of the at least one object relative to the current field of view of the user and the identifying information for the at least one object; and

providing, by the at least one processor, the at least one visual indicator to a display associated with the user for display in the current field of view of the user.

16. The method of claim 15, further comprising:

receiving, by the at least one processor, at least one second signal indicative of at least one image of the external scene captured by the at least one sensor associated with the user;

overlaying, by the at least one processor, the at least one visual indicator onto the at least one image of the external scene captured by the at least one sensor associated with the user; and

providing, by the at least one processor, the at least one image of the external scene captured by the at least one sensor associated with the user and the at least one visual indicator to the display for display in the current field of view of the user.

17. The method of claim 16, further comprising:

receiving, by the at least one processor, at least one third signal indicative of at least one image of the external scene captured by the at least one object;

overlaying, by the at least one processor, the at least one visual indicator onto the at least one image of the external scene captured by the at least one object; and

providing, by the at least one processor, the at least one image of the external scene captured by the at least one object and the at least one visual indicator to the display for display in the current field of view of the user.