Near real-time dissemination of surveillance video

• US 7,460,148 B1
• Filed: 02/18/2004
• Issued: 12/02/2008
• Est. Priority Date: 02/19/2003
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rockwell Collins, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Castello, Vincent, Clark, J. Edwin, Horton, Glenn J., Wells, Daniel W.
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US10/779,814
• Time in Patent Office: 1,749 Days
• Field of Search: 348/142, 348/143, 348/333.02, 348/169, 348/170, 340/539.13, 340/541, 725/105
• US Class Current: 348/143
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The disclosed Near Real-Time Video (NRTV) system is a complete video processing, smart dissemination and user display capability for dissemination of video data from surveillance systems, such as manned and unmanned aerial surveillance platforms. The system requires significantly less bandwidth for dissemination than traditional, full analog video distribution. The exemplary system ingests an analog video source signal from a surveillance platform, digitizes the video, and allows a local or remote user to visualize frames of the video field of view on a standard map display. From an overlay of the field(s) of view, the system allows the user to request selected images or a video clip from the actual video surveillance information. Using available communications, the request is relayed to the video processing subsystem, and a video file of the requested information is generated and sent back to the user for display, all in near real-time.

Claims

1. A method of obtaining and presenting video surveillance information, comprising:

over a period of time, receiving position and orientation telemetry regarding operation of a camera mounted on an airborne moveable surveillance platform;

processing the telemetry over the period of time to determine a plurality of earth-oriented fields of view of the camera during movement of the airborne surveillance platform over the period of time;

displaying over time, representations of the determined earth-oriented fields of view of the camera on a map display on a user interface;

detecting a user selection of a displayed representation of at least one field of view via the user interface, the selected representation corresponding to a portion of the period of time;

formulating an information request indicating the corresponding portion of the period of time;

sending the information request via a network to a facility having access to stored video information received from the camera, over the period of time;

receiving one or more frames of video information generated by the camera, from the facility via the network, the one or more frames containing actual image data taken by the camera during the corresponding portion of the period of time and corresponding to the user selected representation of one or more fields of view; and

displaying the received one or more frames of video information to the user.

2. The method as in claim 1, wherein the step of displaying the representations of the determined earth-oriented fields of view of the camera on the map display on the user interface comprises:

displaying a map of a geographic region under surveillance by the moveable surveillance platform; and

overlaying outlines of the determined earth-oriented fields of view of the camera, for a plurality of points in time, on the displayed map.

3. The method as in claim 2, wherein the step of detecting the user selection comprises receiving a user selection of at least one displayed outline of a determined view of the camera.

4. The method as in claim 3, wherein the step of detecting the user selection further comprises receiving a user selection of a time interval referenced to the point in time corresponding to the selected displayed outline of a determined view of the camera.

5. The method as in claim 3, wherein the step of receiving a user selection of at least one displayed outline comprises receiving user selection of a plurality of the displayed outlines of determined views of the camera.

6. The method as in claim 5, wherein the one or more frames of video information consist essentially of a plurality of selected still frame images.

7. The method as in claim 4, wherein the received and displayed video information comprises a video clip of a user selected duration.

8. The method as in claim 3, further comprising overlaying an indicator of the current location of the moveable surveillance platform on the displayed map, concurrent with the overlaying of the outlines of the determined views of the camera for the plurality of points in time.

9. A method of disseminating video surveillance information, comprising:

over a period of time, receiving and storing real-time video image information from a camera mounted on an airborne moveable surveillance platform;

with the video image information, receiving position and orientation telemetry regarding operation of the camera over the period of time;

transmitting notification messages to a user'"'"'s client device over limited network communication facilities, the notification messages including information regarding the telemetry sufficient to allow determination of earth-oriented fields of view of the camera during movement of the airborne surveillance platform during the period of time;

receiving an information request from the user'"'"'s client device, the information request identifying a user selected portion of the received real-time video image information; and

transmitting the selected portion of the video image information to the user'"'"'s client device over the limited network communications facilities.

10. The method as in claim 9, wherein the step of receiving and storing video image information comprises:

receiving an analog video signal from the camera;

digitizing the received analog video signal;

indexing the digitized video signal; and

storing the digitized and indexed video signal.

11. The method as in claim 9, wherein:

the notification messages are transmitted periodically, and

each notification message contains information relating to at least one of the earth-oriented fields of view of the camera.

12. The method as in claim 9, wherein:

the received information request identifies a point in the received video image information and a time interval in relation to the identified point; and

the step of transmitting video image information comprises extracting a clip of video information of the identified time interval in relation to the identified point, from the stored video image information, and transmitting the clip of video information to the user'"'"'s client device.

13. The method as in claim 9, wherein:

the received information request identifies a plurality of user selected points in the received video image information; and

the transmitted video image information comprises video frames corresponding to the user selected points.

14. A terminal device comprising:

a processor,

a user interface,

a communication interface,

a program storage device, and

a program in said storage device for execution by the processor,

wherein execution of the program by the processor causes the terminal to implement functions, comprising;

over a period of time, receiving position and orientation telemetry regarding operation of a camera mounted on an airborne moveable surveillance platform, through limited communication facilities;

processing the telemetry to determine earth-oriented fields of view of the camera over the period of time;

displaying over time, representations of the determined fields of view of the camera on a map display on the user interface;

detecting a user selection of a displayed representation of at least one field of view via the user interface, the selected representation corresponding to a portion of the period of time;

formulating an information request indicating the corresponding portion of the period of time;

sending the information request through the limited communication facilities to a facility having access to stored video information received from the camera, over the period of time;

receiving one or more frames of video information generated by the camera, from the facility through the limited communication facilities, the one or more frames containing actual image data taken by the camera during the corresponding portion of the period of time and corresponding to the user selected representation of at least one field of view; and

displaying the received one or more frames of video information to the user.

15. A computer comprising:

a programmable processor,

data storage,

a program storage device,

a program in the program storage device, and

a communication interface,

wherein execution of the program by the processor causes the computer to implement functions for disseminating video surveillance information, the functions comprising;

over a period of time, receiving and storing real-time video image information from a camera mounted on an airborne moveable surveillance platform, in the data storage;

with the video image information, continuously receiving position and orientation telemetry regarding operation of the camera over the period of time;

transmitting notification messages to a user'"'"'s client device, through limited network communication facilities via the communication interface, the notification messages including information regarding the telemetry sufficient to allow determination of earth-oriented fields of view of the camera during movement of the airborne surveillance platform during the period of time;

receiving an information request from the user'"'"'s client device via the communication interface, the information request identifying a user selected portion of the received real-time video image information; and

transmitting the selected portion of the video image information to the user'"'"'s client device through the limited network communication facilities, via the communication interface.

16. The computer as in claim 15, further comprising:

a video processor, for receiving an analog video signal from the camera, digitizing the received analog video signal and indexing the digitized video signal;

wherein the server stores the digitized and indexed video signal in the data storage.

17. The computer as in claim 15, wherein the program causes the processor to authenticate the user in response to the request.

18. A product comprising:

a computer readable storage medium, and

a program embodied in the storage medium,

the program being executable by a processor of a terminal device to cause the terminal device to implement functions, comprising;

over a period of time, receiving position and orientation telemetry regarding operation of a camera mounted on an airborne moveable surveillance platform, through limited communication facilities;

processing the telemetry to determine earth-oriented fields of view of the camera over the period of time;

displaying over time, representations of the determined fields of view of the camera on a map display on a user interface;

detecting a user selection of a displayed representation of at least one field of view via the user interface, the selected representation corresponding to a portion of the period of time;

formulating an information request indicating the corresponding portion of the period of time;

sending the information request through the limited communication facilities to a facility having access to stored video information received from the camera, over the period of time;

receiving one or more frames of video information generated by the camera, from the facility through the limited communication facilities, the one or more frames containing actual image data taken by the camera during the corresponding portion of the period of time and corresponding to the user selected representation of at least one field of view; and

displaying the received one or more frames of video information to the user.

19. The product as in claim 18, wherein the function of displaying the representations of the determined earth-oriented fields of view of the camera on the user interface comprises:

displaying a map of a geographic region under surveillance by the moveable surveillance platform; and

overlaying outlines of the determined earth-oriented fields of view of the camera, for a plurality of points in time, on the displayed map.

20. The product as in claim 19, wherein the function of detecting the user selection of the view via the user interface comprises receiving a user selection of at least one displayed outline of a determined view of the camera.

21. The product of claim 19, wherein the function of detecting the user selection of the view via the interface further comprises receiving a user selection of a time interval referenced to the point in time corresponding to the selected displayed outline of a determined view of the camera.

22. The product of claim 19, wherein the function of receiving a user selection of at least one displayed outline comprises receiving user selection of a plurality of the displayed outlines of determined views of the camera.

23. The product of claim 22, wherein the one or more frames of video information consist essentially of a plurality of selected still frame images corresponding to the user selected outlines.

24. The product of claim 22, wherein the one or more frames of video information comprise a video clip of a user selected duration, and the start and end of the video clip correspond to two of the user selected outlines.

25. The product of claim 20, the functions further comprise overlaying an indicator of the current location of the moveable surveillance platform on the displayed map, concurrent with the overlaying of the outlines of the determined view of the camera for the plurality of points in time.

26. A product comprising:

a computer readable storage medium, and

a program embodied in the computer readable storage medium,

the program being executable by a processor of a computer so as to cause the computer to implement functions for disseminating video surveillance information, the functions comprising;

over a period of time, receiving and storing real-time video image information from a camera mounted on an airborne moveable surveillance platform;

with the video image information, receiving position and orientation telemetry regarding operation of the camera over the period of time;

transmitting notification messages to a user'"'"'s client device through limited network communication facilities, the notification messages including information regarding the telemetry sufficient to allow determination of earth-oriented fields of view of the camera during movement of the airborne surveillance platform during the period of time;

receiving an information request from the user'"'"'s client device, the information request identifying a user selected portion of the received real-time video image information received over the period of time; and

transmitting the selected portion of the video image information to the user'"'"'s client device through the limited network communication facilities.

27. The product as in claim 26, wherein the function of receiving and storing video image information comprises:

receiving an analog video signal from the camera;

digitizing the received analog video signal;

indexing the digitized video signal; and

storing the digitized and indexed video signal.

28. The product as in claim 26, wherein:

the notification messages are transmitted periodically, and

each notification message contains information relating to at least one of the earth-oriented fields of view of the camera at a point in time.

29. The product as in claim 26, wherein:

the received information request identifies a point in the received video image information and a time interval in relation to the identified point; and

the function of transmitting video image information comprises extracting a clip of video information of the identified time interval in relation to the identified point, from the stored video image information, and transmitting the clip of video information to the user'"'"'s client device.

30. The product as in claim 26, wherein:

the received information request identifies a plurality of user selected points in the received video image information; and

the transmitted video image information comprises video frames corresponding to the user selected points.

31. The product as in claim 26 wherein:

the received information request identifies a user selected frame in the received video image information; and

the transmitted video information comprises the user selected frame.

32. The product of claim 31, wherein the transmitted information consists of the user selected frame.