Method and system for providing scene data in a video stream

• US 10,070,075 B2
• Filed: 03/17/2017
• Issued: 09/04/2018
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: DRS Network & Imaging Systems, LLC (Leonardo S.p.A.)
• Original Assignee: DRS Network & Imaging Systems, LLC (Leonardo S.p.A.)
• Inventors: Hutchison, David C., Williamson, Ralph K.
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US15/462,432
• Publication Number:
  US 20170195583A1
• Time in Patent Office: 536 Days
• Field of Search: None
• CPC Class Codes: G01J 2005/0077 Imaging
• CPC Class Codes: G01J 5/025 Interfacing a pyrometer to an external device or network; User interface
• CPC Class Codes: G01J 5/0806 Focusing or collimating elements, e.g. lenses or concave mirrors
• CPC Class Codes: G01J 5/10 using electric radiation detectors
• CPC Class Codes: G01J 5/48 Thermography; Techniques using wholly visual means
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/23605 Creation or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Methods of and systems for providing temperature data in a video stream are provided. The method includes receiving a video stream having a plurality of video frames with a first frame rate and receiving temperature data including a temperature map associated with the video stream and having a plurality of temperature frames with a second frame rate, which can be slower than the first frame rate. To interlace the temperature data, a subset of temperature frames in the plurality of temperature frames can be extracted. The method further includes transmitting each temperature frame in the subset of temperature frames with the plurality of video frames in a data stream.

Claims

1. A method of providing a data stream, the method comprising:

receiving a video stream having a plurality of video frames characterized by a first frame rate;

receiving a thermal data stream having a plurality of thermal data frames with a second frame rate, each thermal data frame including a plurality of data lines and a temperature map associated with one or more of the plurality of video frames;

generating a plurality of reduced data frames, wherein each of the plurality of reduced data frames is generated by;

(a) extracting a subset of data lines from the plurality of data lines, wherein the subset of data lines is 1/N of the data lines;

(b) forming a portion of a reduced data frame using the subset of data lines; and

(c) iterating (a) and (b) N times; and

transmitting the data stream including the plurality of video frames at the first frame rate and the plurality of reduced data frames at the second frame rate.

2. The method of claim 1 wherein the first frame rate and the second frame rate are a same frame rate.

3. The method of claim 1 wherein the first frame rate ranges from 9 Hz to 30 Hz and the second frame rate ranges from 1 Hz to 120 Hz.

4. The method of claim 1, further comprising:

receiving the data stream;

separating the plurality of video frames from the plurality of reduced data frames;

for each reduced data frame in the plurality of reduced data frames;

identifying one or more missing data lines in the subset of data lines,

correlating the plurality of reduced data frames with the plurality of video frames,

providing the one or more missing data lines to the subset of data lines of the reduced data frame based on the correlation of the plurality of reduced data frames with the plurality of video frames, and

creating a reconstructed data frame including the one or more missing data lines and the subset of data lines of the reduced data frame; and

generating a plurality of reconstructed data frames using the reconstructed data frame created for each reduced data frame; and

transmitting the plurality of reconstructed data frames.

5. The method of claim 4, further comprising transmitting the plurality of reconstructed data frames with the plurality of video frames to an external display.

6. The method of claim 4 wherein the second frame rate is lower than the first frame rate.

7. The method of claim 4 wherein the data stream includes a plurality of packets, each packet including one reduced data frame in the plurality of reduced data frames and a subset of video frames in the plurality of video frames.

8. The method of claim 7 wherein the reduced data frame is reduced by a fraction inversely proportional to a number of video frames in the subset of video frames.

9. A system for providing a data stream, the system comprising:

a detector configured to receive a video stream having a plurality of video frames with a first frame rate and receive a thermal data stream including a temperature map associated with the video stream and having a plurality of thermal data frames with a second frame rate, each thermal data frame including a plurality of thermal data lines;

a frame processing module configured to, for each thermal data frame in the plurality of thermal data frames;

extract a subset of thermal data lines in the plurality of thermal data lines, thereby interlacing the thermal data stream,

create a reduced thermal data frame including the subset of thermal data lines, and

generate a plurality of reduced thermal data frames using the reduced thermal data frame created for each thermal data frame; and

a first transmitter configured to transmit the plurality of reduced thermal data frames at the second frame rate with the plurality of video frames at the first frame rate in the data stream.

10. The system of claim 9, further comprising:

a receiver configured to receive the data stream and separate the plurality of video frames from the plurality of reduced thermal data frames;

a frame reconstructing module configured to, for each reduced thermal data frame in the plurality of reduced thermal data frames;

identify one or more missing data lines in the subset of thermal data lines,

correlate the plurality of reduced thermal data frames with the plurality of video frames,

provide the one or more missing data lines to the subset of thermal data lines of the reduced thermal data frame based on the correlation of the plurality of reduced thermal data frames with the plurality of video frames,

create a reconstructed thermal data frame including the one or more missing data lines and the subset of thermal data lines of the reduced thermal data frame, and

generate a plurality of reconstructed thermal data frames using the reconstructed thermal data frame created for each reduced thermal data frame; and

a second transmitter configured to transmit the plurality of reconstructed thermal data frames.

11. The system of claim 10 wherein the second transmitter transmits the plurality of reconstructed thermal data frames with the plurality of video frames to an external display.

12. The system of claim 10 wherein the second frame rate is lower than the first frame rate.

13. The system of claim 10 wherein the data stream includes a plurality of packets, each packet including one reduced thermal data frame in the plurality of reduced thermal data frames and a subset of video frames in the plurality of video frames.

14. The system of claim 13 wherein the reduced thermal data frame is reduced by a fraction inversely proportional to a number of video frames in the subset of video frames in each packet.