Apparatus and method employing pre-ATR-based real-time compression and video frame segmentation

• US 20080310742A1
• Filed: 06/15/2007
• Published: 12/18/2008
• Est. Priority Date: 06/15/2007
• Status: Active Grant

• Current Assignee: Physical Optics Corporation
• Original Assignee: Physical Optics Corporation
• Inventors: Jannson, Tomasz, Kostrzewski, Andrew, Wang, Wenjian
• Granted Patent: US 8,798,148 B2
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

One subject of this invention is the development of a novel region of interest (ROI) method, or Frame Segmentation Method that can be provided within a video stream, in real-time, or more precisely within a few milliseconds of video frame duration of 30 msec, or even in the sub-millisecond range. This video frame segmentation is the basis of Pre-ATR-based Ultra-Real-Time (PATURT) video compression. Still other subjects of this invention are morphing compression, and watermarking, also based on the PATURT. The applications of the PATURT innovation include ROI-based real-time video recording that has special applications for aircraft pilot/cockpit video recording in “black-box” devices, recording aircraft accidents, or catastrophes. Such black-box devices usually need to pass high impact (3400 g), high temperature (1100° C., in 1 h), and other harsh environmental tests. In this invention, they also have the capability of reporting the last cockpit events up to 0.5 seconds before an accident, including all cockpit sensor readings, as well as pilots'"'"' behavior, the latter with fully scrambled and non-recoverable facial information. Further applications include video surveillance. The latter can be also applied to missile defense (recognizing real target or real missile, from false targets (decoys)), or to other Ultra-Real-Time (URT) civilian and military scenarios.

Claims

1. A method of intraframe real time, video image processing for reducing transmission bandwidth or computing time;

the method comprising the following steps;

a) selecting principal and secondary image frame signatures;

b) selecting and extracting contours of regions of interest;

c) applying multifacet inhomogeneous compression for the regions of interest of step b) and for the remaining portions of the video frame; and

d) performing steps a), b) and c) all within the duration of the frame.

2. The method recited in claim 1 wherein step a) is carried out using software agents analogous to sensors for rejecting false targets in a pool of true and false targets.

3. The method recited in claim 1 wherein step a) is carried out using the step of pixel-wise intensity subtracting.

4. The method recited in claim 1 wherein step a) is carried out using the step of speed vector flow mapping.

5. The method recited in claim 1 wherein step a) is carried out using the step of color matching operation.

6. The method recited in claim 1 wherein step b) is carried out using the step of edge extraction based on phase space scaling.

7. The method recited in claim 6 wherein said edge extraction step is carried out using image line decimation and smoothing.

8. The method recited in claim 1 wherein step b) is carried out by determining the center of gravity of each region of interest using a polar contour compliance procedure.

9. The method recited in claim 1 wherein step c) is carried out using different compression ratios for regions of interest and for said remaining portions.

10. The method recited in claim 1 wherein step c) is carried out using different compression schemes for regions of interest and for said remaining portions.

11. The method recited in claim 10 wherein at least one of said different compression schemes comprises one of an MPEG standard form of compression and wavelet transform compression.

12. The method recited in claim 1 wherein in step d) the time for performing steps a), b) and c) is less than 30 milliseconds.

13. The method recited in claim 1 wherein in step d) the time for performing steps a), b) and c) is less than 1 millisecond.

14. A method of nearly real time, video image processing for reducing transmission bandwidth or computing time;

the method comprising the following steps;

a) selecting principal and secondary image frame signatures;

b) selecting and extracting contours of regions of interest; and

c) applying predictive morphing compression to the regions of interest of step b).

15. A method of nearly real time, video image processing for reducing transmission bandwidth or computing time;

the method comprising the following steps;

a) selecting principal and secondary image frame signatures;

b) selecting and extracting contours of regions of interest; and

c) applying nearly real time target acquisition expansion.

16. A method of watermarking a video image frame;

the method comprising the steps of;

a) selecting principal and secondary image frame signatures;

b) selecting and extracting contours of regions of interest;

c) selecting video data bits of least significance in a portion of the frame not in any region of interest; and

d) altering the bits selected in step c) to provide some hidden information distinct from said image frame.

17. An ATR method of representing an original full-resolution video image as a phase-space Gaussian Mixture to provide a reduced pixel count version of the image;

the method comprising the steps of;

a) generating a multi-resolution pyramid of the original image using a wavelet transform; and

b) modeling wavelet coefficients of said wavelet transform as a Gaussian Scale Mixture where wavelet coefficient magnitudes are arranged in phase-scale Gaussian vector form.

18. A flight recorder for recording at least one of video, audio and sensor data information generated in an aircraft up to 0.5 seconds before a catastrophic crash during a flight;

the recorder comprising;

at least one solid state memory device for storing said information, said device being enclosed in a container, said container having a material surrounding said device for cushioning said device against an impact force generated during said crash of said aircraft.

19. The recorder recited in claim 18 wherein said device is cushioned to withstand an impact force having a magnitude of at least 3,400 G'"'"'s.

20. The recorder recited in claim 18 wherein said container comprises titanium.

21. The recorder recited in claim 18 further comprising a removable solid state memory device for storing said information, said removable memory device being located outside said container for being accessible without opening said container.

22. The recorder recited in claim 18 further comprising a connector for integrating said recorder into said aircraft, said connector being connected in said recorder to a dedicated removable printed circuit board whereby replacement of said connector is effected by replacement of said removable printed circuit board.

23. The recorder recited in claim 18 wherein said memory device has the capacity to record at least two hours of video, audio and sensor data information.

24. The recorder recited in claim 18 wherein said memory device is configured for recording up to four channels of audio information and up to two channels of video information.

25. The recorder recited in claim 24 wherein said memory device is further configured for recording up to eight channels of discrete data and up to sixteen channels of analog data.

26. A solid state crash survivable recorder comprising a memory device configured for recording at least two hours of up to two channels of video information, up to four channels of audio information, up to eight channels of discrete sensor data and up to sixteen channels of analog sensor data generated up to 0.5 seconds prior to a crash.

27. The recorder recited in claim 26 further comprising a capsule housing said memory device, said capsule being filled with a shock absorbing material surrounding said memory device.

28. The recorder recited in claim 27 wherein said shock absorbing material comprises Aerogel.

29. The recorder recited in claim 27 wherein said capsule is made of a material comprising titanium.

30. The recorder recited in claim 26 wherein said memory device comprises a removable memory device.

31. The recorder recited in claim 26 further comprising a connector for integrating said recorder into a system having video cameras, microphones and sensors, said connector being connected in said recorder to a dedicated removable printed circuit board whereby replacement of said connector is effected by replacement of said removable printed circuit board.

32. A crash survivable memory comprising:

a solid state memory device;

a crash-resistant capsule for enclosing said memory device; and

a shock absorbing material within the capsule and surrounding said memory device.

33. The memory recited in claim 32 wherein said shock absorbing material comprises Aerogel.

34. The memory recited in claim 32 wherein said capsule is made of a material comprising titanium.

35. A crash survivable video recorder for aircraft cockpit event recording up to no more than 0.5 seconds prior to some catastrophic event;

the recorder comprising;

at least one video camera; and

image processing electronics for analyzing each frame of video data generated in said camera and determining various regions of interest in said frame for selectively compressing the data in each said region of interest by a compression ratio dependent upon the information content in said each region of interest.

36. The video recorder recited in claim 35 wherein any said region of interest having a pilot'"'"'s facial features is automatically compressed by a compression ratio sufficiently high to obscure the pilot'"'"'s face.

37. A method of intraframe real time, video image processing for reducing bandwidth and computation time;

the method comprising the following steps;

a) selecting image frame signatures;

b) filtering and decimation;

c) pixel-wise subtraction; and

d) edge smoothing;

wherein steps a), b), c) and d) are all performed within the duration of a single frame of video.

38. The method recited in claim 37 wherein step b) is carried out using the steps of filtering, decimation and pixel subtraction.

39. The method recited in claim 38 wherein the steps of filtering and decimation are carried out using a single step of filtering and decimation employing phase space filter banks.

40. The method recited in claim 38 wherein the steps of filtering and decimation are carried out using a single step of filtering and decimation by a phase space scaling operation.