METHOD AND APPARATUS FOR IMAGE PROCESSING

• US 20130216144A1
• Filed: 02/22/2012
• Published: 08/22/2013
• Est. Priority Date: 02/22/2012
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: ROBINSON, Ian S., BLOOMER, John D., FLANDERS, Bradley A.
• Granted Patent: US 9,230,333 B2
• Time in Patent Office: Days
• US Class Current: 382/233
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

In accordance with various aspects of the disclosure, a system, a method, and computer readable medium having instructions for processing images is disclosed. For example, the method includes receiving, at an image processor, a set of images corresponding to a scene changing with time, decomposing, at the image processor, the set of images to detect static objects, leaner objects, and mover objects in the scene, the mover objects being objects that change spatial orientation in the scene with time, and compressing, using the image processor, the mover objects in the scene separately at a rate different from that of the static objects and the leaner objects for storage and/or transmission.

Claims

1. A method for processing images, comprising:

receiving, at an image processor, a set of images corresponding to a scene changing with time;

decomposing, at the image processor, the set of images to detect static objects, leaner objects, and mover objects in the scene, the mover objects being objects that change spatial orientation in the scene with time; and

compressing, using the image processor, the mover objects in the scene separately at a rate different from that of the static objects and the leaner objects for storage and/or transmission.

2. The method of claim 1, wherein the decomposing comprises:

creating difference frames from two or more successive images in the set of images;

applying thresholding to the difference frames;

applying filtering to the thresholded difference frames based upon at least one of a shape and a size of the static objects, the leaner objects, and the mover objects for the decomposing; and

creating, at the image processor, a first video stream comprising only pixels identified as the mover objects and a second video stream comprising the mover objects and pixels not identified as the mover objects in the scene.

3. The method of claim 2, wherein the compressing is performed separately for the first video stream and the second video stream at respective different rates of compression.

4. The method of claim 2, wherein the creating the difference frames comprises creating positive and negative difference frames and the thresholding followed by the filtering is applied to each of the positive and negative difference frames.

5. The method of claim 2, wherein the decomposing comprises:

creating a reference frame to remove the mover objects from the scene, such that in the reference frame, the mover objects are replaced by background image from frames adjacent to the two or more successive images, the reference frame being utilized for reconstruction of the scene.

6. The method of claim 5, wherein the reference frame is a single reference frame transmitted for processing separate from the mover objects, and wherein the reference frame is compressed using lossless compression and the mover objects are compressed using a lossy technique.

7. The method of claim 2, wherein the applying the filtering comprises:

specifying an object box of pixels and a background box of pixels around each object of interest in the scene;

convolving positive and negative difference frames in the created difference frames separately for the object box and the background box; and

determining, based upon the convolving, whether or not the object of interest in the object box of pixels can be detected as a mover object.

8. The method of claim 1, wherein the scene is observed along a known direction of motion of an imaging platform for the receiving.

9. The method of claim 1, wherein the mover objects are compressed using track files that store position vectors associated with the mover objects.

10. The method of claim 1, wherein the leaner objects are annotated for respective heights.

11. The method of claim 1 further comprising:

decompressing, using the image processor, data associated with the mover objects, the static objects, and the leaner objects; and

reconstructing, using the image processor, the set of images corresponding to the scene based upon the decompressing.

12. An image processing system, comprising:

an imaging platform having a sensor that is configured to capture images of a scene, each image comprising a plurality of pixels; and

an image processor coupled to the imaging platform and to one or more memory devices having instructions thereupon, which when executed by the image processor cause the image processor to;

receive a set of images corresponding to a scene changing with time;

decompose the set of images to detect static objects, leaner objects, and mover objects in the scene, the mover objects being objects that change spatial orientation in the scene with time; and

compress the mover objects in the scene separately at a rate different from that of the static objects and the leaner objects for storage and/or transmission.

13. The system of claim 12, wherein the image processor is configured to decompose the set of images by:

creating difference frames from two or more successive images in the set of images;

applying thresholding to the difference frames;

applying filtering to the thresholded difference frames based upon at least one of a shape and a size of the static objects, the leaner objects, and the mover objects for the decomposing; and

creating a first video stream comprising only pixels identified as the mover objects and a second video stream comprising the mover objects and pixels not identified as the mover objects in the scene.

14. The system of claim 13, wherein the image processor is further configured to:

compress the first video stream separately from the second video stream and at different respective rates of compression.

15. The system of claim 13, wherein the image processor is configured to create the difference frame by creating positive and negative difference frames and applying the thresholding and filtering to each of the positive and negative difference frames.

16. The system of claim 13, wherein the image processor is configured to decompose the set of images by creating a reference frame to remove the mover objects from the scene, such that in the reference frame, the mover objects are replaced by background image from frames adjacent to the two or more successive images, the reference frame being utilized for reconstruction of the scene.

17. The system of claim 16, wherein the reference frame is a single reference frame transmitted for processing separate from the mover objects, and wherein the reference frame is compressed using lossless compression and the mover objects are compressed using a lossy technique.

18. The system of claim 13, wherein the image processor is configured to apply the filtering by:

specifying an object box of pixels and a background box of pixels around each object of interest in the scene;

convolving positive and negative difference frames in the created difference frames separately for the object box and the background box; and

determining, based upon the convolving, whether or not the object of interest in the object box of pixels can be detected as a mover object.

19. The system of claim 12, wherein the scene is observed along a known direction of motion of the imaging platform for the image processor to receive the set of images.

20. The system of claim 12, wherein the mover objects are compressed using track files that store position vectors associated with the mover objects, which position vectors are used by the image processor to reconstruct the scene.

21. The system of claim 12, wherein the leaner objects are annotated for respective heights.

22. The system of claim 12, wherein the image processor is further configured to:

decompress data associated with the mover objects, the static objects, and the leaner objects; and

reconstruct the set of images corresponding to the scene using the decompressed data.

23. A tangible non-transitory computer-readable storage medium having computer-readable instructions stored therein for processing images, which when executed by one or more processors cause the one or more processors to:

receive a set of images corresponding to a scene changing with time;

decompose the set of images to detect static objects, leaner objects, and mover objects in the scene, the mover objects being objects that change spatial orientation in the scene with time; and

compress the mover objects in the scene separately at a rate different from that of the static objects and the leaner objects for storage and/or transmission.