Fast computation of kernel descriptors

• US 9,330,332 B2
• Filed: 10/04/2013
• Issued: 05/03/2016
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Raytheon BBN Technologies Corp. (Rtx Corporation)
• Original Assignee: Raytheon BBN Technologies Corp. (Rtx Corporation)
• Inventors: Natarajan, Pradeep, Wu, Shuang, Prasad, Rohit, Natarajan, Premkumar
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/046,194
• Publication Number:
  US 20140099033A1
• Time in Patent Office: 942 Days
• Field of Search: 382/154, 382/190, 382/205, 382/282, 382/305, 382/107, 382/209, 358/419, 358/664
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis

Abstract

An approach to computation of kernel descriptors is accelerated using precomputed tables. In one aspect, a fast algorithm for kernel descriptor computation that takes O(1) operations per pixel in each patch, based on pre-computed kernel values. This speeds up the kernel descriptor features under consideration, to levels that are comparable with D-SIFT and color SIFT, and two orders of magnitude faster than STIP and HoG3D. In some examples, kernel descriptors are applied to extract gradient, flow and texture based features for video analysis. In tests of the approach on a large database of internet videos used in the TRECVID MED 2011 evaluations, the flow based kernel descriptors are up to two orders of magnitude faster than STIP and HoG3D, and also produce significant performance improvements. Further, using features from multiple color planes produces small but consistent gains.

Claims

1. A method for processing images in a video processing system comprising:

accepting a video signal having a series of images acquired by a camera for processing and identifying a plurality of patches within said images;

reading a plurality of stored kernel tables, each kernel table representing a mapping from a corresponding feature to a vector of values;

computing a feature vector F(P) for each patch P of the plurality of patches, including computing one or more summations over locations z in the patch P of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch P at the location z; and

processing the images according to the computed feature vectors for the plurality of patches to provide a video processor output.

2. The method of claim 1 wherein F(P) approximates kernel descriptor based on a plurality of kernels.

3. The method of claim 2 wherein F(P) equals or approximates a sum of a form

4. The method of claim 1 wherein processing the images comprises computing similarities between patches as combinations of the computed feature vectors for the patches.

5. The method of claim 1 wherein the images are frames of a video, and wherein at least some of the patches are formed from multiple frames.

6. A method for image processing in a video processing system comprising:

accepting an input video having a series of images acquired by a camera for processing, and identifying patches within said images;

reading a plurality of stored kernel tables, each kernel table representing a mapping from a corresponding feature to a vector of values;

repeatedly computing similarities between pairs of patches for images being processed, computation of a similarity between a patch P and a patch Q comprises

computing for patch P one or more summations over locations z in the patch P of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch P at the location z,

computing for patch Q one or more summations over locations z in the patch Q of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch Q at the location z, and

combining the sums of the one or more summations for P and one or more summations for Q to determine a kernel descriptor similarity between P and Q; and

providing a video processor output comprising a result of processing the images using the computed similarities between the patches.

7. The method of claim 6 further comprising precomputing the kernel tables prior to accepting the images for processing.

8. A video processing system comprising:

a kernel preprocessor to provide a plurality of stored kernel tables, each kernel table representing a mapping from a corresponding feature to a vector of values;

an input to accept an input video having a series of images acquired by a camera for processing, and identifying a plurality of patches within said images;

a similarity computation module to compute a feature vector F(P) for each patch P of the plurality of patches, including computing one or more summations over locations z in the patch P of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch P at the location z; and

process the images according to the computed feature vectors for the plurality of patches; and

an output to provide a video processor output resulting from processing the images.

9. Software stored on a non-transitory computer-readable medium comprising instructions for causing a processor to:

accept a video input having a series of images acquired by a camera for processing, and identifying patches within said images;

read a plurality of stored kernel tables, each kernel table representing a mapping from a corresponding feature to a vector of values;

repeatedly compute similarities between pairs of patches for images being processed, computation of a similarity between a patch P and a patch Q comprises

computing for patch P one or more summations over locations z in the patch P of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch P at the location z,

computing for patch Q one or more summations over locations z in the patch Q of terms, each term being a product of terms including a term obtained by a lookup in a corresponding kernel table according to the location z and/or an attribute of the patch Q at the location z, and

combining the sums of the one or more summations for P and one or more summations for Q to determine a kernel descriptor similarity between P and Q; and

provide a video processor output comprising a result of processing the images using the computed similarities between the patches.