VIDEO ANALYTICS SYSTEM, COMPUTER PROGRAM PRODUCT, AND ASSOCIATED METHODOLOGY FOR EFFICIENTLY USING SIMD OPERATIONS

• US 20100080483A1
• Filed: 11/21/2008
• Published: 04/01/2010
• Est. Priority Date: 09/26/2008
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Almbladh, Johan
• Granted Patent: US 8,260,002 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining

Abstract

A video analytics system and associated methodology for performing low-level video analytics processing divides the processing into three phases in order to efficiently use SIMD instructions of many modern data processors. In the first phase, pixels of interest are gathered using a predetermined mask and placed into a pixel matrix. In the second phase, video analytics processing is performed on the pixel matrix, and in the third phase the pixels are scattered using the same predetermined mask. This allows many pixels to be processed simultaneously, increasing overall performance. A DMA unit may also be used to offload the processor during the gathering and scattering of pixels, further increasing performance. A network camera integrates the video analytics system to reduce network traffic.

Claims

1. A video analytics processing method comprising:

gathering, in a computer-readable storage medium, a plurality of pixels of interest from an image according to a predetermined binary mask, the predetermined binary mask defining at least one location of the plurality of pixels of interest on the image;

arranging the plurality of pixels of interest in a pixel matrix including at least the pixels of interest, each pixel of interest being stored in the pixel matrix adjacent to other pixels of interest;

performing video analytics processing on the pixel matrix, using SIMD instructions on a data processor; and

scattering the pixels of interest into the image after processing according to the predetermined binary mask, wherein

the gathering step and the scattering step are performed independently of the performing video analytics processing step.

2. The video analytics method processing according to claim 1, wherein the gathering step includes for each pixel of interest,

dilating the predetermined binary mask by at least one pixel position in a horizontal direction,

gathering at least one neighboring pixel from a pixel row above the pixel of interest in accordance with the dilated binary mask,

gathering, in accordance with the dilated binary mask, at least one neighboring pixel and the pixel of interest from a pixel row of the pixel of interest, and

gathering at least one neighboring pixel from a pixel row below the pixels of interest in accordance with the dilated binary mask.

3. The video analytics method processing according to claim 2, wherein the gathering step and the scattering step further include using a DMA unit to respectively gather and scatter the plurality of pixels of interest and the neighboring pixels independently of the data processor.

4. The video analytics processing method according to claim 1, wherein performing video analytics processing includes filtering the pixels of interest with a N×

N filter kernel using a SIMD instruction set, the SIMD instruction set including an averaging instruction, a halved subtraction instruction with biased result and a splice instruction.

5. The video analytics processing method according to claim 4, wherein a predetermined number of bits representing each pixel is held constant throughout the filtering operation.

6. A computer-readable medium storing computer-readable instructions thereon, the instructions when executed by a processor cause the processor to perform the video analytics processing method comprising:

gathering, in a computer-readable storage medium, a plurality of pixels of interest from an image according to a predetermined binary mask, the predetermined binary mask defining at least one location of the plurality of pixels of interest on the image;

arranging the plurality of pixels of interest in a pixel matrix including at least the pixels of interest;

performing video analytics processing on the pixel matrix using a SIMD processor; and

scattering the pixels into the image after processing according to the predetermined binary mask, wherein

the gathering step and the scattering step are performed independently of the performing video analytics processing step.

7. The computer-readable medium according to claim 6, wherein the gathering step includes for each pixel of interest,

dilating the predetermined binary mask by at least one pixel position in a horizontal direction,

gathering at least one neighboring pixel from a pixel row above the pixel of interest in accordance with the dilated binary mask,

gathering, in accordance with the dilated binary mask, at least one neighboring pixel and the pixel of interest from a pixel row of the pixel of interest, and

gathering at least one neighboring pixel from a pixel row below the pixels of interest in accordance with the dilated binary mask.

8. The computer-readable medium according to claim 7, wherein the gathering step and the scattering step further include using a DMA unit to respectively gather and scatter the plurality of pixels of interest and the neighboring pixels independently of the data processor.

9. The computer-readable medium according to claim 6, wherein performing video analytics processing includes filtering the pixels of interest with a N×

N filter kernel using a SIMD instruction set, the SIMD instruction set including an averaging instruction, a halved subtraction instruction with biased result and a splice instruction.

10. The computer-readable medium according to claim 9, wherein a predetermined number of bits representing each pixel is held constant throughout the filtering operation.

11. A network camera comprising:

at least one lens configured to collect incident light;

a sensor arranged behind the lens and configured to convert the incident light into image data;

an electronic memory configured to store, as image data, the electrical data corresponding to the image;

a video encoder configured to encode the image data;

a data processor including

a gathering unit configured to gather, in the electronic memory, a plurality of pixels of interest from an image according to a predetermined binary mask, the predetermined binary mask defining at least one location of the plurality of pixels of interest,

an arranging unit configured to arrange the plurality of pixels of interest in a pixel matrix including at least the pixels of interest, a video analytics unit configured to perform video analytics processing on the pixel matrix independently from the gathering unit,

a scattering unit configured to scatter the pixels of interest into the image, independently of the gathering unit and the video analytics unit; and

a network interface configured to connect to a data network.

12. The network camera according to claim 11, wherein the gathering unit is further configured to,

dilate the predetermined binary mask by at least one pixel position in a horizontal direction,

gather at least one neighboring pixel from a pixel row above the pixel of interest in accordance with the dilated binary mask,

gather, in accordance with the dilated binary mask, at least one neighboring pixel and the pixel of interest from a pixel row of the pixel of interest, and

gather at least one neighboring pixel from a pixel row below the pixels of interest in accordance with the dilated binary mask.

13. The network camera according to claim 11, further comprising:

a DMA unit configured to transport pixels independently of the data processor, the pixel gathering unit and the pixel scattering unit employing the DMA unit when respectively gathering and scattering pixels of interest.

14. The network camera according to claim 11, wherein the data processor is further configured to perform filtering of the pixels of interest with a N×

N filter kernel using SIMD instructions, the SIMD instructions including an averaging instruction, a halved subtraction instruction with biased result, and a splice instruction.

15. The network camera according to claim 14, wherein a predetermined number of bits representing each pixel is held constant throughout the filtering operation.