Apparatus, computer program product and associated methodology for video analytics

• US 8,401,327 B2
• Filed: 11/21/2008
• Issued: 03/19/2013
• Est. Priority Date: 09/26/2008
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Almbladh, Johan
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US12/275,758
• Publication Number:
  US 20100082949A1
• Time in Patent Office: 1,579 Days
• Field of Search: 382/304, 382/307, 382/300, 382/260
• US Class Current: 382/260
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06F 9/30032 Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE
• CPC Class Codes: G06F 9/30036 Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
• CPC Class Codes: G06F 9/30109 having multiple operands in a single register
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining

Abstract

A processor and associated methodology employ a SIMD architecture and instruction set to efficiently perform video analytics operation on images. The processor contains a group of SIMD instructions used by the method to implement video analytic filters that avoid bit expansion of the pixels to be filtered. The filters hold the number of bits representing a pixel constant throughout the entire operation, conserving processor capacity and throughput when performing video analytics.

Claims

1. A processor for video analytics comprising:

a plurality of registers each partitioned into at least two fields, each field being configured to store pixel data corresponding to an individual pixel;

an instruction decoder configured to decode SIMD instructions from a predetermined SIMD instruction set, the predetermined SIMD instruction set including at least a splice instruction, an averaging instruction and a half subtraction instruction, the splice instruction combining pixels from a single row to generate a pixel matrix having more than one row;

an arithmetic-logic unit configured to perform operations corresponding to the decoded SIMD instructions on each of the plurality of registers, each operation being simultaneously performed on each field in a register without causing carry bits to bleed from one field in the register to an adjacent field in the register;

an I/O unit configured to interface with at least one external device and configured to receive data from and transmit data to the external device; and

a data bus configured to interconnect the plurality of registers, the arithmetic-logic unit and the I/O unit.

2. The processor for video analytics according to claim 1, wherein the predetermined instruction set also includes a linear interpolation instruction with a result rounded towards a predetermined value.

3. The processor for video analytics according to claim 2, wherein the linear interpolation instruction is rounded towards zero.

4. The processor for video analytics according to claim 1, wherein the predetermined instruction set includes an n-pixel by n-pixel filtering instruction, the filtering instruction holding a number of bits per pixel constant during filtering.

5. The processor for video analytics according to claim 1, wherein the arithmetic-logic unit includes at least two parallel adders, each of the parallel adders corresponding to one of the at least two fields.

6. A filtering method for video analytics using an n-pixel by n-pixel filter, the filtering method comprising:

packing a source image including a plurality of pixels into a plurality of registers, each of said plurality of registers being partitioned into at least two fields, each field corresponding to a single pixel, and each pixel including a predetermined number of bits;

applying, in a data processor, an n-pixel by n-pixel filter kernel to the plurality of registers using SIMD instructions to generate a plurality of partial results, said SIMD instructions including at least a splice instruction, an averaging instruction and a half subtraction instruction, the splice instruction combining pixels from a single row to generate a pixel matrix having more than one row;

combining the partial results, in a data processor, using SIMD instructions to generate a final result; and

outputting the final result to at least one of a display and an electronic storage medium, wherein the predetermined number of bits is held constant throughout the filtering method.

7. The filtering method according to claim 6, wherein when n equals three, the packing step includes

dividing the source image into a plurality of vertical components and a plurality of horizontal components,

dividing the filter kernel into three vertical components and three horizontal components, and

the applying step includes successively averaging the vertical components in combinations corresponding to filter weights included in the three filter kernel vertical components to generate a vertical partial result,

successively averaging the horizontal components in combinations corresponding to filter weights included in the three filter kernel horizontal components to generate a horizontal partial result, and

combining the vertical and horizontal partial results using a half-subtraction-with-bias operator to generate a filtered result.

8. The filtering method according to claim 7, wherein when n is greater than three, the SIMD instructions include a linear interpolation instruction, and the vertical and partial results are computed using the linear interpolation instruction.

9. A method of performing video analytics comprising:

partitioning a register into at least two fields;

packing a pixel, including a predetermined number of bits, into each field of the register; and

performing SIMD instructions on the entire register, the SIMD instructions corresponding to a predefined SIMD instruction set including at least a splice instruction, an averaging instruction and a half subtraction instruction,

wherein the splice instruction combines pixels from a single row to generate a pixel matrix having more than one row.

10. The method of performing video analytics according to claim 9, further comprising:

masking the register with a predetermined mask to prevent carry bits from crossing fields.

11. A processor for video analytics comprising:

means for storing pixel data contiguously;

means for decoding SIMD instructions from a predetermined SIMD instruction set, the SIMD instruction set including at least a splice instruction, an averaging instruction and a half subtraction instruction, the splice instruction combining pixels from a single row to generate a pixel matrix having more than one row;

means for performing operations, corresponding to the decoded instructions, simultaneously on the contiguously stored pixel data, each pixel data contiguously stored being insulated from carry bits of other pixel data contiguously stored;

means for receiving data from an external device, and

means for transmitting data to an external device.

12. The processor for video analytics according to claim 11, wherein the predetermined SIMD instruction set also includes a linear interpolation instruction with a result rounded towards a predetermined value.

13. The processor for video analytics according to claim 12 wherein the linear interpolation instruction is rounded towards zero.

14. The processor for video analytics according to claim 13, wherein the predetermined SIMD instruction set includes an n-pixel by n-pixel filtering instruction, the filtering instruction holding a number of bits, corresponding to an input argument, constant during filtering.

15. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instruction when executed by a computer cause the computer to perform the steps comprising:

partitioning each register of a plurality of registers into a plurality of fields, each register including at least two fields;

packing a source image including a plurality of pixels into the plurality of registers, each field corresponding to a single pixel, each pixel including a predetermined number of bits;

applying an n-pixel by n-pixel filter kernel to the plurality of registers using SIMD instructions to generate a plurality of partial results, said SIMD instructions including at least a splice instruction, an averaging instruction and a half subtraction instruction, the splice instruction combining pixels from a single pixel row to generate a pixel matrix having more than one row;

combining the partial results, in a data processor, using the SIMD instructions to generate a final result; and

outputting the final result to at least one of a display and an electronic storage medium, wherein

the predetermined number of bits of each pixel is held constant in the applying step and the combining step.

16. The non-transitory computer-readable storage medium according to claim 15, wherein when n equals three, the packing step includes

dividing the source image into a plurality of vertical components and a plurality of horizontal components,

dividing the filter kernel into three vertical components and three horizontal components, and

the applying step includes successively averaging the vertical components in combinations corresponding to filter weights included in the three filter kernel vertical components to generate a vertical partial result,

successively averaging the horizontal components in combinations corresponding to filter weights included in the three filter kernel horizontal components to generate a horizontal partial result, and

combining the vertical and horizontal partial results using a half-subtraction-with-bias operator to generate a filtered result.

17. The non-transitory computer-readable storage medium according to claim 16, wherein when n is greater than three, the SIMD instructions include a linear interpolation instruction, and the vertical and horizontal partial results are computed using the linear interpolation instruction.