Image processing system

• US 20170185864A1
• Filed: 12/15/2016
• Published: 06/29/2017
• Est. Priority Date: 12/23/2015
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: FotoNation Limited (Adeia, Inc.)
• Inventors: NICOARA, Nicolae, RACEALA, Cristina, FULOP, Szabolcs, IOVITA, Oana, ZAHARIA, Corneliu
• Granted Patent: US 10,460,198 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

An image processing system comprises a template matching engine (TME). The TME reads an image from the memory; and as each pixel of the image is being read, calculates a respective feature value of a plurality of feature maps as a function of the pixel value. A pre-filter is responsive to a current pixel location comprising a node within a limited detector cascade to be applied to a window within the image to: compare a feature value from a selected one of the plurality of feature maps corresponding to the pixel location to a threshold value; and responsive to pixels for all nodes within a limited detector cascade to be applied to the window having been read, determine a score for the window. A classifier, responsive to the pre-filter indicating that a score for a window is below a window threshold, does not apply a longer detector cascade to the window before indicating that the window does not comprise an object to be detected.

Claims

1. An image processing system comprising a template matching engine (TME) operatively connected to a memory storing image information, the TME being configured to:

read at least a portion of an image from said memory using a raster scan; and

as each pixel of said image portion is being read, calculate a respective feature value of a plurality of feature maps as a function of said pixel value;

the TME further comprising;

a pre-filter responsive to a current pixel location comprising a node within a limited detector cascade to be applied to a window within said portion of an image to;

compare a feature value from a selected one of said plurality of feature maps corresponding to said pixel location to a threshold value; and

responsive to pixels for all nodes within a limited detector cascade to be applied to said window having been read, determine a score for said window based on the comparisons of said feature values and said threshold values for said nodes; and

a classifier, responsive to said pre-filter indicating that a score for a window is below a window threshold, not applying a longer detector cascade to said window before indicating that said window does not comprise an object to be detected.

2. An image processing system as claimed in claim 1 wherein said TME is arranged to sub-sample said image prior to calculating said feature values.

3. An image processing system as claimed in claim 1 wherein said TME is arranged to simultaneously read a plurality of pixels from said memory, each pixel corresponding to given relative pixel location within a sequence of windows, said pre-filter being responsive to said given relative pixel location corresponding to a node within a limited detector cascade to be applied to said sequence of windows, to simultaneously compare respective feature values from a selected one of said plurality of feature maps corresponding to said given relative pixel location to a threshold value;

and responsive to pixels for all nodes within said limited detector cascade to be applied to said sequence of windows having been read, to determine respective scores for said sequence of windows based on the comparisons of said feature values and said threshold values for said nodes.

4. An image processing system as claimed in claim 3 wherein said TME is arranged to simultaneously read either 4 or 8 pixels.

5. An image processing system as claimed in claim 1 wherein the pre-filter is configured to reject approximately 95% of windows from needing to be analysed subsequently by said classifier with said longer detector cascade.

6. An image processing system as claimed in claim 1 wherein said classifier is configured to apply a plurality of detector cascades to any window not rejected by said pre-filter.

7. An image processing system as claimed in claim 6 wherein said classifier is configured to apply said plurality of detector cascades successively to one window at a time.

8. An image processing system as claimed in claim 1 wherein said classifier is arranged to simultaneously apply a given detector cascade to a sequence windows.

9. An image processing system as claimed in claim 1 wherein said nodes comprise nodes within successive stages of a multi-stage random tree classifier (RTC).

10. An image processing system as claimed in claim 9 wherein said RTC comprises 12 stages, each stage comprising a 3 node decision tree.

11. An image processing apparatus according to claim 1 wherein said TME is further arranged to provide values for a plurality of the following feature maps based on a pixel value:

an Intensity Image;

an Integral Image (II);

an II² map;

a Census map;

a LBP (linear binary pattern) map; and

a HOG (Histogram of Gradients) map.

12. An image processing system comprising a template matching engine (TME) operatively connected to a memory storing image information, the TME being configured to:

read at least a portion of an image from said memory using a raster scan; and

as each pixel of said image portion is being read, calculate a respective feature value of at least three feature maps as a function of said pixel value;

the TME comprising;

a classifier arranged to apply at least one detector cascade to a window within a portion of an image in order to indicate if said window comprises an object to be detected, one of said at least one detector cascades comprising a multi-stage random tree classifier (RTC), each stage comprising a decision tree having at least three nodes corresponding to respective pixel locations within said window, said classifier being arranged to compare a feature value from a selected one of said plurality of feature maps corresponding to a pixel location to a threshold value for each of said nodes of said detector cascade, wherein training of said detector cascade is restricted so that selected feature maps for each node of a stage are each different, said classifier being arranged to simultaneously read feature values for each node of a stage from said feature maps and to simultaneously compare each of said feature map values to respective thresholds to determine a score for a stage, and said classifier being arranged to compare an accumulated score for a window after each stage with a stage threshold to determine whether or not to continue with a next stage of said detector cascade.

13. An image processing system as claimed in claim 12 wherein said TME is arranged to simultaneously read a plurality of pixels from said memory, each pixel corresponding to given relative pixel location within a sequence of windows, said classifier being arranged to simultaneously apply a given stage from said detector cascade to each of said sequence of windows.

14. An image processing system as claimed in claim 12 wherein said TME is arranged to sub-sample said image prior to calculating said feature values.

15. An image processing system as claimed in claim 12 wherein said TME is arranged to simultaneously read a plurality of pixels from said memory, each pixel corresponding to given relative pixel location within a sequence of windows, said pre-filter being responsive to said given relative pixel location corresponding to a node within a limited detector cascade to be applied to said sequence of windows, to simultaneously compare respective feature values from a selected one of said plurality of feature maps corresponding to said given relative pixel location to a threshold value;

and responsive to pixels for all nodes within said limited detector cascade to be applied to said sequence of windows having been read, to determine respective scores for said sequence of windows based on the comparisons of said feature values and said threshold values for said nodes.

16. An image processing system as claimed in claim 12 wherein said classifier is configured to apply a plurality of detector cascades to any window not rejected by said pre-filter.

17. An image processing apparatus according to claim 12 wherein said TME is further arranged to provide values for a plurality of the following feature maps based on a pixel value:

an Intensity Image;

an Integral Image (II);

an II² map;

a Census map;

a LBP (linear binary pattern) map; and

a HOG (Histogram of Gradients) map.

18. An image processing system comprising a template matching engine (TME) operatively connected to a memory storing image information, the TME being configured to:

read at least a portion of an image from said memory using a raster scan; and

as each pixel of said image portion is being read, calculate a respective feature value of a plurality of feature maps as a function of said pixel value;

the TME comprising;

a classifier arranged to apply at least one multi-stage detector cascade to a window within a portion of an image in order to indicate if said window comprises an object to be detected, said classifier being arranged to compare an accumulated score for a window after each stage with a stage threshold to determine whether or not to continue with a next stage of said detector cascade; and

a programmable controller, said controller being arranged to provide said classifier with a plurality of limited stage detector cascades to be successively applied to a window, the programmable controller being arranged to receive a respective accumulated score for each limited stage detector cascade from said classifier and to apply rules from a rules engine to determine which of a plurality of longer detector cascades corresponding to said limited stage detector cascades are to be applied to said window to enable said classifier to indicate if said window comprises an object to be detected.

19. An image processing system as claimed in claim 18 wherein said TME is arranged to simultaneously read a plurality of pixels from said memory, each pixel corresponding to given relative pixel location within a sequence of windows, said classifier being arranged to simultaneously apply a given stage from a limited stage detector cascade to said sequence of windows, said controller being arranged to simultaneously receive from said classifier respective accumulated scores for said sequence of windows before determining which longer detector cascades corresponding to said limited stage detectors should be applied to said sequence of windows.

20. An image processing system as claimed in claim 19 wherein at least one of said plurality of detector cascades comprises a multi-stage random tree classifier (RTC), each stage comprising a decision tree having at least three nodes corresponding to respective pixel locations within said window, said classifier being arranged to compare a feature value from a selected one of said plurality of feature maps corresponding to said pixel location to a threshold value for each of said nodes.

21. An image processing system as claimed in claim 20 wherein training of said at least one detector cascade is restricted so that selected feature maps for each node of a stage are each different, said classifier being arranged to simultaneously read feature values for each node of a stage from said feature maps and to simultaneously compare each of said feature map values to respective thresholds to determine a score for a stage.

23. An image processing system as claimed in claim 18 wherein said TME is arranged to sub-sample said image prior to calculating said feature values.

24. An image processing system as claimed in claim 18 further comprising:

a pre-filter responsive to a current pixel location comprising a node within a limited detector cascade to be applied to a window within said portion of an image to;

compare a feature value from a selected one of said plurality of feature maps corresponding to said pixel location to a threshold value; and

responsive to pixels for all nodes within a limited detector cascade to be applied to said window having been read, determine a score for said window based on the comparisons of said feature values and said threshold values for said nodes, said classifier being responsive to said pre-filter indicating that a score for a window is below a window threshold, not applying a longer detector cascade to said window before indicating that said window does not comprise an object to be detected.

25. An image processing system as claimed in claim 24 wherein said TME is arranged to simultaneously read a plurality of pixels from said memory, each pixel corresponding to given relative pixel location within a sequence of windows, said pre-filter being responsive to said given relative pixel location corresponding to a node within a limited detector cascade to be applied to said sequence of windows, to simultaneously compare respective feature values from a selected one of said plurality of feature maps corresponding to said given relative pixel location to a threshold value;

and responsive to pixels for all nodes within said limited detector cascade to be applied to said sequence of windows having been read, to determine respective scores for said sequence of windows based on the comparisons of said feature values and said threshold values for said nodes.

26. An image processing apparatus according to claim 18 wherein said TME is further arranged to provide values for a plurality of the following feature maps based on a pixel value:

an Intensity Image;

an Integral Image (II);

an II² map;

a Census map;

a LBP (linear binary pattern) map; and

a HOG (Histogram of Gradients) map.

22. An image processing system as claimed in 20 in which each of said plurality of detector cascades is configured to identify a face in a respective one of a plurality of different poses.