FACE OR OTHER OBJECT DETECTION INCLUDING TEMPLATE MATCHING

• US 20150206030A1
• Filed: 03/30/2015
• Published: 07/23/2015
• Est. Priority Date: 12/29/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: PETRESCU, STEFAN, BIGIOI, PETRONEL, CORCORAN, PETER, GANGEA, MIHNEA, SULTANA, BOGDAN, NICOLAU, RADU, URSACHI, VLAD IONUT, ZAHARIA, CORNELIU, FULOP, SZABOLCS
• Granted Patent: US 9,639,775 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/164 using holistic features

Abstract

A template matching module is configured to program a processor to apply multiple differently-tuned object detection classifier sets in parallel to a digital image to determine one or more of an object type, configuration, orientation, pose or illumination condition, and to dynamically switch between object detection templates to match a determined object type, configuration, orientation, pose, blur, exposure and/or directional illumination condition.

Claims

1. A digital image processing device, comprising:

an optoelectronic system;

a memory;

two or more image processing units;

wherein the optoelectronic system acquires a plurality of digital images;

wherein a first image processing unit of the two or more image processing units;

determines, for a first digital image of the plurality of digital images, a location and a boundary of one or more spatial regions where one can expect to detect a face;

stores in the memory the first digital image and information about the location and the boundary of one or more spatial regions;

processes a second digital image of the plurality of digital images;

wherein, as the first image processing unit processes the second digital image, a second image processing unit of the two or more image processing units;

retrieves from the memory the location and the boundary of one or more spatial regions; and

based on the location and the boundary, retrieves the one or more spatial regions of the first digital image and performs an additional processing on the one or more spatial regions.

2. The digital image processing device of claim 1,

wherein the memory is dual-ported;

and

wherein one port of the memory is used to read one digital image data while another port of the memory is used to store another digital image data.

3. The digital image processing device of claim 1,

wherein the second image processing unit performs the additional processing of the one or more spatial regions while the first image processing unit performs a generic processing of the second digital image.

4. The digital image processing device of claim 3,

wherein the additional processing of the one or more spatial regions is delayed by a frame-processing-cycle delay;

and

wherein the frame-processing-cycle delay in the additional processing of the first digital image is compensated to achieve a pseudo-real-time processing of the first digital image.

5. The digital image processing device of claim 1,

wherein, in response to performing the additional processing on the one or more spatial regions, the first digital image is transmitted to a face tracking unit.

6. The digital image processing device of claim 1,

wherein the first image processing unit of the two or more image processing units stores the first digital image and the information about the location and the boundary of one or more spatial regions in one or more buffers associated with the first image processing unit.

7. The digital image processing device of claim 6,

wherein, upon receiving the second digital image, the first image processing unit retrieves the information about the location and the boundary of one or more spatial regions from the one or more buffers, and performs the additional processing on the one or more spatial regions.

8. A method for a hardware-based acceleration of object detection in digital images, the method comprising:

acquiring a plurality of digital images;

at a first image processing unit of two or more image processing units;

determining, for a first digital image of the plurality of digital images, a location and a boundary of one or more spatial regions where one can expect to detect a face;

storing in a memory the first digital image and information about the location and the boundary of one or more spatial regions;

processing a second digital image of the plurality of digital images;

at a second image processing unit of the two or more processing units, as the first image processing unit processes the second digital image;

retrieving from the memory the location and the boundary of one or more spatial regions; and

based on the location and the boundary, retrieving the one or more spatial regions of the first digital image and performing an additional processing on the one or more spatial regions.

9. The method of claim 8,

wherein the memory is dual-potted;

and

wherein one port of the memory is used to read one digital image data while another port of the memory is used to store another digital image data.

10. The method of claim 8,

wherein the second in/age processing unit performs the additional processing of the one or more spatial regions while the first image processing unit performs a generic processing of the second digital image.

11. The method of claim 10,

wherein the additional processing of the one or more spatial regions is delayed by a frame-processing-cycle delay;

and

wherein the franc-processing-cycle delay in the additional processing of the first digital image is compensated to achieve a pseudo-real-time processing of the first digital image.

12. The method of claim 8,

wherein, in response to performing the additional processing on the one or more spatial regions, the first digital image is transmitted to a face tracking unit.

13. The method of claim 8,

wherein the first image processing unit of the two or more image processing units stores the first digital image and the information about the location and the boundary of one or more spatial regions in one or more buffers associated with the first image processing unit.

14. The method of claim 13,

wherein, upon receiving the second digital image, the first image processing unit retrieves the information about the location and the boundary of one or more spatial regions from the one or more buffers, and performs the additional processing on the one or more spatial regions.

15. A non-transitory computer-readable storage medium storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform:

acquiring a plurality of digital images;

at a first image processing unit of two or more image processing units;

determining, for a first digital image of the plurality of digital images, a location and a boundary of one or more spatial regions where one can expect to detect a face;

storing in a memory the first digital image and information about the location and the boundary of one or more spatial regions;

processing a second digital image of the plurality of digital images;

at a second image processing unit of the two or more processing units, as the first image processing unit processes the second digital image;

retrieving from the memory the location and the boundary of one or more spatial regions; and

based on the location and the boundary, retrieving the one or more spatial regions of the first digital image and performing an additional processing on the one or more spatial regions.

16. The non-transitory computer-readable storage medium of claim 15,

wherein the memory is dual-ported;

and

wherein one port of the memory is used to read one digital image data while another port of the memory is used to store another digital image data.

17. The non-transitory computer-readable storage medium of claim 16,

wherein the second image processing unit performs the additional processing of the one or more spatial regions while the first image processing unit performs a generic processing of the second digital image.

18. The non-transitory computer-readable storage medium of claim 15,

wherein the additional processing of the one or more spatial regions is delayed by a frame-processing-cycle delay;

and

wherein the frame-processing-cycle delay in the additional processing of the first digital image is compensated to achieve a pseudo-real-time processing of the first digital image.

19. The non-transitory computer-readable storage medium of claim 15,

wherein, in response to performing the additional processing on the one or more spatial regions, the first digital image is transmitted to a face tracking unit.

20. The non-transitory computer-readable storage medium of claim 15,

wherein the first image processing unit of the two or more image processing units stores the first digital image and the information about the location and the boundary of the one or more spatial region in one or more buffers associated with the first image processing unit.