Object detection and rendering for wide field of view (WFOV) image acquisition systems

• US 8,872,887 B2
• Filed: 12/02/2010
• Issued: 10/28/2014
• Est. Priority Date: 03/05/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Drimbarean, Alexandru, Gangea, Mihnea, Corcoran, Peter, Stec, Piotr
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Anderson, Matthew J
• Application Number:
  US12/959,089
• Publication Number:
  US 20110216156A1
• Time in Patent Office: 1,426 Days
• Field of Search: 348/36, 348/159
• US Class Current: 348/36
• CPC Class Codes: G02B 21/22 Stereoscopic arrangements
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An image acquisition device having a wide field of view includes a lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°. The device has an object detection engine that includes one or more cascades of object classifiers, e.g., face classifiers. A WFoV correction engine may apply rectilinear and/or cylindrical projections to pixels of the WFoV image, and/or non-linear, rectilinear and/or cylindrical lens elements or lens portions serve to prevent and/or correct distortion within the original WFoV image. One or more objects located within the original and/or distortion-corrected WFoV image is/are detectable by the object detection engine upon application of the one or more cascades of object classifiers.

Claims

1. An image acquisition device, comprising:

at least one lens and an image sensor configured to capture an original image;

a control module;

an object detection engine comprising one or more cascades of object classifiers;

wherein the object detection engine is configured to;

detect a center portion and one or more outer portions of the original image,

wherein the center portion of the original image comprises at least a first center portion and a second center portion;

wherein the first center portion at least partially overlaps the second center portion;

a correction engine configured to process the original image to correct distortions within the original image;

wherein the correction engine processes raw image data of the original image by;

applying a regular-rectilinear projection to the first center portion of the original image to generate first center pixels data;

applying an increasingly-squeezed-rectilinear projection to the second center portion of the original image to generate second center pixels data;

applying a cylindrical projection to the one or more outer portions of the original image to generate outer pixel data; and

combining at least the first center pixels data generated for the first center portion that at least partially overlaps the second center portions, the second center pixels data generated for the second center portion, and the outer pixels data to generate distortion-corrected data for a distortion-corrected image.

2. The device of claim 1, further comprising:

a full frame buffer coupled with the image sensor for acquiring the raw image data;

a mixer; and

a zoom and pan engine.

3. The device of claim 1,

wherein the applying a regular rectilinear projection to the first center portion comprises applying the regular rectilinear projection to an inner portion of the first center portion and a squeezed rectilinear projection to an outer portion of the first center portion;

wherein the object classifiers comprise rectangular object classifiers.

4. The device of claim 3, wherein the applying a squeezed rectilinear projection to the outer portion of the first center portion comprises applying the increasingly-squeezed-rectilinear projection in a direction from a first boundary of the inner portion of the first center portion to a second boundary of the outer portion of the first center portion.

5. The device of claim 1, further comprising an object tracking engine.

6. The device of claim 1, wherein the object classifiers comprise face classifiers.

7. The device of claim 1, wherein the object classifiers comprise rectangular object classifiers.

8. A method, comprising:

acquiring an original image;

detecting a center portion and one or more outer portions of the original image using object classifiers;

wherein the center portion of the original image comprises at least a first center portion and a second center portion;

wherein the first center portion at least partially overlaps the second center portion;

correcting distortions within the original image by;

applying a regular-rectilinear projection to the first center portion of the original image to generate first center pixels data;

applying an increasingly-squeezed-rectilinear projection to the second center portion of the original image to generate second center pixels data;

applying a cylindrical projection to the one or more outer portions of the original image to generate outer pixel data; and

combining at least the first center pixels data generated for the first center portion that at least partially overlaps the second center portions, the second center pixels data generated for the second center portion, and the outer pixels data to generate distortion-corrected data for a distortion-corrected image.

9. The method of claim 8, wherein the applying a regular rectilinear projection to the first center portion comprises applying the regular rectilinear projection to an inner portion of the first center portion and a squeezed rectilinear projection to an outer portion of the first center portion.

10. The method of claim 9, wherein the applying a squeezed rectilinear projection to the outer portion of the first center portion comprises applying an increasingly-squeezed-rectilinear projection in a direction from a first boundary of the inner portion of the first center portion to a second boundary of the outer portion of the first center portion.

11. The method of claim 8, further comprising tracking one or more detected objects over multiple sequential frames.

12. The method of claim 8, wherein the object classifiers comprise face classifiers.

13. The method of claim 8, wherein the object classifiers comprise rectilinear object classifiers.

14. One or more non-transitory computer-readable storage media having embedded therein for programming a processor to cause the processor to perform a:

acquiring a original image;

detecting a center portion and one or more outer portions of the original image using object classifiers;

wherein the center portion of the original image comprises at least a first center portion and a second center portion;

wherein the first center portion at least partially overlaps the second center portion;

correcting distortions within the original image by;

applying a regular-rectilinear projection to the first center portion of the original image to generate first center pixels data;

applying an increasingly-squeezed-rectilinear projection to the second center portion of the original image to generate second center pixels data;

applying a cylindrical projection to the one or more outer portions of the original image to generate outer pixel data; and

combining at least the first center pixels data generated for the first center portion that at least partially overlaps the second center portions, the second center pixels data generated for the second center portion, and the outer pixels data to generate distortion-corrected data for a distortion-corrected image.

15. The one or more computer-readable storage media of claim 14, storing additional code embedded therein for programming the processor to cause the processor to perform:

applying the regular rectilinear projection to an inner portion of the first center portion and a squeezed rectilinear projection to an outer portion of the first center portion.

16. The one or more computer-readable storage media of claim 15, storing additional code embedded therein for programming the processor to cause the processor to perform:

applying the increasingly-squeezed-rectilinear projection in a direction from a first boundary of the inner portion of the first center portion to a second boundary with the outer portion.

17. The one or more computer-readable storage media of claim 14, storing additional code embedded therein for programming the processor to cause the processor to perform:

tracking one or more detected objects over multiple sequential frames.

18. The one or more computer-readable storage media of claim 14, wherein the object classifiers comprise face classifiers.

19. The one or more computer-readable storage media of claim 14, wherein the object classifiers comprises rectilinear object classifiers.