Object Detection and Rendering for Wide Field of View (WFOV) Image Acquisition Systems

• US 20110216157A1
• Filed: 12/02/2010
• Published: 09/08/2011
• Est. Priority Date: 03/05/2010
• Status: Abandoned Application

• Current Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Gangea, Mihnea, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Stec, Piotr
• Application Number:
  US12/959,137
• Publication Number:
  US 20110216157A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G02B 21/22 Stereoscopic arrangements
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An image acquisition device having a wide field of view includes a lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°. The device has an object detection engine that includes one or more cascades of object classifiers, e.g., face classifiers. A WFoV correction engine may apply rectilinear and/or cylindrical projections to pixels of the WFoV image, and/or non-linear, rectilinear and/or cylindrical lens elements or lens portions serve to prevent and/or correct distortion within the original WFoV image. One or more objects located within the original and/or distortion-corrected WFoV image is/are detectable by the object detection engine upon application of the one or more cascades of object classifiers.

Claims

1. An image acquisition device having a wide field of view, comprising:

at least one non-linear lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the non-linear lens is configured to project a center region of a scene onto the middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region;

a control module;

an object detection engine comprising one or more cascades of regular object classifiers;

a WFoV correction engine configured to correct distortion within the original WFoV image;

wherein the WFoV correction engine processes raw image data of the original WFoV image including applying a cylindrical projection of outer pixels of the original WFoV image, and combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image; and

wherein one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image are detectable by the object detection engine upon application of the one or more cascades of regular object classifiers.

2. The device of claim 1, further comprising:

a full frame buffer coupled with the image sensor for acquiring raw image data;

a mixer; and

a zoom and pan engine.

3. The device of claim 1, further comprising an object tracking engine.

4. The device of claim 1, wherein the object classifiers comprise face classifiers.

5. The device of claim 1, wherein the regular object classifiers comprise rectangular object classifiers.

6. An image acquisition device having a wide field of view, comprising:

at least one non-linear lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the non-linear lens is configured to project a center region of a scene onto the middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region;

a control module;

an object detection engine comprising one or more cascades of modified object classifiers;

wherein the modified object classifiers comprise a subset of cylindrical classifiers to be applied to objects appearing in outer pixels of the WFoV image; and

wherein one or more objects located within the center or outer pixels, or both, of the original WFoV image are detectable by the object detection engine upon application of the one or more cascades of modified object classifiers, including a subset of regular classifiers and the subset of cylindrical classifiers, respectively.

7. The device of claim 6, further comprising:

a full frame buffer coupled with the image sensor for acquiring raw image data;

a mixer; and

a zoom and pan engine.

8. The device of claim 6, further comprising an object tracking engine

9. The device of claim 6, further comprising a WFoV correction engine configured to correct distortion within the original image;

and wherein the WFoV correction engine processes raw image data of the original WFoV image including applying a cylindrical mapping of outer pixels of the original WFoV image, and combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image.

10. The device of claim 6, wherein the object classifiers comprise face classifiers.

11. The device of claim 6, wherein the regular object classifiers comprise rectangular object classifiers.

12. A method of acquiring wide field of view images with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

acquiring the original WFoV image, including utilizing at least one non-linear lens to project a center region of a scene onto a middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region;

correcting distortion within the original WFoV image by processing raw image data of the original WFoV image including

applying a cylindrical projection of outer pixels of the original WFoV image, and

combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image; and

detecting one or more objects by applying one or more cascades of regular object classifiers to one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image.

13. The method of claim 12, further comprising tracking one or more detected objects over multiple sequential frames.

14. The method of claim 12, wherein the object classifiers comprise face classifiers.

15. The method of claim 12, wherein the regular object classifiers comprise rectangular object classifiers.

16. A method of acquiring wide field of view images with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

acquiring the original WFoV image, including utilizing at least one non-linear lens to project a center region of a scene onto a middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region;

applying one or more modified object classifiers, comprising

applying a subset of cylindrical classifiers to objects appearing in outer pixels of the WFoV image; and

applying a subset of regular classifiers to objects appearing in center pixels of the WFoV image;

detecting one or more objects located within center or outer pixels, or both, of the original WFoV image by the applying of the one or more cascades of modified object classifiers, including the applying of the subset of regular classifiers and the applying of the subset of cylindrical classifiers, respectively.

17. The method of claim 16, further comprising tracking one or more detected objects over multiple sequential frames.

18. The method of claim 16, further comprising correcting distortion within the original WFoV image by processing raw image data of the original WFoV image including applying a cylindrical mapping of outer pixels of the original WFoV image, and combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image.

19. The method of claim 16, wherein the object classifiers comprise face classifiers.

20. The method of claim 16, wherein the regular object classifiers comprise rectangular object classifiers.

21. One or more processor-readable media having embedded therein code for programming a processor to perform a method of processing wide field of view images acquired with an image acquisition device having an image sensor and at least one non-linear lens to project a center region of a scene onto a middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region to acquire an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

correcting distortion within the original WFoV image by processing raw image data of the original WFoV image including

applying a cylindrical projection of outer pixels of the original WFoV image, and

combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image; and

detecting one or more objects by applying one or more cascades of regular object classifiers to one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image.

22. The one or more processor-readable media of claim 21, wherein the method further comprises tracking one or more detected objects over multiple sequential frames.

23. The one or more processor-readable media of claim 21, wherein the object classifiers comprise face classifiers.

24. The one or more processor-readable media of claim 21, wherein the regular object classifiers comprise rectangular object classifiers.

25. One or more processor-readable media having embedded therein code for programming a processor to perform a method of processing wide field of view images acquired with an image acquisition device having an image sensor and at least one non-linear lens to project a center region of a scene onto a middle portion of the image sensor such as to directly provide a rectilinear mapping of the center region to acquire an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

applying one or more modified object classifiers, comprising

applying a subset of cylindrical classifiers to objects appearing in outer pixels of the WFoV image; and

applying a subset of regular classifiers to objects appearing in center pixels of the WFoV image;

detecting one or more objects located within center or outer pixels, or both, of the original WFoV image by the applying of the one or more cascades of modified object classifiers, including the applying of the subset of regular classifiers and the applying of the subset of cylindrical classifiers, respectively.

26. The one or more processor-readable media of claim 25, wherein the method further comprises tracking one or more detected objects over multiple sequential frames.

27. The one or more processor-readable media of claim 25, wherein the method further comprises correcting distortion within the original WFoV image by processing raw image data of the original WFoV image including applying a cylindrical mapping of outer pixels of the original WFoV image, and combining center pixels and modified outer pixels to generate a distortion-corrected WFoV image.

28. The one or more processor-readable media of claim 25, wherein the object classifiers comprise face classifiers.

29. The one or more processor-readable media of claim 25, wherein the regular object classifiers comprise rectangular object classifiers.