Object Detection and Rendering for Wide Field of View (WFOV) Image Acquisition Systems

• US 20110216156A1
• Filed: 12/02/2010
• Published: 09/08/2011
• Est. Priority Date: 03/05/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Gangea, Mihnea, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Stec, Piotr
• Granted Patent: US 8,872,887 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G02B 21/22 Stereoscopic arrangements
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An image acquisition device having a wide field of view includes a lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°. The device has an object detection engine that includes one or more cascades of object classifiers, e.g., face classifiers. A WFoV correction engine may apply rectilinear and/or cylindrical projections to pixels of the WFoV image, and/or non-linear, rectilinear and/or cylindrical lens elements or lens portions serve to prevent and/or correct distortion within the original WFoV image. One or more objects located within the original and/or distortion-corrected WFoV image is/are detectable by the object detection engine upon application of the one or more cascades of object classifiers.

Claims

1. An image acquisition device having a wide field of view, comprising:

at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

;

a control module;

an object detection engine comprising one or more cascades of regular object classifiers;

a WFoV correction engine configured to correct distortion within the original image;

wherein the WFoV correction engine processes raw image data of the original WFoV image including applying a rectilinear projection of center pixels of the original WFoV image and a cylindrical projection of outer pixels of the original WFoV image, and combining modified center and outer pixels to generate a distortion-corrected WFoV image; and

wherein one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image are detectable by the object detection engine upon application of the one or more cascades of regular object classifiers.

2. The device of claim 1, further comprising:

a full frame buffer coupled with the image sensor for acquiring raw image data;

a mixer; and

a zoom and pan engine.

3. The device of claim 1, wherein the applying a rectilinear projection to center pixels comprises applying a regular rectilinear projection to an inner portion of the center pixels and a squeezed rectilinear projection to an outer portion of the center pixels.

4. The device of claim 3, wherein the applying a squeezed rectilinear projection to the outer portion of the center pixels comprises applying an increasingly squeezed rectilinear projection in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

5. The device of claim 1, further comprising an object tracking engine.

6. The device of claim 1, wherein the object classifiers comprise face classifiers.

7. The device of claim 1, wherein the regular object classifiers comprise rectangular object classifiers.

8. An image acquisition device having a wide field of view, comprising:

at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

;

a control module;

an object detection engine comprising one or more cascades of modified object classifiers;

wherein the modified object classifiers comprise a first subset of rectilinear classifiers to be applied to objects appearing in center pixels of the WFoV image, and a second subset of cylindrical classifiers to be applied to objects appearing in outer pixels of the WFoV image; and

wherein one or more objects located within the center or outer pixels, or both, of the original WFoV image are detectable by the object detection engine upon application of the one or more cascades of modified object classifiers, including the first subset of rectilinear classifiers and the second subset of cylindrical classifiers, respectively.

9. The device of claim 8, further comprising:

a full frame buffer coupled with the image sensor for acquiring raw image data;

a mixer; and

a zoom and pan engine.

10. The device of claim 8, wherein the first subset of rectilinear classifiers comprise a subset of regular rectilinear classifiers with which objects appearing in an inner portion of the center pixels are detectable, and a subset of squeezed rectilinear classifiers with which objects appearing in an outer portion of the center pixels are detectable.

11. The device of claim 10, wherein the subset of squeezed rectilinear classifiers comprises subsets of increasingly squeezed rectilinear classifiers with which objects appearing in the outer portion of the center pixels are increasingly detectable in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

12. The device of claim 8, further comprising an object tracking engine

13. The device of claim 8, further comprising a WFoV correction engine configured to correct distortion within the original image;

and wherein the WFoV correction engine processes raw image data of the original WFoV image including applying a rectilinear mapping of center pixels of the original WFoV image and a cylindrical mapping of outer pixels of the original WFoV image, and combining modified center and outer pixels to generate a distortion-corrected WFoV image.

14. The device of claim 8, wherein the object classifiers comprise face classifiers.

15. The device of claim 8, wherein the regular object classifiers comprise rectangular object classifiers.

16. A method of acquiring wide field of view images with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

acquiring the original WFoV image;

correcting distortion within the original WFoV image, by processing raw image data of the original WFoV image including

applying a rectilinear projection to center pixels of the original WFoV image and a cylindrical projection of outer pixels of the original WFoV image, and

combining modified center and outer pixels to generate a distortion-corrected WFoV image; and

applying one or more cascades of regular object classifiers to detect one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image upon application of the one or more cascades of regular object classifiers.

17. The method of claim 16, wherein the applying a rectilinear projection to center pixels comprises applying a regular rectilinear projection to an inner portion of the center pixels and a squeezed rectilinear projection to an outer portion of the center pixels.

18. The method of claim 17, wherein the applying a squeezed rectilinear projection to the outer portion of the center pixels comprises applying an increasingly squeezed rectilinear projection in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

19. The method of claim 16, further comprising tracking one or more detected objects over multiple sequential frames.

20. The method of claim 16, wherein the object classifiers comprise face classifiers.

21. The method of claim 16, wherein the regular object classifiers comprises rectilinear object classifiers.

28. The method of claim 17, wherein the regular object classifiers comprises rectilinear object classifiers.

22. A method of acquiring wide field of view images with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

acquiring the original WFoV image;

applying one or more cascades of modified object classifiers, comprising

applying a first subset of rectilinear classifiers to objects appearing in center pixels of the WFoV image, and

applying a second subset of cylindrical classifiers to objects appearing in outer pixels of the WFoV image; and

detecting one or more objects located within the center or outer pixels, or both, of the original WFoV image by the applying of the modified object classifiers, including the applying of the first subset of rectilinear classifiers and the applying of the second subset of cylindrical classifiers, respectively.

23. The method of claim 22, wherein the applying of the first subset of rectilinear classifiers comprises applying a subset of regular rectilinear classifiers with which objects appearing in an inner portion of the center pixels are detectable, and applying a subset of squeezed rectilinear classifiers with which objects appearing in an outer portion of the center pixels are detectable.

24. The method of claim 23, wherein the applying of the subset of squeezed rectilinear classifiers comprises applying subsets of increasingly squeezed rectilinear classifiers with which objects appearing in the outer portion of the center pixels are increasingly detectable in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

25. The method of claim 22, further comprising tracking one or more objects over multiple sequential frames.

26. The method of claim 22, further comprising correcting distortion within the original image by processing raw image data of the original WFoV image including applying a rectilinear mapping of center pixels of the original WFoV image and a cylindrical mapping of outer pixels of the original WFoV image, and combining modified center and outer pixels to generate a distortion-corrected WFoV image.

27. The method of claim 22, wherein the object classifiers comprise face classifiers.

29. One or more processor-readable media having embedded therein code for programming a processor to perform a method of processing wide field of view images acquired with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

correcting distortion within the original WFoV image, by processing raw image data of the original WFoV image including

applying a rectilinear projection to center pixels of the original WFoV image and a cylindrical projection of outer pixels of the original WFoV image, and

combining modified center and outer pixels to generate a distortion-corrected WFoV image; and

applying one or more cascades of regular object classifiers to detect one or more objects located within the center or outer pixels, or both, of the distortion-corrected WFoV image upon application of the one or more cascades of regular object classifiers.

30. The one or more processor-readable media of claim 29, wherein the applying a rectilinear projection to center pixels comprises applying a regular rectilinear projection to an inner portion of the center pixels and a squeezed rectilinear projection to an outer portion of the center pixels.

31. The one or more processor-readable media of claim 30, wherein the applying a squeezed rectilinear projection to the outer portion of the center pixels comprises applying an increasingly squeezed rectilinear projection in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

32. The one or more processor-readable media of claim 29, wherein the method further comprises tracking one or more detected objects over multiple sequential frames.

33. The one or more processor-readable media of claim 29, wherein the object classifiers comprise face classifiers.

34. The one or more processor-readable media of claim 29, wherein the regular object classifiers comprises rectilinear object classifiers.

35. One or more processor-readable media having embedded therein code for programming a processor to perform a method of processing wide field of view images acquired with an image acquisition device having at least one lens and image sensor configured to capture an original wide field of view (WFoV) image with a field of view of more than 90°

, wherein the method comprises;

applying one or more cascades of modified object classifiers, comprising

applying a first subset of rectilinear classifiers to be applied to objects appearing in center pixels of the WFoV image, and

applying a second subset of cylindrical classifiers to be applied to objects appearing in outer pixels of the WFoV image; and

detecting one or more objects located within the center or outer pixels, or both, of the original WFoV image by the applying of the modified object classifiers, including the applying of the first subset of rectilinear classifiers and the applying of the second subset of cylindrical classifiers, respectively.

36. The one or more processor-readable media of claim 35, wherein the applying of the first subset of rectilinear classifiers comprises applying a subset of regular rectilinear classifiers with which objects appearing in an inner portion of the center pixels are detectable, and applying a subset of squeezed rectilinear classifiers with which objects appearing in an outer portion of the center pixels are detectable.

37. The one or more processor-readable media of claim 36, wherein the applying of the subset of squeezed rectilinear classifiers comprises applying subsets of increasingly squeezed rectilinear classifiers with which objects appearing in the outer portion of the center pixels are increasingly detectable in a direction from a first boundary with the inner portion of the center pixels to a second boundary with the outer pixels.

38. The one or more processor-readable media of claim 35, wherein the method further comprises tracking one or more objects over multiple sequential frames.

39. The one or more processor-readable media of claim 35, wherein the method further comprises correcting distortion within the original image by processing raw image data of the original WFoV image including applying a rectilinear mapping of center pixels of the original WFoV image and a cylindrical mapping of outer pixels of the original WFoV image, and combining modified center and outer pixels to generate a distortion-corrected WFoV image.

40. The one or more processor-readable media of claim 35, wherein the object classifiers comprise face classifiers.

41. The one or more processor-readable media of claim 35, wherein the regular object classifiers comprises rectilinear object classifiers.