FACE AND OTHER OBJECT DETECTION AND TRACKING IN OFF-CENTER PERIPHERAL REGIONS FOR NONLINEAR LENS GEOMETRIES

• US 20120249726A1
• Filed: 04/02/2011
• Published: 10/04/2012
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Bigioi, Petronel, Stec, Piotr
• Granted Patent: US 8,982,180 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A technique of enhancing a scene containing one or more off-center peripheral regions within an initial distorted image captured with a large field of view includes determining and extracting an off-center region of interest (hereinafter “ROI”) within the image. Geometric correction is applied to reconstruct the off-center ROI into a rectangular or otherwise undistorted or less distorted frame of reference as a reconstructed ROI.

Claims

1. Within an image acquisition system comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said image;

applying an initial object detection process to said off-center ROI;

responsive to an initial determination of an object being detected within the off-center ROI, applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI to refine the initial object detection process; and

performing a further action based on a value of a parameter of the object tracked within the first reconstructed ROI.

2. A method as in claim 1, further comprising compensating for global motion of the imaging device.

3. A method as in claim 1, wherein the parameter comprises location.

4. A method as in claim 1, further comprising determining a quality of reconstructed pixels within said first reconstructed ROI;

and selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels.

5. A method as in claim 1, wherein responsive to said analysis and said pixel quality, selectively applying image enhancement and generating an enhanced output image.

6. A method as in claim 1, further comprising determining if any undetected faces below a size threshold are likely to exist in one or more reduced quality regions of the reconstructed ROI.

7. A method as in claim 6, further comprising, responsive to said determining if any undetected faces exist, acquiring one or more additional initial distorted images, extracting and reconstructing one or more matching ROIs, and combining said one or more matching ROIs with reduced quality pixels of the original reconstructed ROI using a super-resolution technique, and generating, based on the combining, one or more enhanced ROIs.

8. A method as in claim 7, further comprising applying additional face detecting or tracking, or both, to the one or more enhanced ROIs to confirm a presence and location of a face below an original size threshold.

9. A digital image acquisition device, comprising:

a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;

a processor;

a memory having code embedded therein for programming the processor to perform a method of enhancing a scene containing one or more off-center peripheral regions, wherein the method comprises;

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said image;

applying an initial object detection process to said off-center ROI;

responsive to an initial determination of an object being detected within the off-center ROI, applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI to refine the initial object detection process; and

performing a further action based on a value of a parameter of the object tracked within the first reconstructed ROI.

10. A device as in claim 9, wherein the method further comprises compensating for global motion of the imaging device.

11. A device as in claim 9, wherein the parameter comprises location.

12. A device as in claim 9, wherein the method further comprises determining a quality of reconstructed pixels within said first reconstructed ROI;

and selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels.

13. A device as in claim 9, wherein responsive to said analysis and said pixel quality, selectively applying image enhancement and generating an enhanced output image.

14. A device as in claim 9, wherein the method further comprises determining if any undetected faces below a size threshold are likely to exist in one or more reduced quality regions of the reconstructed ROI.

15. A device as in claim 14, wherein the method further comprises, responsive to said determining if any undetected faces exist, acquiring one or more additional initial distorted images, extracting and reconstructing one or more matching ROIs, and combining said one or more matching ROIs with reduced quality pixels of the original reconstructed ROI using a super-resolution technique, and generating, based on the combining, one or more enhanced ROIs.

16. A device as in claim 15, wherein the method further comprises applying additional face detecting or tracking, or both, to the one or more enhanced ROIs to confirm a presence and location of a face below an original size threshold.

17. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein the method comprises:

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within an acquired image;

applying an initial object detection process to said off-center ROI;

responsive to an initial determination of an object being detected within the off-center ROI, applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI to refine the initial object detection process; and

performing a further action based on a value of a parameter of the object tracked within the first reconstructed ROI.

18. A device as in claim 17, wherein the method further comprises compensating for global motion of the imaging device.

19. One or more non-transitory, processor-readable storage media as in claim 17, wherein the parameter comprises location.

20. One or more non-transitory, processor-readable storage media as in claim 17, wherein the method further comprises determining a quality of reconstructed pixels within said first reconstructed ROI;

and selectively applying image analysis to said reconstructed ROI based on the quality of the reconstructed pixels.

21. One or more non-transitory, processor-readable storage media as in claim 17, wherein responsive to said analysis and said pixel quality, selectively applying image enhancement and generating an enhanced output image.

22. One or more non-transitory, processor-readable storage media as in claim 17, wherein the method further comprises determining if any undetected faces below a size threshold are likely to exist in one or more reduced quality regions of the reconstructed ROI.

23. One or more non-transitory, processor-readable storage media as in claim 22, wherein the method further comprises, responsive to said determining if any undetected faces exist, acquiring one or more additional initial distorted images, extracting and reconstructing one or more matching ROIs, and combining said one or more matching ROIs with reduced quality pixels of the original reconstructed ROI using a super-resolution technique, and generating, based on the combining, one or more enhanced ROIs.

24. One or more non-transitory, processor-readable storage media as in claim 23, wherein the method further comprises applying additional face detecting or tracking, or both, to the one or more enhanced ROIs to confirm a presence and location of a face below an original size threshold.