FACE AND OTHER OBJECT TRACKING IN OFF-CENTER PERIPHERAL REGIONS FOR NONLINEAR LENS GEOMETRIES

• US 20120249725A1
• Filed: 04/02/2011
• Published: 10/04/2012
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Bigioi, Petronel, Stec, Piotr
• Granted Patent: US 8,723,959 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A technique of enhancing a scene containing one or more off-center peripheral regions within an initial distorted image captured with a large field of view includes determining and extracting an off-center region of interest (hereinafter “ROI”) within the image. Geometric correction is applied to reconstruct the off-center ROI into a rectangular or otherwise undistorted or less distorted frame of reference as a reconstructed ROI.

Claims

1. Within an image acquisition system comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said image;

applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

determining a quality of reconstructed pixels within said first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI respectively adapted to one or more local reconstructed pixel qualities within the one or more regions;

determining if undetected objects below a predetermined size threshold are likely to exist in one or more reduced quality regions of the first reconstructed ROI;

responsive to said determining if undetected objects exist, acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with reduced quality pixels of said first reconstructed ROI to provide one or more enhanced ROIs.

performing a further action based on a value of a parameter of an object within the one or more enhanced ROIs.

2. The method of claim 1, further comprising using a super-resolution technique to generate at least one of said one or more enhanced ROIs.

3. The method of claim 1, further comprising applying additional object tracking to the enhanced ROI to confirm a location of a object below said predetermined size threshold.

4. A method as in claim 1, further comprising compensating for global motion of the imaging device.

5. A method as in claim 1, wherein the parameter comprises location.

6. A method as in claim 1, further comprising, prior to said applying geometric correction, applying an initial object detection process to the off-center ROI.

7. A method as in claim 6, wherein said applying geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.

8. A method as in claim 7, wherein said object tracking is applied to the reconstructed ROI to refine or confirm, or both, the initial object detection process.

9. A digital image acquisition device, comprising:

a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;

a processor;

a memory having code embedded therein for programming the processor to perform a method of enhancing a scene containing one or more off-center peripheral regions, wherein the method comprises;

acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and imaging sensor;

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within said image;

applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

determining a quality of reconstructed pixels within said first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI respectively adapted to one or more local reconstructed pixel qualities within the one or more regions;

determining if undetected objects below a predetermined size threshold are likely to exist in one or more reduced quality regions of the first reconstructed ROI;

responsive to said determining if undetected objects exist, acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with reduced quality pixels of said first reconstructed ROI to provide one or more enhanced ROIs.

performing a further action based on a value of a parameter of an object within the one or more enhanced ROIs.

10. The device of claim 9, wherein the method further comprises using a super-resolution technique to generate at least one of said one or more enhanced ROIs.

11. The device of claim 9, wherein the method further comprises applying additional object tracking to the enhanced ROI to confirm a location of a object below said predetermined size threshold.

12. A device as in claim 9, wherein the method further comprises compensating for global motion of the imaging device.

13. A device as in claim 9, wherein the parameter comprises location.

14. A device as in claim 9, wherein the method further comprises, prior to said applying geometric correction, applying an initial object detection process to the off-center ROI.

15. A device as in claim 14, wherein said applying geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.

16. A device as in claim 15, wherein said object tracking is applied to the reconstructed ROI to refine or confirm, or both, the initial object detection process.

17. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein the method comprises:

determining and extracting an off-center region of interest (hereinafter “

ROI”

) within an acquired image;

applying geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;

determining a quality of reconstructed pixels within said first reconstructed ROI;

applying object tracking to one or more regions within the first reconstructed ROI respectively adapted to one or more local reconstructed pixel qualities within the one or more regions;

determining if undetected objects below a predetermined size threshold are likely to exist in one or more reduced quality regions of the first reconstructed ROI;

responsive to said determining if undetected objects exist, acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with reduced quality pixels of said first reconstructed ROI to provide one or more enhanced ROIs.

performing a further action based on a value of a parameter of an object within the one or more enhanced ROIs.

18. One or more non-transitory processor-readable storage media as in claim 17, wherein the method further comprises using a super-resolution technique to generate at least one of said one or more enhanced ROIs.

19. One or more non-transitory processor-readable storage media as in claim 17, wherein the method further comprises applying additional object tracking to the enhanced ROI to confirm a location of a object below said predetermined size threshold.

20. One or more non-transitory processor-readable storage media as in claim 17, wherein the method further comprises compensating for global motion of the imaging device.

21. One or more non-transitory processor-readable storage media as in claim 17, wherein the parameter comprises location.

22. One or more non-transitory processor-readable storage media as in claim 17, wherein the method further comprises, prior to said applying geometric correction, applying an initial object detection process to the off-center ROI.

23. One or more non-transitory processor-readable storage media as in claim 22, wherein said applying geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.

24. One or more non-transitory processor-readable storage media as in claim 23, wherein said object tracking is applied to the reconstructed ROI to refine or confirm, or both, the initial object detection process.