OBJECT DETECTION FROM IMAGE PROFILES

• US 20120206618A1
• Filed: 02/15/2011
• Published: 08/16/2012
• Est. Priority Date: 02/15/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia, Inc.)
• Inventors: Albu, Felix, Murray, Larry, Stec, Piotr, Raducan, Ilariu
• Granted Patent: US 8,587,666 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A measure of frame-to-frame rotation is determined. A global XY alignment of a pair of image frames is performed. At least one section of each of the X and Y integral projection vectors is determined, where aligned global vectors demonstrate a significant localized difference. Based on X and Y locations of the at least one section of the X and Y integral projection vectors, location, relative velocity and/or approximate area of at least one moving object within the sequence of image frames is/are determined.

Claims

1. Within an image acquisition device, a method of determining moving objects in integral projection vectors, comprising:

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global XY alignment of a first frame and second, following frame of said image sequence;

determining at least one section of each of the X and Y integral projection vectors where aligned global vectors demonstrate a significant localized difference; and

based on X and Y locations of said at least one section of the X and Y integral projection vectors, determining location, relative velocity or approximate area, or combinations thereof, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined location, relative velocity or approximate area, or combinations thereof, of the at least one moving object.

2. The method of claim 1, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

3. The method of claim 1, further comprising determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is further based also on the determined global rotation.

4. The method of claim 1, further comprising determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

5. The method of claim 1, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

6. The method of claim 1, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.

7. Within an image acquisition device, a method of determining moving objects in integral projection vectors, comprising:

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global alignment of a first and second, following frame of said image sequence;

determining at least one section of an integral projection vector where the aligned global vectors demonstrate a significant localized difference; and

based on a location of each said at least one section of the integral projection vector, determining an approximate size or relative velocity, or both, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined approximate size or relative velocity, or both, of the at least one moving object.

8. The method of claim 7, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

9. The method of claim 7, further comprising determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is also based on the determined global rotation.

10. The method of claim 7, further comprising determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

11. The method of claim 7, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

12. The method of claim 7, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.

13. An image acquisition device, comprising:

a lens, shutter and image sensor for acquiring digital images;

a processor;

a memory having code embedded therein for programming the processor to perform a method of determining moving objects in integral projection vectors, wherein the method comprises;

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global XY alignment of a first frame and second, following frame of said image sequence;

determining at least one section of each of the X and Y integral projection vectors where aligned global vectors demonstrate a significant localized difference; and

based on X and Y locations of said at least one section of the X and Y integral projection vectors, determining location, relative velocity or approximate area, or combinations thereof, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined location, relative velocity or approximate area, or combinations thereof, of the at least one moving object.

14. The device of claim 13, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

15. The device of claim 13, further comprising determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is further based also on the determined global rotation.

16. The device of claim 13, further comprising determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

17. The device of claim 13, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

18. The device of claim 13, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.

19. An image acquisition device, comprising:

a lens, shutter and image sensor for acquiring digital images;

a processor;

a memory having code embedded therein for programming the processor to perform a method of determining moving objects in integral projection vectors, wherein the method comprises;

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global alignment of a first and second, following frame of said image sequence;

determining at least one section of an integral projection vector where the aligned global vectors demonstrate a significant localized difference; and

based on a location of each said at least one section of the integral projection vector, determining an approximate size or relative velocity, or both, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined approximate size or relative velocity, or both, of the at least one moving object.

20. The device of claim 19, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

21. The device of claim 19, further comprising determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is also based on the determined global rotation.

22. The device of claim 19, further comprising determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

23. The device of claim 19, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

24. The device of claim 19, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.

25. One or more non-transitory processor-readable media having code embedded therein for programming the processor to perform a method of determining moving objects in integral projection vectors, wherein the method comprises:

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global XY alignment of a first frame and second, following frame of said image sequence;

determining at least one section of each of the X and Y integral projection vectors where aligned global vectors demonstrate a significant localized difference; and

based on X and Y locations of said at least one section of the X and Y integral projection vectors, determining location, relative velocity or approximate area, or combinations thereof, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined location, relative velocity or approximate area, or combinations thereof, of the at least one moving object.

26. The one or more non-transitory processor-readable media of claim 25, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

27. The one or more non-transitory processor-readable media of claim 25, wherein the method further comprises determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is further based also on the determined global rotation.

28. The one or more non-transitory processor-readable media of claim 25, wherein the method further comprises determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

29. The one or more non-transitory processor-readable media of claim 25, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

30. The one or more non-transitory processor-readable media of claim 25, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.

31. One or more non-transitory processor-readable media having code embedded therein for programming the processor to perform a method of determining moving objects in integral projection vectors, wherein the method comprises:

acquiring a sequence of image frames;

determining a measure of frame-to-frame rotation, including;

performing a global alignment of a first and second, following frame of said image sequence;

determining at least one section of an integral projection vector where the aligned global vectors demonstrate a significant localized difference; and

based on a location of each said at least one section of the integral projection vector, determining an approximate size or relative velocity, or both, of at least one moving object within the sequence of image frames; and

performing a further action based on the determined approximate size or relative velocity, or both, of the at least one moving object.

32. The one or more non-transitory processor-readable media of claim 31, wherein the further action comprises joining adjacent frames of a panorama image;

predicting a location of a tracked object in a subsequent frame;

providing an image with reduced blur than at least one of the first and second frames;

or providing a more stable video sequence by removing one or more undesired motion artifacts, or combinations thereof.

33. The one or more non-transitory processor-readable media of claim 31, further comprising determining delta-X and delta-Y differences between the at least two matching corner regions;

and determining a global rotation based on said delta-X and delta-Y differences, and wherein the further action is also based on the determined global rotation.

34. The one or more non-transitory processor-readable media of claim 31, further comprising determining horizontal and vertical integral projection vector gradients for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

normalizing said integral projection vector gradients; and

determining locations of primary maximum and minimum peaks of said integral projection vector gradients; and

determining a global rotation based on normalized distances between the primary maximum and minimum peaks, and wherein the further action is also based on said determined global rotation.

35. The one or more non-transitory processor-readable media of claim 31, further comprising determining horizontal and vertical integral projection vectors for each of the same or a different first frame and the same or a different second, following, image frame in said sequence;

successively convoluting a first integral projection vector with a variable length vector;

convoluting a second integral projection vector with a fixed length vector; and

determining an absolute sum of differences between filtered vectors based on the convoluting of the first and second integral projection vectors; and

determining a global rotation based on a length of a variable vector N that leads to a minimum value of the absolute sum of the differences, and wherein the further action is also based on said determined global rotation.

36. The one or more non-transitory processor-readable media of claim 31, further comprising performing a global XY alignment of the same or a different first frame and the same or a different second, following frame of said sequence of image frames;

determining local XY alignments in at least two matching corner regions of the aligned image pair; and

based on the differences between said local XY alignments, determining a global rotation of the second image frame relative to the first; and

wherein the further action is also based on the determined global rotation.