Apparatus and method for spatio-temporal alignment of image sequences

• US 20020094135A1
• Filed: 05/10/2001
• Published: 07/18/2002
• Est. Priority Date: 05/11/2000
• Status: Active Grant

• Current Assignee: Yeda Research and Development Co. Ltd. (Weizmann Institute of Science)
• Original Assignee: Yeda Research and Development Co. Ltd. (Weizmann Institute of Science)
• Inventors: Irani, Michal, Caspi, Yaron
• Granted Patent: US 7,428,345 B2
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A method and system for aligning in at least one of time and space temporally ordered sequences of images comprising receiving a plurality of sequences of images, each sequence including a multiplicity of images, each plurality of sequences defining a space-time volume, and providing an output indication relating at least one point in a space-time volume corresponding to one of the plurality of sequences to at least one point in a space-time volume corresponding to at least another one of the plurality of sequences.

Claims

1. A method for aligning in at least one of time and space temporally ordered sequences of images comprising:

receiving a plurality of sequences of images, each sequence containing a multiplicity of images, each of said plurality of sequences defining a space-time volume; and

providing an output indication relating at least one point in a space-time volume corresponding to one of said plurality of sequences to at least one point in a space-time volume corresponding to at least another one of said plurality of sequences.

2. A method for aligning according to claim 1 and wherein said providing an output indication comprises:

finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment matches spatial-temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.

3. A method for aligning according to claim 1 and wherein said providing an output indication comprises:

finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment correlates spatial-temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.

4. A method for aligning according to claim 1 and wherein said providing an output indication comprises:

finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment matches at least temporal variations occurring at plural locations in said first space-time volume to at least temporal variations occurring at plural locations in said second space-time volume.

5. A method for aligning according to claim 1 and wherein said providing an output indication comprises:

finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment correlates at least temporal variations occurring at plural locations in said first space-time volume to at least temporal variations occurring at plural locations in said second space-time volume.

6. A method according to claim 2 wherein said step of finding at least one global alignment in at least one of time and space comprises finding at least one global alignment in time.

7. A method according to claim 2 wherein said step of finding at least one global alignment in at least one of time and space comprises finding at least one global alignment in space.

13. A method according to claim 1 wherein said images comprise color images.

8. A system for aligning in at least one of time and space temporally ordered sequences of images comprising:

a space-time volume generator, receiving a plurality of sequences of images, each sequence containing a multiplicity of images, each of said plurality of sequences defining a space-time volume; and

an aligner, providing an output indication relating at least one point in a space-time volume corresponding to one of said plurality of sequences to at least one point in a space-time volume corresponding to at least another one of said plurality of sequences.

9. A system for aligning according to claim 8 and wherein said aligner comprises a match-based global alignment finder, finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment matches spatial-temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.

10. A system for aligning according to claim 8 and wherein said aligner comprises a correlation-based global alignment finder, finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment correlates spatial-temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.

11. A system for aligning according to claim 8 and wherein said aligner comprises a match-based global alignment finder, finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment matches at least temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.

12. A system for aligning according to claim 8 and wherein said aligner comprises a correlation-based global alignment finder, finding at least one global alignment in at least one of time and space between at least a portion of a first space-time volume corresponding to one of said plurality of sequences and at least a portion of a second space-time volume corresponding to at least another one of said plurality of sequences, which said at least one global alignment correlates at least temporal variations occurring at plural locations in said first space-time volume to spatial-temporal variations occurring at plural locations in said second space-time volume.