In camera implementation of selecting and stitching frames for panoramic imagery
First Claim
1. A method for selecting frames of an image sequence for panoramic stitching that is performed by a hardware processor, comprising:
- for at least some chosen frames of the image sequence, the hardware processor executes software code that performs actions, including;
evaluating frame overlap constraints between a current chosen frame and a previous chosen frame;
evaluating frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame;
selectively saving the current chosen frame as being a qualifying frame if the frame overlap constraints and the frame motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and
outputting the qualifying frames.
3 Assignments
0 Petitions
Accused Products
Abstract
A system, method, and computer program product for selecting qualifying frames from an image sequence for use in subsequent stitching into a composite panoramic image are disclosed. Incoming frames from any source may be cropped and downscaled prior to evaluation against qualifying criteria relating to image overlap and local motion. Qualifying images are saved and/or output. The resulting panoramic image generally uses fewer qualifying images and appears smoother and has fewer artifacts than those of the prior art. The qualifying criterion for image overlap is a predetermined overlap margin or percentage between a current image and a previous image from the sequence. The qualifying criterion for image motion includes a maximum amount of local motion, often due to passing objects. The embodiments may process incoming images in real time or from stored sequences. Problems may trigger user warnings.
69 Citations
25 Claims
-
1. A method for selecting frames of an image sequence for panoramic stitching that is performed by a hardware processor, comprising:
for at least some chosen frames of the image sequence, the hardware processor executes software code that performs actions, including; evaluating frame overlap constraints between a current chosen frame and a previous chosen frame; evaluating frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame; selectively saving the current chosen frame as being a qualifying frame if the frame overlap constraints and the frame motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and outputting the qualifying frames. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
-
21. A camera that, for at least some chosen frames of an image sequence:
-
evaluates frame overlap constraints between a current chosen frame and a previous chosen frame; evaluates frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame; selectively saves the current chosen frame as being a qualifying frame if the frame overlap constraints and the frame motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and outputs the qualifying frames.
-
-
22. A computer program product comprising a machine-readable medium tangibly embodying non-transitory program instructions thereon that, when executed by the machine, cause the machine to, for at least some chosen frames of an image sequence:
-
evaluate frame overlap constraints between a current chosen frame and a previous chosen frame; evaluate frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame; selectively save the current chosen frame as being a qualifying frame if the frame overlap constraints and the frame motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and output the qualifying frames.
-
-
23. An integrated circuit, comprising:
for at least some chosen frames of an image sequence; a first circuit element for evaluating frame overlap constraints between a current chosen frame and a previous chosen frame; a second circuit element for evaluating frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame; a third circuit element for selectively saving the current chosen frame as being a qualifying frame if the frame overlap constraints and the frame local motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and a fourth circuit element for outputting the qualifying frames.
-
24. A system, comprising:
-
software code stored in an non-transitive storage device, wherein execution of the software code by a hardware processor performs actions, including; for at least some chosen frames of an image sequence; means for evaluating frame overlap constraints between a current chosen frame and a previous chosen frame; means for evaluating frame motion constraints between the current and previous chosen frames, wherein the evaluation includes generating a histogram comprising a plurality of motion vectors for the current chosen frame with respect to the previous chosen frame; means for selectively saving a current frame as being a qualifying frame if the frame overlap constraints and the frame motion constraints are met, wherein a motion vector corresponding to the largest bin the in the histogram is employed in determining if the frame overlap constraints and the frame motion constraints are met; and means for outputting the qualifying frames.
-
-
25. A method for processing video frames for generation of a panoramic image with a hardware processor that executes software code that performs actions, the method comprising:
-
capturing a sequence of video frames; selecting a first video frame from the sequence of video frames; storing the first video frame in a first memory location; selecting a second video frame from the sequence of video frames that succeeds the first video frame and that is within a predefined overlap margin with said first video frame; storing the second video frame in a second memory location; estimating local motion between the first selected video frame and the second video frame; selecting a third video frame from the sequence of video frames that succeeds the second video frame and that is within the predefined overlap margin with said first video frame; estimating local motion between the third selected video frame and the first video frame; storing the selected third video frame in the second memory location if the local motion estimated for the selected third video frame is smaller than the local motion of the selected second video frame; repeating selection of the third video frame until the predefined overlap margin is fully consumed, wherein a histogram comprising a plurality of motion vectors is generated for each selected frame with respect to a previously selected frame and wherein a motion vector corresponding to the largest bin in the histogram is employed to determine if the overlap margin is consumed; stitching the image stored in the first memory location and the image stored in the second memory location into a panoramic image having a minimal distortion caused by overlap margin and local motion there between; and storing the minimally distorted panoramic image in the first memory location for output.
-
Specification