Image processing system and method

• US 7,730,406 B2
• Filed: 10/20/2004
• Issued: 06/01/2010
• Est. Priority Date: 10/20/2004
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chen, Mei
• Primary Examiner(s): Campbell; Joshua D
• Assistant Examiner(s): ALVESTEFFER, STEPHEN D
• Application Number:
  US10/969,569
• Publication Number:
  US 20060083440A1
• Time in Patent Office: 2,050 Days
• Field of Search: 715/723, 382/284, 382/254
• US Class Current: 715/723
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A machine-implemented image processing method includes segmenting a scene into a background scene portion and a foreground scene portion, forming a mosaic for the background scene portion, enhancing the foreground scene portion, and constructing a composite of the background mosaic and the enhanced foreground.

Claims

1. A machine-implemented method of processing a sequence of input images of a scene, comprising:

determining a respective set of motion vectors for each of one or more pairs of the input images;

segmenting each of two or more of the input images into at least one respective background scene portion and at least one respective foreground scene portion based on respective ones of the motion vectors;

stitching together ones of the respective background scene portions segmented from different ones of the input images to form background mosaic image data;

enhancing at least one of the foreground scene portions segmented from the input images to form enhanced foreground scene image data, wherein the enhancing comprises increasing image resolution of the at least one foreground scene portion; and

generating an output image, wherein the generating comprises combining the background mosaic image data and the enhanced foreground scene image data.

2. The method of claim 1, wherein the increasing comprises computing the enhanced foreground scene image data based on image value contributions from corresponding ones of the foreground scene portions segmented from respective ones of the input images.

3. A machine-implemented method of processing a sequence of input images of a scene, comprising:

determining a respective set of motion vectors for each of one or more pairs of the input images;

segmenting each of two or more of the input images into at least one respective background scene portion and at least one respective foreground scene portion based on respective ones of the motion vectors;

stitching together ones of the respective background scene portions segmented from different ones of the input images to form background mosaic image data;

enhancing at least one of the foreground scene portions segmented from the input images to form enhanced foreground scene image data, wherein the enhancing comprises determining image values corresponding to the at least one foreground scene portion at an increased image resolution level; and

generating an output image, wherein the generating comprises combining the background mosaic image data and the enhanced foreground scene image data, wherein the combining comprises up-sampling the background mosaic image data to the increased image resolution level and merging the up-sampled background mosaic image data with the enhanced foreground scene image data.

4. The method of claim 3, wherein the combining comprises blending at least one edge in the enhanced foreground scene image data with the up-sampled background mosaic image data.

5. The method of claim 3, wherein the enhancing comprises enhancing multiple occurrences of at least one of the foreground scene portions to form the enhanced foreground scene image data, and the combining comprises placing each of the occurrences of the at least one enhanced foreground scene portion into the up-sampled background mosaic image data.

6. An apparatus for processing a sequence of input images of a scene, comprising:

a motion estimation module that determines a respective set of motion vectors for each of one or more pairs of the input images;

a scene segmentation module that segments each of two or more of the input images into at least one respective background scene portion and at least one respective foreground scene portion based on respective ones of the motion vectors received from the motion estimation module;

a mosaic module coupled to the scene segmentation module and operable to stitch together ones of the respective background scene portions segmented from different ones of the input images to form background mosaic image data;

an image enhancement module coupled to the scene segmentation module and operable to enhance at least one of the foreground scene portions segmented from the input images to form enhanced foreground scene image data, wherein the image enhancement module determines image values corresponding to the at least one foreground scene portion at an increased image resolution level; and

a mosaic composition module coupled to the mosaic module and to the image enhancement module and operable to generate an output image, wherein in generating the output image the mosaic composition module combines the background mosaic image data and the enhanced foreground scene image data.

7. The apparatus of claim 6, wherein the mosaic composition module up-samples the background mosaic image data to the increased image resolution level and merges the up-sampled background mosaic image data with the enhanced foreground scene image data.

8. The apparatus of claim 7 wherein the image enhancement module enhances multiple occurrences of at least one of the foreground scene portions to form the enhanced foreground scene image data, and the mosaic composition module dynamically places each of the occurrences of the at least one enhanced foreground scene portions into the up-sampled background mosaic image data in succession to produce an image sequence of the at least one respective foreground scene portions moving across the background mosaic image.

9. A machine-readable medium that provides instructions that, when executed by a machine, cause the machine to perform operations comprising:

determining a respective set of motion vectors for each of one or more pairs of the input images;

segmenting each of two or more of the input images into at least one respective background scene portion and at least one respective foreground scene portion based on respective ones of the motion vectors;

stitching together ones of the respective background scene portions segmented from different ones of the input images to form background mosaic image data;

enhancing at least one of the foreground scene portions segmented from the input images to form enhanced foreground scene image data,

wherein in performing the enhancing the instructions cause the machine to perform operations comprising determining image values corresponding to the at least one foreground scene portion at an increased image resolution level; and

generating an output image, wherein the generating comprises combining the background mosaic image data and the enhanced foreground scene image data.

10. The machine-readable medium of claim 9, wherein in performing the combining the instructions cause the machine to perform operations comprising up-sampling the background mosaic image data to the increased image resolution level and merging the up-sampled background mosaic image data with the enhanced foreground scene image data.