Mosaic based image processing system

• US 6,393,163 B1
• Filed: 12/02/1999
• Issued: 05/21/2002
• Est. Priority Date: 11/14/1994
• Status: Expired due to Term

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Hansen, Michael W., Irani, Michal, Hsu, Stephen Charles, Anandan, Padmanabhan, Burt, Peter J.
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): Miller, Martin
• Application Number:
  US09/453,749
• Time in Patent Office: 901 Days
• Field of Search: 382/294, 382/284, 345/629, 345/635, 358/450
• US Class Current: 382/294
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8047 using transform coding

Abstract

A system for automatically generating a mosaic from a plurality of input images. The system sequentially executes an image alignment process, an editing process, and a combining process such that, from a sequence of images, the system automatically produces a seamless mosaic for various applications. Illustrative applications for the mosaic include: (1) a mosaic based display system including an image printing system, (2) a surveillance system and (3) a mosaic based compression system. The mosaic based display system permits a system user to display, manipulate and alter a mosaic. The mosaic based compression system exploits the temporal and spatial redundancy in image sequences and efficiently compresses the image information. The compression system can be used for compressing image information for storage in a storage device or can be used for compressing image information for transmission through a band-limited transmission channel. The surveillance system combines the mosaic construction system, the compression system and the display system into a comprehensive system for remote motion detection.

Claims

1. A method for automatically generating a display mosaic from a sequence of images, comprising:

automatically aligning image information in each of said images in said sequence of images with common image information in at least one other image in said sequence of images, where said alignment is performed with respect to an arbitrary reference coordinate system;

composing said aligned images to produce a first mosaic;

modifying said first mosaic to product said display mosaic; and

displaying said display mosaic.

2. The method of claim 1, further comprising inserting ancillary image information into the display mosaic.

3. The method of claim 2, wherein said providing ancillary image information further comprises at least one of highlighting of specific areas of the display mosaic, color enhancing of specific areas of the display mosaic, outlining latest images combined in the display mosaic, and fading older images combined in the display mosaic.

4. The method of claim 1, wherein prior to displaying said display mosaic, said method further comprises:

tiling the display mosaic to form a series of sub-mosaics; and

storing said series of sub-mosaics.

5. The method of claim 4, further comprising:

combining with at least one sub-mosaic, at least one second sub-mosaic and one or more input images to recreate any previously displayed display mosaic.

6. The mosaic of claim 1, wherein said modifying step further comprises stabilizing objects in said display mosaic by freezing said objects to a reference coordinate system.

7. The method of claim 1, wherein said modifying step further comprises stabilizing objects in said display mosaic by controlling motion.

8. The method of claim 6, wherein said stabilizing step further comprises stabilizing background objects of said display mosaic, wherein foreground objects appear to move relative to the stabilized background.

9. The method of claim 6, wherein said stabilizing step further comprises stabilizing foreground objects of said display mosaic, wherein background objects appear to freely move relative to the stabilized foreground.

10. The method of claim 1, wherein said displaying step comprises enabling said display mosaic to be viewed on a viewport.

11. The method of claim 10, further comprising providing at least one of scrolling, zooming-in, and zooming-out functions of said display mosaic on said viewport.

12. The method of claim 10, further comprising enabling image editing and enhancement of said display mosaic on said viewport.

13. The method of claim 1, further comprising generating overlays for the display mosaic, wherein said overlays comprise defining a position of an latest image added to the display mosaic.

14. The method of claim 1, further comprising generating overlays for the display mosaic, wherein said overlays comprise outlining motion of foreground objects within the display mosaic, and indicating areas of the display mosaic containing untimely information.

15. The method of claim 1, further comprising generating overlays for the display mosaic, wherein said overlays comprise indicating areas of the display mosaic containing inaccurate and untimely information.

16. The method of claim 1, wherein said displaying step further comprises enabling printing of said display mosaic on a hard copy medium.

17. The method of claim 16, wherein said printing step further comprises adjusting resolution of a printed hard copy higher than a resolution of the display mosaic.