Image mosaicing system and method adapted to mass-market hand-held digital cameras

• US 6,834,128 B1
• Filed: 06/16/2000
• Issued: 12/21/2004
• Est. Priority Date: 06/16/2000
• Status: Expired due to Term

• Current Assignee: ASML Netherlands BV (ASML Holding NV), Carl Zeiss AG (Carl-Zeiss-Stiftung), Tarsium B.V.
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Altunbasak, Yucel, Drukarev, Alexander I.
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Kassa, Yosef
• Application Number:
  US09/596,133
• Time in Patent Office: 1,649 Days
• Field of Search: 382/282, 382/284, 382/289, 382/287, 382/298, 382/291, 382/293, 382/294, 382/295, 382/302, 358/450, 358/452, 358/453, 358/465, 348/49, 348/50, 348/64, 348/194, 348/155, 348/207.1, 348/208.4
• US Class Current: 382/284
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An imaging system includes a camera for capturing image frames, a frame pre-screen unit for pre-screening the captured image frames, a frame store unit for storing the pre-screened frames, and a refined frame mosaicing unit for generating a refined mosaic from the pre-screened image frames that are stored in the frame store unit. The frame pre-screen unit may include a coarse align unit for generating a coarse image mosaic from the pre-screened image frames, and a display for displaying the coarse image mosaic. In an embodiment, the coarse image mosaic is generated and displayed in real-time relative to the image frames that are captured by the camera. In an embodiment of the imaging system, the display is located within the camera and in another embodiment, the frame pre-screen unit and the frame store unit are located within the camera.

Claims

1. An imaging system comprising:

a camera that includes a lens and detector array for capturing image frames;

means for pre-screening said image frames that are captured by said camera and for outputting pre-screened image frames;

means for storing said pre-screened image frames that arm output from said pre-screening means; and

means for generating a refined mosaic from said pre-screened image frames that are stored in said storage means;

wherein said pre-screening means and said storing means are located within said camera wherein said pre-screening means includes;

means for coarse motion tracking of said image frames that are captured by said camera, said coarse motion tacking means generating coarse motion vectors for said image frames; and

means for rejecting a first image frame if said coarse motion vector for said first image frame does not exceed a motion threshold and for providing said first image frame to said storing means if said coarse motion vector does exceed said motion thresholds.

2. The imaging system of claim 1 wherein said means for pre-screening includes:

means for generating a coarse image mosaic from said pre-screened image frames; and

further including a display means for displaying said coarse image mosaic.

3. The imaging system of claim 2 wherein said coarse image mosaic is generated and displayed in real-time relative to said image frames that are captured by said camera.

4. The imaging system of claim 2 wherein said display means includes a display screen located within said camera.

5. The imaging system of claim 1 wherein said means for generating a refined mosaic includes a support computer system that supports an image mosaicing application.

6. A hand-held digital camera comprising:

a lens and detector array for capturing image frames;

means for prescreening said image frames that are captured by said detector array and for outputting pre-screened image frames;

means for displaying a coarse image mosaic of said pre-screened image frames;

means for storing said pre-screened image frames that are output from said pre-screening means; and

means for transferring said pre-screened image frames from said storing means to a support system that generates a refined image mosaic from said pre-screened image frames wherein said means for pre-screening said image frames includes;

means for coarse motion tracking of said image frames that are captured by said detector array, said coarse motion tacking means generating coarse motion vectors for said image frames;

means for rejecting a first image frame if said coarse motion vector for said first image frame does not exceed a motion threshold and for providing said first image frame to said storing means if said coarse motion vector does exceed said motion threshold; and

means for generating said coarse image mosaic from image frames having coarse motion vectors that exceed said motion threshold.

7. The hand-held digital camera of claim 6 wherein said pre-screening means generates said coarse image mosaic in real-time.

8. The hand-held digital camera of claim 6 further including a support element that locates said detector array a fixed distance away from an object that is to be imaged.

9. An imaging system comprising:

a camera including;

a lens and detector array for capturing image frames;

a support computer system connected to said camera by a real-time communications link, said support computer system including;

means for pre-screening said image frames that are captured by said camera and for outputting pre-screened image frames;

means for generating a coarse image mosaic from said pre-screened image frames;

means for storing said pre-screened image frames that are output from said pre-screening means;

means for generating a refined image mosaic from said pre-screened image frames that are stored in said storage means; and

a display for displaying said coarse image mosaic and said refined image mosaic;

wherein said pre-screening means within said support computer system includes;

means for coarse motion tracking of said image frames that are captured by said camera said coarse motion tracking means generating coarse motion vectors for said image frames; and

means for rejecting a first image frame if said coarse motion vector for said first image frame does not exceed a notion threshold and for providing said first image frame to said storing means if said coarse motion vector does exceed said motion threshold.

10. The imaging system of claim 9 wherein said camera includes a display for displaying said coarse image mosaic in real-time.

11. The imaging system of claim 9 wherein said coarse image mosaic is generated and displayed in real-time on said support computer system display.

12. An image mosaicing method comprising the steps of:

capturing image frames;

making a coarse motion determination for a first image frame relative to a reference frame, said coarse motion determination being represented as a coarse motion vector;

comparing said coarse motion vector for said first image frame to a motion threshold;

rejecting said first image frame if said coarse motion vector for said first image frame does not exceed said motion threshold; and

storing said first image frame if said coarse motion vector for said first image frame exceeds said motion threshold.

13. The image mosaicing method of claim 12 further including a step of aligning image frames having coarse motion vectors that exceed said motion threshold to crate a coarse image mosaic.

14. The image mosaicing method of claim 13 further including a step of displaying said coarse image mosaic in real-time.

15. The image mosaicing method of claim 14 further including a step of displaying said coarse image mosaic in real-time relative to said captured image frames.

16. The image mosaicing method of claim 12 further including a step of generating a refined image mosaic from image frames having coarse motion vectors that exceed said motion threshold.

17. An imaging system comprising:

a camera that includes a lens and detector array for capturing image francs;

means for pre-screening said image frames that are captured by said camera and for outputting pre-screened image frames;

means for storing said pre-screened image frames that are output from said pre-screening means; and

means for generating a refined mosaic from said pre-screened image frames that are stored in said storage means;

wherein said pre-screening means includes;

means for coarse motion tracking of said image frames that are captured by said camera, said coarse motion tracking means generating coarse motion vectors for said image frames; and

means for rejecting a first image frame if said coarse motion vector for said first image frame does not exceed a motion threshold and for providing said first image frame to said storing means if said coarse motion vector does exceed said motion threshold.

18. A hand-held digital camera comprising:

a lens and detector array for capturing image frames;

means for pre-screening said image frames that are captured by said detector array and for outputting pre-screened image frames;

means for displaying a coarse image mosaic of said pre-screened image frames;

means for storing said pre-screened image frames that are output from said pre-screening means; and

means for transferring said prescreened image frames from said storing means to a support system that generates a refined image mosaic from said pre-screened image frames;

wherein said means for pre-screening said image frames includes;

means for coarse motion tracking of said image frames that are captured by said detector array, said coarse tracking means generating coarse motion vectors for said image frames;

means for rejecting a first image frame if said coarse motion vector for said first image frame does not exceed a motion threshold and for providing said first image frame to said storing means if said coarse motion vector does exceed said motion threshold; and

means for generating said coarse image mosaic from image frames having coarse motion vectors that exceed said motion threshold.

19. An image mosaicing method comprising the steps of:

capturing image frames;

making a coarse motion determination for a first image frame relative to a reference frame, said coarse motion determination being represented as a coarse motion vector;

comparing said coarse motion vector for said first image frame to a motion threshold;

injecting said first image frame if said coarse motion vector for said first image frame does not exceed said motion threshold;

storing said first image frame if said coarse motion vector for said first image frame exceeds said motion threshold; and

generating a refined image mosaic from image frames having coarse motion vectors that exceed said motion threshold.