Self-calibrating, digital, large format camera with single or multiple detector arrays and single or multiple optical systems

• US 7,009,638 B2
• Filed: 05/06/2002
• Issued: 03/07/2006
• Est. Priority Date: 05/04/2001
• Status: Active Grant

• Current Assignee: Vexcel Imaging US Inc. (Microsoft Corporation)
• Original Assignee: Vexcel Imaging GmbH (Microsoft Corporation)
• Inventors: Ponticelli, Martin, Leberl, Franz W., Gruber, Michael A.
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Villecco, John M.
• Application Number:
  US10/140,532
• Publication Number:
  US 20020163582A1
• Time in Patent Office: 1,401 Days
• Field of Search: 348/117, 348/123, 348/144, 348/145, 348/146, 348/147, 348/159, 348/218.1, 348/262, 348/263, 348/264, 348/265, 348/267, 348/268, 348/239, 350/450, 382/294, 382/295, 382/297, 382/299
• US Class Current: 348/218.1
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

Large format, digital camera systems (10, 100, 150, 250, 310) expose single detector arrays 20 with multiple lens systems (12, 14, 16, 18) or multiple detector arrays (104, 106, 108, 110, 112, 114, 116, 118, 120, 152, 162, 172, 182, 252, 262, 272, 282, 322, 324) with one or more single lens systems (156, 166, 176, 186) to acquire sub-images of overlapping sub-areas of large area objects. The sub-images are stitched together to form a large format, digital, macro-image (80, 230″, 236″, 238″, 240″), which can be colored. Dampened camera carrier (400) and accelerometer (404) signals with double-rate digital signal processing (306, 308) are used.

Claims

1. A method of acquiring a large area image of a large area object, comprising:

acquiring a plurality of small area electronic images of divers portions of a large area object by;

focusing multiple, overlapping divers portions of the large area object onto an array of photosensitive elements through multiple, individual lens systems, each of which is fixed in immoveable relation with respect to the array of photosensitive elements and which are positioned in still, non-moving relation to the large area object, in such a manner that each of the overlapping divers portions of the large area object is focused onto the array of photosensitive elements by a respective one of the lens systems;

exposing the array of photosensitive elements sequentially to each of the overlapping, divers portions of the large area object without moving the optical paths from the large area object to the array of photosensitive elements;

reading the electronic signals out of the photosensitive elements for each such exposure before making the next sequential exposure;

forming a mosaic of the small area electronic images by merging the small area electronic images together in such a manner that overlapping areas of adjacent small area electronic images coincide or register with each other; and

converting the mosaic of small area electronic images to the large area image of the large area object.

2. A method of acquiring a large area image of a large area object, comprising:

focusing a first set of a plurality of non-overlapping divers portions of the large area object simultaneously onto a first set of respective small format arrays of photosensitive elements with a first lens system to acquire a first set of non-overlapping small area electronic images of divers portions of the large area object;

focusing a second set of a plurality of divers portions of the large area object, which do not overlap each other, but which do include and span portions of the large area object that extend between the non-overlapping divers portions of the large area object in the first set of such divers portions as well as overlapping marginal edge areas of such divers portions of the large area object in the first set of such divers portions, simultaneously onto a second set of respective small area format arrays of photosensitive elements with a second lens system to acquire a second set of non-overlapping small area electronic images of divers portions of the large area object;

forming a mosaic comprising the first set of non-overlapping small area electronic images and the second set of non-overlapping small area electronic images positioned together in such a manner that the second set of non-overlapping small area electronic images extend between and overlap marginal edge areas of respective ones of the first set of non-overlapping small area electronic images; and

converting the mosaic to the large area image of the large area object.

3. The method of claim 2, including focusing a third set of a plurality of divers portions of the large area object, which do not overlap each other, but which do include and span additional portions of the large area object that extend between the non-overlapping divers portions of the large area object in the first set of such divers portions as well as overlapping marginal edge areas of such divers portions of the large area object in the first set of such divers portions, simultaneously onto a third set of respective small area format arrays of photosensitive elements with a third lens system.

4. The method of claim 3, including focusing a fourth set of a plurality of divers portions of the large area object, which do not overlap each other, but which do include and span portions of the large area object that extend between the non-overlapping divers portions of the large area object in the second set of such divers portions as well as overlapping marginal edge areas of such divers portions of the large area object in the second set of such divers portions, simultaneously onto a fourth set of respective small area format arrays of photosensitive element with a fourth lens system.

5. The method of claim 4, including synchronizing focusing of the first, second, third, and fourth sets of divers portions of the large area object onto the respective first, second, third, and fourth sets of small area format arrays of photosensitive elements with the respective first, second, third, and fourth lens systems.

6. The method of claim 4, including focusing sequentially the first, second, third, and fourth sets of divers portions of the large area object onto the respective first, second, third, and fourth sets of small area format arrays of photosensitive elements with the respective first, second, third, and fourth lens systems.

7. The method of claim 3, including synchronizing focusing of the first, second, and third sets of divers portions of the large area object onto the respective first, second, and third sets of small area format arrays of photosensitive elements with the respective first, second, and third lens systems.

8. The method of claim 3, including focusing sequentially the first, second, and third sets of divers portions of the large area object onto the respective first, second, and third sets of small area format arrays of photosensitive elements with the respective first, second, and third lens systems.

9. The method of claim 2, including synchronizing focusing of the first set of divers portions of the large area object onto the first set of small area format arrays of photosensitive elements with the first lens system with focusing of the second set of divers portions of the large area object onto the second set of small area format arrays of photosensitive elements with the second lens system.

10. The method of claim 2, including focusing the first set of divers portions of the large area object onto the first set of small area format arrays of photosensitive elements with the first lens system simultaneously with focusing the second set of divers portions of the large area object onto the second set of small area format arrays of photosensitive elements with the second lens system.

11. The method of claim 2, including focusing the second set of divers portions of the large area object onto the second set of small area format arrays of photosensitive elements with the second lens system after focusing the first set of divers portions of the large area object onto the first set of small area format arrays of photosensitive elements with the first lens system.

12. The method of claim 11, including:

setting the first and second lens systems to have their respective optic axes parallel to each other;

moving the first and second lens systems in relation to the large area object;

actuating a shutter associated with the first lens system to focus the first set of divers portions of the large area object onto the first set of small area format arrays with the optic axis of the first lens system extending through an object center of the large area object; and

after the first and second lens systems have moved enough for the optic axis of the second lens system to extend through the same object center of the large area object, actuating a shutter associated with the second lens system to focus the second set of divers portions of the large area object onto the second set of small area format arrays.

13. A method of acquiring a large area image of a large area object, comprising:

acquiring a plurality of small area electronic images of different divers portions of the large area object by focusing the different divers portions of the large area object on different two dimensional arrays of photosensitive elements, wherein respective individual photosensitive elements in the arrays produce electronic signals that are indicative of intensity of panchromatic light energy from pixels of the respective divers portions of the large area object, whereby all of such electronic signals from the photosensitive elements in each of the arrays of photosensitive elements form a pixel array of electronic signals, which together are indicative of spatial distribution of panchromatic light energy from the respective divers portion of the large object area that is focused on such array of photosensitive elements;

converting the electronic signals of each of said pixel arrays to respective pixel values;

establishing a master coordinate system based on a first one of said pixel arrays to define spatial relationships among the individual pixels in the first pixel array;

establishing an associate coordinate system for each of the other pixel arrays to define spatial relationships among the individual pixels in each of the other pixel arrays;

recording the respective pixel values for each of the pixel arrays in such a manner that the light energy intensities and spatial relationships of said pixel arrays are preserved in accord with said respective coordinate systems;

scanning all of the divers portions of the large area object with one or more linear arrays of photosensitive elements to detect colors of the large area object image;

recording the colors in a color pixel array with a color coordinate system in which individual color pixels in the color pixel array have respective color coordinates that comprise the color coordinate system;

adjusting the associate coordinate system coordinates of pixels in arrays that have associate coordinate systems to master coordinate system coordinates to stitch said other pixel arrays to said first pixel array in a manner that preserves light energy intensities and spatial relationships of pixels within the pixel arrays as well as between the pixel arrays to be indicative of the light energy intensities and spatial relationships of distinct features in the large area object;

adjusting the color coordinates of color pixels containing color pixel values representative of the distinct features to coincide with the coordinates of the pixels containing the pixel values (gray values) of those respective distinct features in the master coordinate system; and

adjusting the color coordinates of other color pixels commensurate with the adjustment of the color coordinates of the pixels containing the color pixel values representative of the distinct features to conform such color coordinates to the master coordinate system.

14. The method of claim 13, including using the color pixel values for pixels at the coordinates as adjusted in the master coordinate system to produce the visual large area image of the large area object in color.

15. Camera apparatus for producing a large area image of a large area object, comprising:

a plurality of camera subsets, wherein each camera subset comprises a plurality of small format arrays of photosensitive elements and a lens system for focusing different, non-overlapping, small area portions of the large area object onto the respective small format arrays of photosensitive elements in such camera subset, but wherein small area portions of the large area object focused on small format arrays of respective different camera subsets cover areas between, and overlap, the small area portions of the large area objects focused on the small format arrays of other camera subsets;

a shutter on each of the camera subsets;

a shutter control system that synchronizes actuation of the shutters of the camera subsets; and

an electronic signal processing circuit that is capable of acquiring pixel data of the small area portions of the large area object from the photosensitive elements of each of the small format arrays and of storing the pixel data according to their relative positions in a coordinate system for each small format array.

16. The camera apparatus of claim 15, including data processing means for combining the respective pixel data from the respective small area arrays together in a master coordinate system based on matching pixels of data from distinct features of the large area object that are in overlapping areas of the different small area portions of the large area object.

17. The camera system of claim 16, including:

an additional camera subset comprising a single array of photosensitive elements set up to detect only near infrared radiation and a lens system for focusing all of the small area portions of the large area object onto the additional single array of photosensitive elements to detect near infrared radiation from all of the small area portions with said additional single array;

a shutter on said additional camera subset, which is also controlled by the shutter control system for synchronization with the actuation of the shutters of the other camera subsets;

wherein the electronic signal processing circuit is also capable of acquiring near infrared pixel portions of the large area object from said single array and of storing the near infrared pixel data according to their relative spatial positions in a coordinate system for said additional array; and

wherein the data processing means is also for combining the near infrared pixel data from said additional array together with the pixel data from said small area arrays in a master coordinate system based on matching pixels of data from the distinct features of the large area object detected by said additional array with pixels of data from said distinct features detected by said small area arrays.

18. The camera system of claim 16, including:

three additional camera subsets, each of which additional camera subsets comprises a single array of photosensitive elements, the single array of one of the three additional camera subsets being set up to detect only a first primary color, the single array of one of the three additional camera subsets being set up to detect only a second primary color, and the single array of the three additional camera subsets being set up to detect only a third primary color, and each of which additional camera subsets has a lens system for focusing all of the small area portions of the large area object onto each of the three additional arrays of photosensitive elements to detect each of the three primary colors radiation from all of the small area portions;

a shutter on each of the three additional camera subsets, which shutters are also controlled by the shutter control system for synchronization with the activation of the shutters of the other camera subsets;

wherein the electronic signal processing circuit is also capable of acquiring primary color pixel data of the small area portions of the large area object from said additional three arrays and of storing the primary color pixel data according to their relative spatial positions in a coordinate system for each of the three additional arrays; and

wherein the data processing means is also for combining the primary color pixel data from said three additional arrays together with the pixel data from said small area arrays in a master coordinate system based on matching pixels of data from the distinct features of the large area object detected by said three additional arrays with pixels of data from said distinct features detected by said small area arrays.

19. The camera system of claim 16, including:

an additional linear array of photosensitive elements set up to detect color radiation through a lens system capable of focusing enough width of the large area object to include all of the small area portions of the large area object so that scanning the linear array over the large area object enables the linear array to detect color radiation from all of the small area portions;

wherein the electronic signal processing circuit is also capable of acquiring color pixel data of the small area portions of the large area object from said linear array and of storing color pixel data according to their relative spatial positions; and

wherein the data processing means is also for combining the primary color pixel data from the linear arrays together with the pixel data from said small area arrays in a master coordinate system based on matching pixels of data from the distinct features of the large area object detected by said linear arrays with pixels of data from said distinct features detected by said small area arrays.

20. The camera system of claim 19, including an additional camera subset comprising the lens system for focusing the color radiation onto the linear array of photosensitive elements.

21. The camera system of claim 20, wherein the linear array detects primary color radiation from all of the small area portions.

22. The camera apparatus of claim 15, including photosensitive elements that detect primary colors.