Imaging apparatuses, mosaic image compositing methods, video stitching methods and edgemap generation methods

• US 20030194149A1
• Filed: 04/12/2002
• Published: 10/16/2003
• Est. Priority Date: 04/12/2002
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Sobel, Irwin, Hubel, Paul M.
• Granted Patent: US 7,006,706 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

Imaging apparatuses, mosaic image compositing methods, video stitching methods and edgemap generation methods are provided. An exemplary imaging apparatus includes data circuitry configured to provide image data for a plurality of frames of a source and processing circuitry configured to generate mosaic data for the source using the image data, wherein the processing circuitry is configured to convert the image data into a plurality of edgepoints for individual ones of the frames, to identify a plurality of common edgepoints in a first frame and a second frame, to determine a positional representation with respect to the first frame and the second frame using the common edgepoints, and to map image data from one of the first frame and the second frame into the mosaic data using the positional representation to generate a mosaic image of the source using the mosaic data.

Claims

1. An imaging apparatus comprising:

data circuitry configured to provide image data for a plurality of frames of a source; and

processing circuitry configured to generate mosaic data for the source using the image data, wherein the processing circuitry is configured to convert the image data into a plurality of edgepoints for individual ones of the frames, to identify a plurality of common edgepoints in a first frame and a second frame, to determine a positional representation with respect to the first frame and the second frame using the common edgepoints, and to map image data from one of the first frame and the second frame into the mosaic data using the positional representation to generate a mosaic image of the source using the mosaic data.

2. The apparatus of claim 1 wherein the processing circuitry is configured to exclude at least some of the image data from the one of the first frame and the second frame to define new image data not within the mosaic data, and to map the new image data using the positional representation.

3. The apparatus of claim 1 wherein the processing circuitry is configured to determine the positional representation indicative of movement information of an image device between the first frame and the second frame.

4. The apparatus of claim 1 wherein the processing circuitry is configured to equate a mosaic coordinate system and a frame coordinate system to map the image data.

5. The apparatus of claim 1 wherein the processing circuitry is configured to determine a delta of the positional representation responsive to identifying the common edgepoints, to update a back-transform of the positional representation using the delta, and to map the image data using the back-transform after the updating.

6. The apparatus of claim 1 wherein the data circuitry comprises memory.

7. The apparatus of claim 1 wherein the data circuitry comprises an image device.

8. The apparatus of claim 1 wherein the processing circuitry is configured to convert luminance data of the image data into the edgepoints for the individual ones of the frames.

9. The apparatus of claim 1 wherein the processing circuitry is configured to analyze a match quality of the common edgepoints in the first frame and the second frame comprising at least one acceleration information between the frames, edgepoint difference information, and grayscale difference information, and to disregard the common edgepoints responsive to the match quality.

10. The apparatus of claim 1 wherein the processing is configured to refine relative motion of the common edgepoints after the identification of the edgepoints.

11. The apparatus of claim 1 wherein the processing circuitry is configured to identify the common edgepoints comprising locating interesting edgepoints in the first frame, and searching the second frame to locate the interesting edgepoints in the second frame.

12. The apparatus of claim 11 wherein the processing circuitry is configured to determine gradient data of the image data including gradient magnitude data and gradient directional data, and to utilize the gradient magnitude data and the gradient directional data to define the interesting edgepoints in the first frame.

13. The apparatus of claim 1 wherein the processing circuitry is configured to adaptively provide a plurality of frame thresholds corresponding to respective ones of the frames, and the processing circuitry is configured to compare image data of the frames with respective ones of the frame thresholds to convert the image data into the edgepoints.

14. A mosaic image compositing method comprising:

providing image data of a plurality of frames of an image;

converting the image data to edgemap data;

creating a mosaic image using image data from a plurality of the frames; and

coordinating placement of the image data into the mosaic image using the edgemap data.

15. The method of claim 14 wherein the coordinating comprises determining a positional representation using the edgemap data and corresponding to movement of an image device between frames of the image, and placing the image data from a selected one of the frames into the mosaic image using the positional representation.

16. The method of claim 15 wherein the determining the positional representation comprises identifying a plurality of common edgepoints of the edgemap data for the frames.

17. The method of claim 15 wherein the determining the positional representation comprises calculating a delta using luminance data of the image data, and updating a back-transform using the delta.

18. The method of claim 14 wherein the coordinating comprises equating a mosaic coordinate system and a frame coordinate system, and identifying the image data from the selected one of the frames responsive to the equating.

19. The method of claim 14 wherein the converting comprises converting luminance data of the image data.

20. The method of claim 14 wherein the coordinating comprises identifying a bounding box in the mosaic image using the edgemap data, identifying new pixels of the image data using the bounding box, and placing image data corresponding to the new pixels into the mosaic image.

21. A video stitching method comprising:

providing image data for a plurality of frames;

converting the image data into edgemap data;

first identifying a frame subset in a first frame using respective edgemap data of the first frame;

second identifying the frame subset in a second frame using respective edgemap data of the second frame;

providing movement information indicating movement between the first frame and the second frame responsive to the first identifying and the second identifying; and

combining at least some of the image data of the second frame with mosaic data using the movement information.

22. The method of claim 21 further comprising equating a mosaic coordinate system and a frame coordinate system, and wherein the combining occurs after the equating.

23. The method of claim 21 wherein the converting comprises converting luminance data of the image data.

24. The method of claim 23 wherein the converting comprises:

converting the luminance data into gradient data for the respective frames; and

converting the gradient data into the edgemap data comprising a plurality of edgepoints.

25. The method of claim 21 further comprising:

analyzing the frame subset of the first frame and the frame subset of the second frame;

disregarding the second frame responsive to the analyzing; and

third identifying the frame subset in a third frame responsive to the disregarding.

26. The method of claim 25 wherein the analyzing comprises analyzing acceleration information, edgepoint difference information, and grayscale difference information between the first frame and the second frame.

27. The method of claim 21 further comprising refining the second frame subset after the second identifying and before the providing the movement information.

28. The method of claim 21 wherein the second identifying comprises:

identifying a starting location in the second frame using the frame subset of the first frame; and

searching from the starting location.

29. The method of claim 21 wherein the first identifying comprises identifying a plurality of frame subsets in the first frame, and the second identifying comprises identifying the plurality of frame subsets in the second frame.

30. The method of claim 29 further comprising creating a forbidden zone in the first frame using an identified one of frame subsets of the first frame, and wherein the first identifying further comprises searching for another one of the frame subsets in the first frame, and excluding the forbidden search zone during the searching.

31. The method of claim 21 wherein the first identifying comprises identifying an interesting frame subset comprising calculating luminance gradient data, and analyzing directional data of the luminance gradient data to filter at least one other frame subset having insufficient different directional data.

32. The method of claim 21 wherein the first identifying comprises identifying an interesting frame subset comprising calculating luminance gradient data and analyzing a magnitude of the luminance gradient data of a plurality of pixels of the frame subset with the respect to a threshold.

33. The method of claim 21 wherein the first identifying comprises calculating a plurality of different thresholds for respective frames using image data of the respective frames, and comparing image data of the respective frames with respective ones of the thresholds.

34. The method of claim 21 wherein the combining comprises:

generating a delta using the movement information;

updating a back-transform using the delta;

identifying new image data in the second frame not within the first frame; and

positioning the new image data with the mosaic data using the back-transform after the updating.

35. An edgemap generation method comprising:

providing image data for a plurality of frames and corresponding to a plurality of pixels for individual ones of the frames;

defining a plurality of frame thresholds using the image data of the respective frames and wherein the frame thresholds correspond to respective ones of the frames;

individually selecting the frames and the respective frame thresholds for the selected frames;

individually comparing the image data of the selected frame with the respective selected frame threshold; and

identifying selected pixels as comprising edgepoints responsive to the comparing for individual ones of the frames.

36. The method of claim 35 further comprising determining gradient magnitude data of the image data, and wherein the defining comprises defining using the gradient magnitude data.

37. The method of claim 35 wherein the defining comprises:

calculating gradient data using the image data; and

varying the frame thresholds using the gradient data of the respective frames.

38. The method of claim 35 wherein the defining the frame thresholds comprises defining different thresholds corresponding to the respective different image data for the frames.

39. The method of claim 35 further comprising:

filtering the image data; and

calculating gradient data of the image data after the filtering, and wherein the defining comprises defining the frame thresholds using the gradient data.

40. The method of claim 35 wherein the providing comprises providing image data having a plurality of luminance magnitudes corresponding to respective pixels.

41. The method of claim 40 wherein the defining an individual one of the frame thresholds for a respective frame comprises:

arranging respective image data for the respective frame in a plurality of groups corresponding to the respective intensity magnitudes of the image data;

providing a magnitude threshold;

integrating the image data starting at an extreme one of the intensity magnitudes;

comparing the integration to the magnitude threshold;

identifying one of the groups responsive to the comparing;

setting the respective individual frame threshold responsive to the identifying;

comparing image data of individual pixels of the respective frame with the respective individual frame threshold; and

identifying selected pixels as comprising edgepoints responsive to the comparing.

42. The method of claim 41 further comprising repeating the defining for the other frame thresholds.

43. The method of claim 41 further comprising providing a plurality of different magnitude thresholds corresponding to the type of source.