Image and video stitching and viewing method and system

• US 20080253685A1
• Filed: 02/25/2008
• Published: 10/16/2008
• Est. Priority Date: 02/23/2007
• Status: Abandoned Application

• Current Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Original Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Inventors: Nathan, Vaidhi, Gaikwad, Deepak, Kuranov, Alexander, Dhoble, Tejashree, Egorov, Sergey
• Application Number:
  US12/072,186
• Publication Number:
  US 20080253685A1
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

Multiple images taken from different locations or angles and viewpoints are joined and stitched. After the stitching and joining a much larger video or image scenery may be produced than any one image or video form which the final scenery was produced or an image of a different perspective than any of the input images may be produced.

Claims

1. A method comprising:

determining at least at least one unique aspect that is found in at least two images or videos, each image or video representing a different perspective;

determining a transformation that maps the at least two images or videos into a single image or video; and

forming a single image or video from the at least one image or video by at least applying the transformation determined.

2. The method of claim 1, the single image or video formed is a panorama that includes at least a first portion that is shown in multiple images or videos of the at least two images or videos, a second portion that is not shown in at least a first of the at least two images, and a third portion that is not shown in at least a second of the at least two images.

3. The method of claim 2, further comprising within the first portion offering a user a choice of displaying image information taken from a first of the at least two images or videos or displaying image information from taken from a second of the at least two images or videos.

4. The method of claim 1, the single image or video formed has a perspective that is not shown in any of the at least two images or videos.

5. The method of claim 4, the determining of the transformation including at least computing a three dimensional model of objects in the at least two images, and the single image formed having the perspective is based on the transformation.

6. The method of claim 1, further comprising

capturing a first image or video from a first input image or video

capturing a second image or video from a second input image or video,

information for a first portion of the single image formed being taken from the first image

information for a second portion of the single image formed being taken from the second image

adjusting a brightness of a first portion of the single image or video formed to match a brightness of the second portion of the single image or video formed.

7. The method of claim 1, further comprising

capturing a first image or video from a first input image or video

capturing a second image or video from a second input image or video,

information for a first portion of the single image formed being taken from the first image

information for a second portion of the single image formed being taken from the second image

adjusting a contrast of a first portion of the single image or video formed to match a contrast of the second portion of the single image or video formed.

8. The method of claim 1, the at least one unique aspect including a moving object that is found in each of the at least two images or videos.

9. The method of claim 1, the at least one unique aspect including an edge feature of an object that is found in each of the at least two images or videos.

10. The method of claim 1, the determining of the transformation including at least

determining a first mesh of points on at least a first of the at least two images or videos;

and

determining a second mesh of points on at least a second of the at least two images or videos; and

determining a transformation includes at least determining a transformation between at least one of the points of the first mesh and at least one of the points of the second mesh.

11. The method of claim 1, the determining of the transformation including at least

determining a mesh of points on at least one of the images or videos;

constraining movement of at least one of the points of the mesh; and

moving at least one unconstrained point, which is a point that does not have a constrained movement.

12. The method of claim 1, further comprising prior to the forming of the single image,

identifying an object that is being videoed;

determining that the object moves within a video segment from a camera in a manner that is expected to be a result of the camera shaking, the video segment including a set of frames from the camera;

adjusting positions at least portions of the video to remove motion of the object that is expected to be a result of the camera shaking.

13. The method of claim 1, the determining of the transformation including at least

determining a first texture of map of a first of the at least two images;

determining a second texture of map of a second of the at least two images;

determining a transformation from the first texture map to the second texture map; and

determining a transformation that maps the at least two texture maps into an image based on the transformation of the first texture map to the second texture map.

14. The method of claim 1, further comprising:

determining a portion of the image or video formed that changed;

rendering an update of only the portion of the image or video formed that changed;

combining the update with a portion of the image or video formed that did not change; and

sending for display a resulting image or video of the combining.

15. The method of claim 1, the method further comprising

capturing a first image or video from a first input image or video

capturing a second image or video from a second input image or video,

information for a first portion of the single image formed being taken from the first image;

information for a second portion of the single image formed being taken from the second image;

adjusting a brightness and a contrast of a first portion of the single image or video formed to match a brightness and a contrast of the second portion of the single image or video formed;

the single image or video formed is a panorama that includes at least

a first portion that is shown in multiple images or videos of the at least two images or videos,

a second portion captured by the first camera that is not shown in at least a first of the at least two images, and

a third portion captured by the second camera that is not shown in at least a second of the at least two images;

the determining of the transformation including at least computing a three dimensional model of objects in the at least two images, and the single image formed being based on the transformation;

within the first portion offering a user a choice of displaying image information taken from a first of the at least two images or videos or displaying image information from taken from a second of the at least two images or videos;

determining a first texture of map of a first of the at least two images;

determining a second texture of map of a second of the at least two images;

determining a transformation from the first texture map to the second texture map; and

determining a transformation that maps the at least two texture maps into an image based on the transformation of the first texture map to the second texture map;

prior to the forming of the single image,

identifying an object that is being videoed;

determining that the object moves within a video segment from a camera in a manner that is expected to be a result of the camera shaking, the video segment including a set of frames from the camera;

adjusting positions at least portions of the video to remove motion of the object that is expected to be a result of the camera shaking.

16. A system comprising a machine-readable medium that stores instructions that cause a processor to implement the method of claim 1.

17. A system comprising:

one or more machines configured for

merging or joining two videos, two frames of one video, or two images into one video or still image, and

efficiently storing, displaying, and transmitting to the one video or still image, results from the merging or joining, to the one or more machines or to another external device.

18. The system of claim 17, the two videos or images being a multiplicity of videos or images, the method further comprising:

a multiplicity of cameras, each camera photographing a different portion of a scene, and the merging or joining of the two videos or images being a merging or joining of the multiplicity of videos or images that forms a panorama of the scene.

19. A system comprising

only one video input;

a processor configured for

extracting from only the one video or set of still images a sequence of images including different viewing angles,

inputting the sequence of images to a panorama creation portion of the system and

creating a panorama from the sequence of images; and

an output for displaying, storing, or sending the result wirelessly or over a network for display;

20. The system of claim 19, the processor being configured such that the sequence of images does not include all images of the one video or set of still images.