System and method for creating navigable views

• US 10,558,884 B2
• Filed: 10/29/2018
• Issued: 02/11/2020
• Est. Priority Date: 07/30/2013
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Inventors: Loui, Alexander C., Manico, Joseph A.
• Primary Examiner(s): Seth, Manav
• Application Number:
  US16/173,984
• Publication Number:
  US 20190065886A1
• Time in Patent Office: 470 Days
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video

Abstract

A method for creating navigable views includes receiving digital images, computing a set of feature points for each of the digital images, selecting one of the digital images as a reference image, identifying a salient region of interest in the reference image, identifying other digital images containing a region of interest similar to the salient region of interest in the reference image using the set of feature points computed for each of other digital images, designating a reference location for the salient region of interest in the reference image, aligning the other digital images to the image that contains the designated reference location, ordering the image that contains the designated reference location and the other digital images, and generating a navigable view.

Claims

1. A computer-implemented method for creating virtual environments, the method comprising:

receiving a plurality of digital images from an image collection;

using a processor to compute a set of feature points for each digital image in the plurality of digital images;

using the processor to semantically identify, for each digital image in the plurality of digital images, primary image object types appearing in the digital image, wherein the primary image object types comprise foreground, background, stationary, and movable;

using the processor to categorize each digital image in the plurality of digital images according to the primary image object types appearing the in the digital image;

selecting, by the processor, one of the plurality of digital images that has been categorized as having the stationary primary image object type;

identifying, by the processor, a stationary object in the selected digital image as a salient region of interest;

using the processor to identify a set of matching digital images in the plurality of digital images, wherein each digital image in the set of matching digital images includes at least a portion of the salient region of interest; and

using the processor to stitch together the set of matching digital images to create a panoramic virtual environment comprising concentric bands or spheres.

2. The method of claim 1, wherein the primary image object type of movable comprises people, vehicles, toys, and animals.

3. The method of claim 1, further comprising extracting the semantically identified movable primary image object types and the semantically identified foreground primary image object types in each digital image in the plurality of digital images.

4. The method of claim 3, further comprising:

using the processor to compute the set of feature points corresponding to the stationary primary image object types and the background primary image object types in each digital image in the plurality of digital images; and

using the processor to identify feature points from the set of feature points that correspond to recurring stationary primary image object types and recurring background primary image object types in each digital image in the plurality of digital images; and

using the feature points corresponding to the recurring stationary primary image object types and the recurring background primary image object types to fill in any image data holes that are created by extracted movable objects.

5. The method of claim 1, wherein the movable primary image object types and the foreground primary image object types for each digital image in the plurality of digital images are modified to render the movable primary image object types and the foreground primary image object types as moving objects.

6. The method of claim 1, further comprising extracting the semantically identified movable primary image object types and the semantically identified foreground primary image object types for each digital image in the plurality of digital images and then automatically presenting animation options for the identified movable foreground primary image object types to a user.

7. The method of claim 1, further comprising presenting the panoramic virtual environment is as a series of concentric bands or spheres and in such manner to allow an observer to independently control objects in the panoramic virtual environment.

8. The method of claim 7, wherein the objects in the panoramic virtual environment are automatically animated when they enter the observer'"'"'s field of view.

9. The method of claim 7, wherein the observer uses camera monitored gestures, a pointing device, or touch screen to pan the series of concentric bands left or right, or to move through the series of concentric bands by going forward and back.

10. The method of claim 1 wherein the set of feature points are computed using SIFT (Shift Invariant Feature Transform), SURF (Speeded up Robust Features), 2D Haar wavelet, HOG (Histogram of Oriented Gradients), or GLOH (Gradient Location and Orientation Histogram) or combinations thereof.

11. The method of claim 1, further comprising the processor using an Active Shape Model algorithm to compute facial feature points in the plurality of digital images;

and the processor using the facial feature points to recognize and group similar faces.

12. The method of claim 1, further comprising the processor using K-means or ISODATA clustering algorithms to identify the set of matching digital images.

13. The method of claim 1, further comprising the processor using image-based criteria to identify the set of matching digital images, wherein the image-based criteria comprises one or more criteria selected from the group consisting of:

image quality, image composition, and image aesthetics.

14. The method of claim 1, wherein the stationary primary image object type includes furnished rooms, walls, floors, wall art, flowering gardens, natural settings, buildings, and cityscapes appearing in the plurality of digital images.