Removing transient objects to synthesize an unobstructed image

• US 9,286,707 B1
• Filed: 04/04/2014
• Issued: 03/15/2016
• Est. Priority Date: 01/27/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Switkin, Daniel Jules
• Primary Examiner(s): Yeh, Eueng-Nan
• Application Number:
  US14/245,439
• Time in Patent Office: 711 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

In one aspect, the present subject matter is directed to a computer-implemented method for removing transient objects to synthesize an unobstructed image. The method includes receiving a plurality of images from a user. A most common value at a same pixel of each image is determined to infer pixels associated with a background of the images. A composition is generated from the received images, wherein the composition includes one of the pixels associated with the background of the images in place of a pixel associated with a foreground of at least one of the images.

Claims

1. A computer-implemented method for removing transient objects to synthesize an unobstructed image comprising:

receiving, by one or more computing devices, a plurality of images from a user, wherein the plurality of images comprise one or more transient objects;

determining, by the one or more computing devices, a most common value at a same pixel of each image to infer pixels associated with a background of the images;

determining, by the one or more computing devices, a most common value at a same pixel of each image to infer pixels associated with a foreground of at least one of the images wherein at least one of the pixels associated with the foreground of the at least one of the images are located in the same portion of the images as at least one of the pixels associated with a background of the images; and

generating, by the one or more computing devices, a composition from the received images, wherein the composition comprises one of the pixels associated with the background of the images based on the determined most common value at a same pixel of each image as a replacement for a pixel associated with the foreground of at least one of the images based on the determined most common value at a same pixel of each image to synthesize an unobstructed image.

2. The computer-implemented method of claim 1, wherein the most common value at the same pixel of each image is not present in a majority of the same pixels.

3. The computer-implemented method of claim 1, wherein the most common value comprises luminance, hue, saturation, or combinations thereof.

4. The computer-implemented method of claim 1, wherein the most common value is determined by calculating a median value of each pixel of each image.

5. The computer-implemented method of claim 1, further comprising preprocessing one or more of the images prior to determining the most common value.

6. The computer-implemented method of claim 5, wherein preprocessing comprises determining sharpness of one or more of the images.

7. The computer-implemented method of claim 1, further comprising providing the composition to the user.

8. The computer-implemented method of claim 1, further comprising:

aligning one or more of the images used in each generated composition.

9. The computer-implemented method of claim 1, further comprising:

normalizing one or more of the images used in each generated composition.

10. The computer-implemented method of claim 1, further comprising:

smoothing one or more of the images used in each generated composition.

11. A system for removing transient objects to synthesize an unobstructed image, the system comprising:

one or more computing devices including one or more processors and associated memory, the memory storing instructions that, when executed by the one or more processors, configure the one or more computing devices to;

receive a plurality of images from a user, wherein the plurality of images comprise one or more transient objects;

determine a most common value at a same pixel of each image to infer pixels associated with a background of the images;

determine a most common value at a same pixel of each image to infer pixels associated with a foreground of at least one of the images wherein at least one of the pixels associated with the foreground of the at least one of the images are located in the same portion of the images as at least one of the pixels associated with a background of the images; and

generate a composition from the received images, wherein the composition comprises one of the pixels associated with the background of the images based on the determined most common value at a same pixel of each image as a replacement for a pixel associated with the foreground of at least one of the images based on the determined most common value at a same pixel of each image to synthesize an unobstructed image.

12. The system of claim 11, wherein the most common value at the same pixel of each image is not present in a majority of the same pixels.

13. The system of claim 11, wherein the most common value comprises luminance, hue, saturation, or combinations thereof.

14. The system of claim 11, wherein the most common value is determined by calculating a median value of each pixel of each image.

15. The system of claim 11, further comprising providing the composition to the user.

16. A tangible, non-transitory computer-readable medium comprising:

computer-executable instructions that, when executed by one or more processors,

cause the one or more processors to perform operations, comprising;

receiving a plurality of images from a user, wherein the plurality of images comprise one or more transient objects;

determining a most common value at a same pixel of each image to infer pixels associated with a background of the images;

determining, by the one or more computing devices, a most common value at a same pixel of each image to infer pixels associated with a foreground of at least one of the images wherein at least one of the pixels associated with the foreground of the at least one of the images are located in the same portion of the images as at least one of the pixels associated with a background of the images; and

generating, a composition from the received images, wherein the composition comprises one of the pixels associated with the background of the images based on the determined most common value at a same pixel of each image as a replacement for a pixel associated with the foreground of at least one of the images based on the determined most common value at a same pixel of each image to synthesize an unobstructed image.

17. The computer readable medium of claim 16, wherein the most common value at the same pixel of each image is not present in a majority of the same pixels.

18. The computer readable medium of claim 16, wherein the most common value comprises luminance, hue, saturation, or combinations thereof.

19. The computer readable medium of claim 16, wherein the most common value is determined by calculating a median value of each pixel of each image.