IMAGE GRID WITH SELECTIVELY PROMINENT IMAGES

• US 20180150433A1
• Filed: 11/28/2016
• Published: 05/31/2018
• Est. Priority Date: 11/28/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: SOWDEN, Paul, KHANDELWAL, Madhur
• Granted Patent: US 10,140,675 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 1/00185 Image output for details of particular output arrangements see the relevant sub-class, e.g. B41J, G03B, G09G, G11B

Abstract

Implementations relate to update and display of panoramic image montages. In some implementations, a computer-implemented method includes identifying a plurality of images, where each image of the plurality of images has a respective importance score. A subset of the images is selected based at least in part on the respective importance score for each image. The method determines respective one or more cells in a grid for occupation by each of the images, where at least one image of the subset is placed in the grid such that it occupies at least two cells in the grid. The method causes the images to be displayed in a user interface on a display screen based on the grid.

Claims

1. A computer-implemented method comprising:

identifying a plurality of images, wherein each image of the plurality of images has a respective importance score;

selecting a subset of the plurality of images based at least in part on the respective importance score for each image;

determining respective one or more cells of a plurality of cells in a grid for occupation by each of the plurality of images, wherein at least one image of the subset is placed in the grid such that it occupies at least two cells in the grid; and

causing the plurality of images to be displayed in a user interface on a display screen based on the grid.

2. The computer-implemented method of claim 1, wherein determining the respective one or more cells in the grid for occupation by each of the plurality of images comprises, for each given image of the at least one image of the subset placed in the grid such that it occupies at least two cells in the grid:

determining an available cell in the grid;

determining one or more subsequent cells in the grid, wherein each subsequent cell is adjacent to at least one cell of the available cell and the one or more subsequent cells; and

placing the given image across the available cell and the one or more subsequent cells.

3. The computer-implemented method of claim 2, wherein determining the one or more subsequent cells is based on an aspect ratio of the given image.

4. The computer-implemented method of claim 2, wherein the available cell is a first empty cell in a right-then-down order in the grid.

5. The computer-implemented method of claim 1, wherein determining the respective one or more cells in the grid for occupation by each of the plurality of images comprises:

rearranging one or more previously-placed images in the grid after placement of the at least one image that occupies at least two cells in the grid, wherein the rearranging includes placing the one or more previously-placed images in a grid direction and in a sort order in cells of the grid available after the placement of the at least one image that occupies at least two cells in the grid.

6. The computer-implemented method of claim 1, wherein the at least one image of the subset placed in the grid includes multiple images of the subset placed in the grid, wherein each of the multiple images occupies at least two respective cells in the grid.

7. The computer-implemented method of claim 2, further comprising:

determining whether the available cell is adjacent to a previously-placed image that occupies at least two cells in the grid,

wherein in response to determining that the available cell is not adjacent to a previously-placed image that occupies at least two cells in the grid, performing the determining one or more subsequent cells in the grid and the placing the given image across the available cell and the one or more subsequent cells, and

wherein in response to determining that the available cell is adjacent to a previously-placed image that occupies at least two cells in the grid, placing the given image in the next available cell.

8. The computer-implemented method of claim 1, further comprising cropping one or more given images of the plurality of images, wherein the cropping of each given image of the one or more given images is based on a number of the respective one or more cells in the grid occupied by the given image.

9. The computer-implemented method of claim 1, wherein the respective importance score for each image is based upon one or more of:

one or more visual characteristics of the image, and prior user interaction with the image, and

wherein selecting the subset of the plurality of images comprises;

determining, for each image of the plurality of images, whether the respective importance score of the image meets an importance threshold; and

if the respective importance score of the image meets the importance threshold, adding the image to the subset.

10. The computer-implemented method of claim 9, wherein selecting the subset of the plurality of images further comprises:

determining whether a count of images in the subset divided by a count of the plurality of images meets a prominence threshold; and

in response to determining that the count of images in the subset divided by the count of the plurality of images meets the prominence threshold, removing one or more images from the sub set.

11. The computer-implemented method of claim 9, further comprising:

programmatically analyzing the subset of the plurality of images to determine at least one image of the subset that meets a similarity threshold with reference to other images of the subset; and

in response to determination that the at least one image meets the similarity threshold, removing the at least one image from the subset.

12. A device comprising:

a memory; and

at least one processor configured to access the memory, the at least one processor configured to perform operations comprising;

identifying a plurality of images, wherein each image of the plurality of images has a respective importance score;

selecting a subset of the plurality of images based at least in part on the respective importance score for each image;

determining a display layout for the plurality of images, wherein the display layout includes a plurality of cells arranged in a grid and wherein at least one image of the subset of images occupies at least two cells in the display layout; and

causing the plurality of images to be displayed in a user interface on a display device based on the display layout.

13. The device of claim 12, wherein the at least one processor is configured to determine the display layout, comprising:

inserting the plurality of images in the grid, wherein the inserting comprises, for one or more of the plurality of images;

selecting a particular image from the plurality of images;

selecting a particular cell from the plurality of cells in the grid;

determining if the particular image is in the subset of images;

in response to determining that the particular image is not in the subset of images, placing the particular image in the particular cell; and

in response to determining that the particular image is in the subset of images,

determining one or more subsequent cells from the plurality of cells in the grid to place the image; and

placing the particular image such that the particular image occupies the particular cell and the one or more subsequent cells.

14. The device of claim 13, wherein in response to determining that the particular image is in the subset of images, the at least one processor is further configured to perform operations comprising:

determining whether there is at least one cell adjacent to the particular cell that includes a previously-placed image that occupies more than one cell in the display layout; and

in response to determining that there is at least one cell adjacent to the particular cell that includes the previously-placed image, placing the particular image in the particular cell,

wherein the placing the particular image such that the particular image occupies the particular cell and the one or more subsequent cells is performed in response to determining that there is no cell adjacent to the particular cell that includes the previously-placed image.

15. The device of claim 13, wherein in response to determining that the particular image is in the subset of images, the at least one processor is further configured to perform operations comprising:

determining an aspect ratio of the particular image,

wherein the determining the one or more subsequent cells to place the image is based at least in part on the aspect ratio.

16. The device of claim 13, wherein the operation of determining the one or more subsequent cells further comprises:

determining a type of the image, wherein the type is one of;

square, landscape image, portrait image, horizontal panorama, and vertical panorama;

if the type of the image is square, selecting cells adjacent to the next available cell in a m×

n pattern, where m is equal to n;

if the type of the image is landscape or a horizontal panorama, selecting cells adjacent to the next available cell in a m×

n pattern, where m is greater than n; and

if the type of the image is portrait or vertical panorama, selecting cells adjacent to the next available cell in a m×

n pattern, where m is less than n,

wherein m refers to a number of columns in a horizontal direction along the grid and n refers to a number of rows in a vertical direction along the grid.

17. The device of claim 13, wherein the operation of selecting the particular image is performed in a sort order determined based on a sort parameter, and

wherein the at least one processor is further configured to perform operations comprising, prior to determining the display layout, sorting the plurality of images in the sort order based on the sort parameter.

18. The device of claim 13, wherein the grid includes a plurality of columns and wherein each of the plurality of cells is square-shaped, and

wherein the at least one processor is further configured to perform operations comprising, prior to causing the plurality of images to be displayed, cropping the plurality of images, wherein the cropping comprises, for each image of the plurality of images:

if the image is to be placed in the particular cell, cropping the image to a square shape; and

if the image is to be placed in the particular cell and the one or more subsequent cells, cropping the image based on an aspect ratio of the image.

19. The device of claim 12, wherein the respective importance score for each image is based upon one or more of:

one or more visual characteristics of the image, a number of faces in the image, a view count for the image, a share count for the image, an edit history for the image, whether a face over a threshold size appears in the image, whether a recognized face appears in the image, a location score for the image, and a number of near similar images in the plurality of images.

20. A non-transitory computer readable medium having stored thereon software instructions that, when executed by a processor, cause the processor to perform operations including:

identifying a plurality of images, wherein each image of the plurality of images has a respective importance score;

selecting a subset of the plurality of images based at least in part on the respective importance score for each image;

determining respective one or more cells of a plurality of cells in a grid for occupation by each of the plurality of images, wherein at least one image of the subset is placed in the grid such that it occupies at least two cells in the grid, wherein the plurality of cells in the grid are in arranged in a plurality of rows and columns, and wherein determining the respective one or more cells in the grid for each of the plurality of images comprises, for a given image in the plurality of images;

determining a next available cell in the grid;

placing the given image, wherein the placing comprises;

if the given image is not in the subset, placing the given image in the next available cell; and

if the given image is in the subset,

determining one or more subsequent cells in the grid; and

placing the given image across the next available cell and the one or more subsequent cells; and

causing the plurality of images to be displayed in a user interface on a display screen based on the grid.