Systems and methods of building and using an image catalog

• US 10,318,575 B2
• Filed: 11/13/2015
• Issued: 06/11/2019
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Zorroa Corporation
• Original Assignee: Zorroa Corporation
• Inventors: Wexler, Daniel Elliott, Buhler, Juan Jose
• Primary Examiner(s): Hoang, Son T
• Application Number:
  US14/941,502
• Publication Number:
  US 20160140146A1
• Time in Patent Office: 1,306 Days
• Field of Search: None
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information

Abstract

A method of managing an image catalog is performed by one or more servers. The process receives from a first user identification of one or more images in an image database. The image database is distinct from the servers. For each of the images, the process analyzes the image to extract keywords that describe the image and creates an index entry in the image catalog. The index entry includes the keywords. The process receives a query from a second user and matches the query to an index entry in the image catalog. The index entry corresponds to a first image in the image database. The process determines whether the second user is authorized to view the first image. When the second user is authorized to view the first image, the process retrieves the first image from the image database and transmits the first image to the second user.

Claims

1. A method of managing an image catalog, performed by one or more servers, each having one or more processors and memory, the method comprising:

receiving, by the one or more servers, reduced-resolution versions of one or more images, the one or more images stored in a first image database of an external service, wherein the first image database of the external service is distinct from the one or more servers;

for each respective image of the one or more images stored in the first image database of the external service;

analyzing, by the one or more servers, a reduced-resolution version of the respective image to extract respective keywords that describe the respective image; and

creating, by the one or more servers, a respective index entry in the image catalog, wherein the respective index entry includes the respective keywords;

receiving, by the one or more servers, a query from user;

matching, by the one or more servers, the query to a first index entry in the image catalog, wherein the first index entry corresponds to a first image stored as a full-resolution version in the first image database;

transmitting, by the one or more servers to an owner of the first image, a request for authorization of the user to view the first image;

in accordance with a determination that the user is authorized to view the first image;

retrieving, by the one or more servers, the full-resolution version of the first image from the first image database of the external service;

temporarily storing, b the one or more servers, the full-resolution version of first image in a temporary storage of the one or more servers; and

transmitting, by the one or more servers, the full-resolution version of the first image to the user, and

releasing, by the one or more servers, the full-resolution version of the first image from the temporary storage in response to the transmitting the full-resolution version of the first image to the user.

2. The method of claim 1, wherein the analyzing of the respective image to extract the respective keywords comprises performing deep convolutional neural network semantic analysis on the respective image.

3. The method of claim 2, further comprising receiving a specified subject matter for the one or more images, wherein the deep convolutional neural network semantic analysis uses a neural network trained on images corresponding to the specified subject matter.

4. The method of claim 1, wherein the analyzing of the respective image to extract the respective keywords comprises using optical character recognition.

5. The method of claim 1, wherein the analyzing of the respective image to extract the respective keywords comprises extracting a color palette for the respective image.

6. The method of claim 1, wherein the analyzing of the respective image to extract the respective keywords comprises identifying one or more known faces in the respective image.

7. The method of claim 1, wherein the analyzing of the respective image to extract the respective keywords comprises identifying one or more known human bodies based on body features.

8. The method of claim 1, further comprising for each respective image:

identifying metadata associated the respective image, wherein the respective metadata is selected from the group consisting of date or time when the respective image was created, location where the respective image was created, identification of a camera that took the respective image, and identification of camera attributes that took the respective image; and

storing the respective metadata as part of the respective index entry for the respective image.

9. A computer system for managing an image catalog, comprising:

one or more processors;

a memory; and

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs comprising instructions for;

receiving, by the computer system, reduced-resolution versions of one or more images, the one or more images stored in a first image database of an external service, wherein the first image database of the external service is distinct from the computer system;

for each respective image of the one or more images stored in the first image database of the external service;

analyzing, by the computer system, a reduced-resolution version of the respective image to extract respective keywords that describe the respective image; and

creating, by the computer system, a respective index entry in the image catalog, wherein the respective index entry includes the respective keywords;

receiving, by the computer system, a query from user;

matching, by the computer system, the query to a first index entry in the image catalog, wherein the first index entry corresponds to a first image stored as a full-resolution version in the first image database;

transmitting, by the computer system to an owner of the first image, a request for authorization of the user to view the first image;

in accordance with a determination that the user is authorized to view the first image;

retrieving, by the computer system, the full-resolution version of the first image from the first image database of the external service;

temporarily storing, by the computer system, the full-resolution version of first image in a temporary storage of the computer system; and

transmitting, by the computer system, the full-resolution version of the first image to the second user, and

releasing, by the computer system, the full-resolution version of the first image from the temporary storage in response to the transmitting the full-resolution version of the first image to the user.

10. The computer system of claim 9, wherein the analyzing of the respective image to extract the respective keywords comprises performing deep convolutional neural network semantic analysis on the respective image.

11. The computer system of claim 9, wherein the analyzing of the respective image to extract the respective keywords comprises using optical character recognition.

12. The computer system of claim 9, wherein the analyzing of the respective image to extract the respective keywords comprises extracting a color palette for the respective image.

13. The computer system of claim 9, wherein the analyzing of the respective image to extract the respective keywords comprises identifying one or more known faces in the respective image.

14. The computer system of claim 9, wherein the analyzing of the respective image to extract the respective keywords comprises identifying one or more known human bodies based on body features in the respective image.

15. The computer system of claim 9, wherein the one or more programs further comprising instructions that execute for each respective image for:

identifying metadata associated the respective image, wherein the respective metadata is selected from the group consisting of date or time when the respective image was created, location where the respective image was created, identification of a camera that took the respective image, and identification of camera attributes that took the respective image; and

storing the respective metadata as part of the respective index entry for the respective image.

16. A non-transitory computer readable storage medium storing one or more programs configured for execution by one or more processors of a computer system for managing an image catalog, the one or more programs comprising instructions for:

receiving, by the computer system, reduced-resolution versions of one or more images, the one or more images stored in a first image database of an external service, wherein the first image database of the external service is distinct from the computer system;

for each respective image of the one or more images stored in the first image database of the external service;

analyzing, by the computer system, a reduced-resolution version of the respective image to extract respective keywords that describe the respective image; and

creating, by the computer system, a respective index entry in the image catalog, wherein the respective index entry includes the respective keywords;

receiving, by the computer system, a query from user;

matching, by the computer system, the query to a first index entry in the image catalog, wherein the first index entry corresponds to a first image stored as a full-resolution version in the first image database;

transmitting, by the computer system to an owner of the first image, a request for authorization of the user to view the first image;

in accordance with a determination that the user is authorized to view the first image;

retrieving, by the computer system, the full-resolution version of the first Image from the first image database of the external service;

temporarily storing, by the computer system, the full-resolution version of first image in a temporary storage of the computer system; and

transmitting, by the computer system, the full-resolution version of the first image to the second user, and

releasing, by the computer system, the full-resolution version of the first image from the temporary storage in response to the transmitting the full-resolution version of the first image to the user.

17. The non-transitory computer readable storage medium of claim 16, wherein the analyzing of the respective image to extract the respective keywords comprises at least one of:

performing deep convolutional neural network semantic analysis on the respective image, using optical character recognition, extracting a color palette for the respective image, identifying one or more known faces in the respective image, or identifying one or more known human bodies based on body features.

18. The non-transitory computer readable storage medium of claim 17, further comprising receiving a specified subject matter for the one or more images, wherein the deep convolutional neural network semantic analysis uses a neural network trained on images corresponding to the specified subject matter.

19. The non-transitory computer readable storage medium of claim 16, wherein the one or more programs further comprising instructions that execute for each respective image for:

identifying metadata associated the respective image, wherein the respective metadata is selected from the group consisting of date or time when the respective image was created, location where the respective image was created, identification of a camera that took the respective image, and identification of camera attributes that took the respective image; and

storing the respective metadata as part of the respective index entry for the respective image.

20. The non-transitory computer readable storage medium of claim 16, wherein the retrieving of the first image from the first image database includes:

retrieving the first image from an offline storage medium when the offline storage medium is back online.