Database systems and user interfaces for dynamic and interactive mobile image analysis and identification

• US 10,043,102 B1
• Filed: 03/16/2016
• Issued: 08/07/2018
• Est. Priority Date: 01/20/2016
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Goenka, Aakash, Jain, Arzav, Taheri, Alexander, Isaza, Daniel, Zhu, Jack, Manson, William, Turan, Vehbi Deger, Jastrzebski, Stanislaw
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US15/071,886
• Time in Patent Office: 874 Days
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 40/166 using acquisition arrangements

Abstract

Embodiments of the present disclosure relate to systems, techniques, methods, and computer-readable mediums for one or more database systems configured to perform image identification of an image captured using a remote mobile device, and display of identity information associated with the captured image on the remote mobile device, in communication with the database system(s). A system obtains an image captured using a camera on a remote mobile device and performs image analysis to identify the captured image using reference images in one or more databases. The system can present the results for display an interactive user interface on the remote mobile device.

Claims

1. A system comprising:

a mobile device comprising;

a camera for capturing images;

a data store configured to store computer-executable instructions;

a communication component for communicating over a mobile communication network; and

a processor, wherein the computer-executable instructions, when executed, configure the processor to;

obtain an image captured by the camera;

transmit the image to a system back-end unit via the communication component;

receive one or more images representing closest matches to the transmitted image and information associated with the one or more images from the system back-end unit via the communication component; and

generate for display a user interface, the user interface comprising;

a result display of the one or more closest matches of images; and

a display of the information associated with any of the one or more closest matches of images; and

transmit a request, to the system back-end unit and via the communication component, for additional information associated with a first match selected from the one or more closest matches;

receive additional information associated with the first match;

update the user interface to include the additional information, wherein the additional information comprises information that was not previously displayed;

the system back-end unit comprising;

one or more data stores configured to store computer-executable instructions, image data, and information associated with images in the image data;

a communication component for communicating over the mobile communication network; and

one or more processors, wherein the computer-executable instructions, when executed, configure the one or more processors to;

receive, via the communication component, an image transmitted by the mobile device;

determine one or more closest matches to the received image among the image data in the one or more data stores;

retrieve from the one or more data stores information associated with the one or more closest matches;

transmit the one or more closest matches and the associated information to the mobile device via the communication component;

receive, via the communication component, a request for additional information associated with the first match; and

transmit additional information associated with the first match, wherein the additional information comprises information that was not previously transmitted.

2. The system of claim 1, wherein the one or more processors in the system back-end unit determine one or more closest matches to the received image among the image data in the one or more data stores based on a threshold of confidence level.

3. The system of claim 2, wherein the threshold of confidence level is dynamically adjusted by the system back-end unit to change the number of closest matches presented on the result display of the mobile device user interface.

4. The system of claim 3, wherein the number of closest matches presented on the result display of the mobile device user interface is based at least in part on a preconfigured maximum number of matches.

5. The system of claim 1, wherein:

the image data in the one or more data stores in the system back-end unit includes visual images and feature sets extracted from the visual images; and

the one or more processors in the system back-end unit determine one or more closest matches to the received image among the image data in the one or more data stores through the use of the extracted feature sets.

6. The system of claim 1, wherein the computer-executable instructions in the mobile device, when executed, further configure the processor in the mobile device to:

extract a feature set from the captured image; and

transmit the extracted feature set in lieu of the captured image to the system back-end unit via the communication component.

7. The system of claim 1, wherein:

the computer-executable instructions in the mobile device, when executed, further configure the processor to;

obtain one or more pieces of information associated with the captured image; and

transmit the one or more pieces of information to the system back-end unit; and

the computer-executable instructions in the system back-end unit, when executed, further configure the processor to;

determine one or more closest matches to the received image among the image data in the one or more data stores, based in part on closeness of the one or more pieces of information associated with the received image and the information associated with the image data in the one or more data stores.

8. The system of claim 7, wherein:

the image data or information in the one or more data stores in the system back-end unit is dynamically updated; and

the system is further configured to update the information in the one or more data stores in the system back-end unit by adding the one or more pieces of information associated with a captured image to the one or more data stores.

9. The system of claim 1, wherein the image data or information in the one or more data stores in the system back-end unit is dynamically updated.

10. The system of claim 9, wherein the system is further configured to update the image data or information in the one or more data stores in the system back-end unit by adding a captured image to the one or more data stores.

11. The system of claim 9, wherein the system is further configured to retain an image captured by the camera in the mobile device for match redetermination after the one or more data stores in the system back-end unit is dynamically updated with additional image data.

12. The system of claim 9, wherein the system is further configured to retain a search result for result re-presentation after the one or more data stores in the system back-end unit is dynamically updated with additional information.

13. The system of claim 1, wherein:

the computer-executable instructions in the mobile device, when executed, further configures the processor in the mobile device to;

obtain a plurality of captured images; and

transmit the plurality of captured images to the system back-end unit; and

the computer-executable instructions in the system back-end unit, when executed, further configures the processor in the system back-end unit to;

determine one or more closest matches to an image transmitted by the mobile device among the image data in the one or more data stores, based in part on association between a closest match to the image and a closest match to another image in the plurality of images.

14. The system of claim 1, wherein the number of closest matches presented on the result display of the mobile device user interface is sorted by match confidence levels associated with the respective closest matches.

15. A computer-implemented method comprising:

obtaining a visual image via a mobile electronic device;

transmitting the visual image from the mobile electronic device;

receiving the visual image by an electronic system;

determining, in the electronic system, one or more closest matches to the visual image among image data stored in the electronic system;

retrieving, from one or more storages in the electronic system, information associated with the one or more closest matches;

transmitting image data and information associated with the one or more closest matches from the electronic system;

receiving image data and information associated with the one or more closest matches by the mobile electronic device;

generating, in the mobile electronic device, one or more user interfaces to display the image data or information associated with any one or more of the one or more closest matches

receiving a request for additional information associated with a first match selected from the one or more closest matches via the mobile electronic device;

retrieving, from one or more storages in the electronic system, additional information associated with the one or more closest matches;

transmitting, from the electronic system, the retrieved additional information associated with the first match, wherein the additional information is different from the information associated with the one or more closest matches that was previously transmitted to the mobile electronic device; and

updating, in the mobile electronic device, the user interface to include the additional information.

16. The method of claim 15, further comprising:

obtaining one or more additional visual images using the mobile electronic device; and

determining, in the electronic system, one or more closest matches to any one or more of the visual images among image data stored in the electronic system, based at least in part on association between a closest match to a visual image and a closest match to another visual image.

17. The method of claim 15, further comprising:

obtaining one or more pieces of information associated with the visual image via the mobile electronic device;

transmitting the one or more pieces of information from the mobile electronic device to the electronic system; and

determining, in the electronic system, one or more closest matches to the visual image among image data stored in the electronic system, based in part on closeness of the one or more pieces of information associated with the visual image and the information associated with the image data stored in the electronic system.

18. The method of claim 15, wherein determining, in the electronic system, one or more closest matches to the visual image among image data stored in the electronic system is based on a threshold of confidence level.

19. The method of claim 15, wherein:

image data stored in the electronic system includes visual images and feature sets extracted from the visual images; and

determining, in the electronic system, one or more closest matches to the received visual image among image data stored in the electronic system is performed through the use of the extracted feature sets.

20. The method of claim 15, wherein image data and information stored in the electronic system are dynamically updated.