Iterative image search algorithm informed by continuous human-machine input feedback

• US 10,268,702 B2
• Filed: 02/26/2016
• Issued: 04/23/2019
• Est. Priority Date: 08/15/2014
• Status: Active Grant

• Current Assignee: Ask Sydney, LLC
• Original Assignee: Paul Lawrence Epstein, Sydney Nicole Epstein
• Inventors: Epstein, Sydney Nicole, Epstein, Paul Lawrence, Maric, Luka
• Primary Examiner(s): Pond, Robert M
• Application Number:
  US15/054,979
• Publication Number:
  US 20160179847A1
• Time in Patent Office: 1,152 Days
• Field of Search: 705 26, 705 27
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0631 Item recommendations

Abstract

System and computer-implemented image search engine of analyzing tags associated with a sequence of images presented to a user to present a current object of interest of the user is disclosed. An image from among a plurality of images is presented on an electronic display. The image is associated with a set of tags. An input is received indicating a user'"'"'s preference for the image. A plurality of tags is processed based on the preference and the set of tags to determine a next set of tags from the plurality of tags. A next image is determined from the plurality of images based on the next set of tags. The next image represents a physical object, different from a physical object represented by the previous image. A sequence of images is generated by repeating the above process with the next image in place of the previous image for present a user'"'"'s current object of interest.

Claims

1. A computer-implemented method comprising:

determining, by one or more computer devices, a plurality of tags specific to a user, from among a pool of tags, based on each tag of the plurality of tags specific to the user being associated with a profile of the user;

transmitting, from the one or more computer devices, one electronic image, from among a plurality of electronic images stored on the one or more computer devices, to an electronic device, the one image being associated with one set of tags from the plurality of tags specific to the user, each tag of the one set of tags describing or characterizing attributes of the one image;

receiving, from the electronic device, an input from the user indicating a preference for the one image;

processing, by the one or more computer devices, the plurality of tags specific to the user based on the preference and the one set of tags to determine a next set of tags from the plurality of tags and, in response to the preference for the one image being positive, the processing of the plurality of tags further comprising;

determining tags, from the plurality of tags, having a threshold association with one or more tags of the one set of tags;

determining a weighting of the tags having the threshold association based, at least in part, on a number of times each tag of the plurality of tags specific to the user is associated with a positive and/or a negative preference by the user; and

determining the next set of tags based on the next set of tags including at least one tag of the tags having the threshold association,

wherein the at least one tag is a highest weighted tag of the tags having the threshold association;

determining, by the one or more computer devices, a next image from the plurality of images associated with the next set of tags, the next image being different from the one image, and the next set of tags describing or characterizing attributes of the next image; and

generating a sequence of images by repeating the presenting, the receiving, the processing, and the determining with the next image in place of the one image during a session of presenting a current interest of the user,

wherein the input from the user indicating the preference for the one image is positive at least once during the session of presenting the current interest of the user.

2. The method of claim 1, in response to the preference for the one image being negative, the processing of the plurality of tags comprising:

removing tags from the one set of tags from the plurality of tags that are processed to determine the next set of tags.

3. The method of claim 2, wherein the one set of tags includes primary tags and secondary tags, and the tags removed from the plurality of tags include only the primary tags.

4. The method of claim 2, in response to the preference for the one image being negative, the processing of the plurality of tags further comprising:

determining tags, from the plurality of tags, having a threshold association with one or more tags of the one set of tags; and

removing the tags having the threshold association from the plurality of tags that are processed to determine the next set of tags, for the remainder of the session.

5. The method of claim 4, wherein the removing includes marking the tags having the threshold association so as to not consider the tags having the threshold association among the plurality of tags that are processed to determine the next set of tags.

6. The method of claim 2, wherein the removing includes marking the tags of the one set of tags so as to not consider the tags among the plurality of tags that are processed to determine the next set of tags.

7. The method of claim 1, wherein the weighting is based on a profile of the user, physical objects associated with the tags having the threshold association, locations corresponding to the physical objects associated with the tags having the threshold association, entities corresponding to the physical objects associated with the tags having the threshold association, or a combination thereof.

8. The method of claim 1, further comprising:

determining a location associated with the user, the electronic device, or a combination thereof; and

selecting the plurality of images from a pool of images based on each image of the plurality of images being associated with the location.

9. The method of claim 8, wherein each image of the plurality of images is associated with the location based on each physical object represented by each image being available at the location.

10. The method of claim 1, further comprising:

logging the one or more inputs during the generating of the sequence of images as interactions of the user with one or more tags, one or more sets of tags, one or more images, one or more physical objects, one or more physical entities associated with the physical objects, or a combination thereof.

11. The method of claim 10, further comprising:

modifying associations between (1) tags, (2) sets of tags, (3) an image and a tag, a set of tags, or a combination thereof, or (4) a combination thereof based on the interactions.

12. The method of claim 1, wherein the plurality of tags includes primary tags and secondary tags, the primary tags being defined by an administrator and the secondary tags being defined by one or more users.

13. The method of claim 1, further comprising:

receiving, via a user interface of the electronic device, an input by the user indicating that a last image of the sequence of images presented to the user represents the current interest of the user.

14. The method of claim 13, further comprising:

determining a location associated with the user, the electronic device, or a combination thereof; and

presenting, via a display of the electronic device, a list of physical entities proximate the location that provide a physical object represented by the last image.

15. The method of claim 14, further comprising:

receiving, via the user interface of the electronic device, an input by the user selecting one of the physical entities provided in the list; and

presenting, via the display of the electronic device, a user interface element displaying a profile associated with the selected physical entity.

16. One or more computer-readable, non-transitory, storage media encoding machine-readable instructions that, when executed by one or more computer devices, cause operations to be carried out comprising:

determining, by the one or more computer devices, a plurality of tags specific to a user, from among a pool of tags, based on each tag of the plurality of tags specific to the user being associated with a profile of the user;

transmitting, from the one or more computer devices, one electronic image, from among a plurality of electronic images stored on the one or more computer devices, to an electronic device, the one image being associated with one set of tags from the plurality of tags specific to the user, each tag of the one set of tags describing or characterizing attributes of the one image;

receiving, from the electronic device, an input from the user indicating a preference for the one image;

processing, by the one or more computer devices, the plurality of tags specific to the user based on the preference and the one set of tags to determine a next set of tags from the plurality of tags and, in response to the preference for the one image being positive, the processing of the plurality of tags further comprising;

determining tags, from the plurality of tags, having a threshold association with one or more tags of the one set of tags;

determining a weighting of the tags having the threshold association based, at least in part, on a number of times each tag of the plurality of tags specific to the user is associated with a positive and/or a negative preference by the user; and

determining the next set of tags based on the next set of tags including at least one tag of the tags having the threshold association,

wherein the at least one tag is a highest weighted tag of the tags having the threshold association;

determining, by the one or more computer devices, a next image from the plurality of images associated with the next set of tags, the next image being different from the one image, and the next set of tags describing or characterizing attributes of the next image; and

generating a sequence of images by repeating the presenting, the receiving, the processing, and the determining with the next image in place of the one image during a session of presenting a current interest of the user.

17. A computer-implemented method comprising:

receiving, from an electronic device associated with a user, an indication of an instance of an application executed on the electronic device, wherein the application, together with one or more computer devices, is configured to direct the user to a current interest;

determining, by the one or more computer devices, a plurality of tags specific to the user, from among a pool of tags, based on each tag of the plurality of tags specific to the user being associated with a profile of the user;

causing a presentation on a display of the electronic device of one electronic image, from among a plurality of electronic images stored on the one or more computer devices, the one image representing an item and being associated with one set of tags from the plurality of tags specific to the user, each tag of the one set of tags describing or characterizing attributes of the item represented by the one image;

processing, by the one or more computer devices, the plurality of tags specific to the user to determine a first potential set of tags corresponding to a positive preference from the user for the item represented by the one image and a second potential set of tags corresponding to a negative preference from the user for the item represented by the one image, prior to receiving an input from the user indicating a preference for the item;

wherein the processing, by the one or more computer devices, of the plurality of tags specific to the user to determine the first potential set of tags includes determining at least one additional tag from the plurality of tags specific to the user to add to the one set of tags, generating the first potential set of tags, and

wherein the determining of the at least one additional tag includes;

determining weightings of the tags within the plurality of tags specific to the user based, at least in part, on (i) a number of times each tag of the plurality of tags specific to the user appears with at least one of the one or more tags of the one set of tags for the plurality of electronic images and/or (ii) a number of times each tag of the plurality of tags specific to the user is associated with a positive and/or a negative preference by the user; and

determining the at least one additional tag based on the at least one additional tag having a highest weighting among the plurality of tags specific to the user;

transmitting, from the one or more computer devices, a first potential electronic image and a second potential electronic image, prior to receiving the input from the user indicating the preference for the item, the first potential electronic image being associated with the first potential set of tags and the second potential electronic image being associated with the second potential set of tags;

receiving, from the electronic device, the input from the user indicating the preference for the item represented by the one image;

causing, by the one or more computer devices, a presentation of the first potential electronic image or the second potential electronic image based on the preference for the item represented by the one image being the positive preference or the negative preference; and

generating a sequence of electronic images presented to the user one at a time on the display of the electronic device by repeating the processing, the transmitting, the receiving the input form the user, and the causing of the presentation of the first potential electronic image or the second potential electronic image in place of the one image to direct the user to the current interest during a session of the instance of the application executed on the electronic device.

18. The method of claim 17, wherein the processing, by the one or more computer devices, of the plurality of tags specific to the user to determine the second potential set of tags includes:

in response to the one image being a first electronic image presented to the user during the session of the instance of the application, removing the tags of the one set of tags from the plurality of tags specific to the user that are processed to determine any potential set of tags, for a remainder of the session; and

in response to the one image being not the first electronic image presented to the user during the session of the instance of the application, removing tags of the one set of tags that are new relative to an immediately previous set of tags from the plurality of tags specific to the user that are processed to determine the any set of tags, for the remainder of the session.

19. The method of claim 18, wherein the removing includes marking the tags of the one set of tags so as to not consider the tags among the plurality of tags that are processed to determine the next set of tags.

20. A computer-implemented method comprising:

determining, by one or more computer devices, a plurality of tags specific to a user, from among a pool of tags, based on each tag of the plurality of tags specific to the user being associated with a profile of the user;

transmitting, from the one or more computer devices, one electronic image, from among a plurality of electronic images stored on the one or more computer devices, to an electronic device, the one image being associated with one set of tags from the plurality of tags specific to the user, each tag of the one set of tags describing or characterizing attributes of the one image;

receiving, from the electronic device, an input from the user indicating a preference for the one image;

processing, by the one or more computer devices, the plurality of tags specific to the user based on (i) the preference, (ii) the one set of tags, and (iii) weightings of the tags within the plurality of tags specific to the user based, at least in part, on a number of times each tag of the plurality of tags specific to the user is associated with a positive and/or a negative preference by the user to determine a next set of tags from the plurality of tags and, in response to the preference for the one image being negative, the processing of the plurality of tags comprising;

removing tags from the one set of tags from the plurality of tags that are processed to determine the next set of tags, for a remainder of a session of presenting a current interest of the user, wherein the one set of tags includes primary tags and secondary tags, and the tags removed from the plurality of tags include only the primary tags;

determining, by the one or more computer devices, a next image from the plurality of images associated with the next set of tags, the next image being different from the one image, and the next set of tags describing or characterizing attributes of the next image; and

generating a sequence of images by repeating the presenting, the receiving, the processing, and the determining with the next image in place of the one image during the session of presenting the current interest of the user,

wherein the input from the user indicating the preference for the one image is negative at least once during the session of presenting the current interest of the user.

21. The method of claim 20, in response to the preference for the one image being negative, the processing of the plurality of tags further comprising:

determining tags, from the plurality of tags, having a threshold association with one or more tags of the one set of tags; and

removing the tags having the threshold association from the plurality of tags that are processed to determine the next set of tags, for the remainder of the session.

22. The method of claim 21, wherein the removing includes marking the tags having the threshold association so as to not consider the tags having the threshold association among the plurality of tags that are processed to determine the next set of tags.

23. The method of claim 20, wherein the removing includes marking the tags of the one set of tags so as to not consider the tags among the plurality of tags that are processed to determine the next set of tags.