Recommending similar content identified with a neural network

• US 8,032,469 B2
• Filed: 05/06/2008
• Issued: 10/04/2011
• Est. Priority Date: 05/06/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Atallah, Antoine J
• Primary Examiner(s): Holmes; Michael B
• Application Number:
  US12/115,886
• Publication Number:
  US 20090281975A1
• Time in Patent Office: 1,246 Days
• Field of Search: 706/18
• US Class Current: 706/18
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06N 3/02 Neural networks

Abstract

Methods, systems and computer-readable media for finding similarities between visual objects by evaluating user interactions with a collection of visual objects are provided. Using a neural network, human interactions with a collection of visual objects are evaluated to ascertain relationships or connections between visual objects. The relationship between visual objects indicates that the visual objects are similar. Once relationships between visual objects are identified, a user may select one or more visual objects and receive suggested visual objects that are similar to the one or more visual objects selected by the user.

Claims

1. One or more computer-readable media having computer-executable instructions embodied thereon for performing a method of finding similar visual objects within a plurality of visual objects, the method comprising:

storing the plurality of visual objects in a data store, wherein each visual object within the plurality of visual objects includes an identifier and visual content;

storing a neural network that includes a plurality of nodes, wherein each node in the neural network is associated with an individual visual object in the plurality of visual objects, and wherein each node includes at least one similarity coefficient that indicates a degree of similarity between the node and another node;

receiving a first selection of a first visual object from a user, wherein the first visual object is one of the plurality of visual objects;

determining that a second visual object is similar to the first visual object because a similarity coefficient between a first node and a second node indicates that the second visual object is similar to the first visual object, wherein the first node corresponds to the first visual object and the second node corresponds to the second visual object, and wherein the similarity coefficient is the highest similarity coefficient associated with the first node;

communicating the second visual object for presentation to the user.

2. The media of claim 1, wherein the each visual object also includes keywords that describe the visual content.

3. The media of claim 2, wherein the method further includes:

receiving a query from the user; and

presenting one or more visual objects that are responsive to the query based on the keywords that are associated with the one or more visual objects, wherein the first visual object is included within the one or more visual objects.

4. The media of claim 1, wherein the each node includes the identifier for a visual object with which the each node is associated.

5. The media of claim 1, wherein the method further comprised receiving a plurality of identifiers from the neural network that correspond to a second plurality of visual objects that are similar to the first visual object based on similarity coefficients between nodes associated with the plurality of identifiers and the first visual object.

6. The media of claim 1, wherein a sum of similarity coefficients associated with a node in the neural network equals a first number or zero, wherein if the sum of the similarity coefficients equals zero the node is not related to a different node in the neural network.

7. The media of claim 6, wherein a high value for the similarity coefficient indicates a strong relationship between nodes in the neural network.

8. The media of claim 1, wherein the method further comprises:

receiving a second selection of the second visual object from the user, wherein the second visual object is one of the plurality of visual objects;

determining that a third visual object is similar to the first visual object and the second visual object because an aggregated similarity coefficient that is calculated by adding a first similarity coefficient between the first node and a third node with a second similarity coefficient between the second node and the third node indicates that the third visual object is similar to the first visual object and the second visual object, wherein the third node corresponds to the third visual object; and

presenting the third visual object to the user.

9. A computerized system, including one or more computer-readable media, containing a two-layer neural network for finding similar visual objects within a plurality of visual objects, the system comprising:

the two-layer neural network containing a plurality of nodes that are linked by similarity coefficients, wherein the similarity coefficients in the two-layer neural network are updated using training pairs, wherein a training pair includes identification information for one or more input visual objects and the identification information for one or more output visual objects, and wherein the training pair is generated based on user interactions with the one or more input visual objects and the one or more output visual objects, and wherein;

(1) each node is associated with a visual object in a data store, and

(2) a sum of the similarity coefficients associated with the each node must equal a first number or zero, wherein if the sum of the similarity coefficients is zero for a single node then the single node is not related to any other nodes in the two-layer neural network; and

a similarity search component for;

(1) receiving a selection of one or more selected visual objects that are each associated with a corresponding node,

(2) identifying one or more similar visual objects by evaluating the similarity coefficients associated with the each node associated with each of the one or more selected visual objects, and

(3) causing the one or more similar visual objects to be presented to a user.

10. The system of claim 9, wherein the data store includes the plurality of visual objects that include visual content, keywords that describe the visual content, and identification information.

11. The system of claim 10, wherein the visual content includes one or more of:

a clip art,

a picture,

a digital photograph,

a document, and

a spreadsheet.

12. The system of claim 10, further including an alphanumeric search component for receiving an alphanumeric query from the user and identifying visual objects in the data store that are responsive to the alphanumeric query and causing identified visual objects to be presented to the user.

13. The system of claim 10, wherein a first plurality of similarity coefficients associated with a corresponding plurality of nodes in the two-layer neural network is calculated based on keywords associated with the plurality of visual objects.

14. A method for training a neural network to determine that a first visual object is similar to a second visual object, the method comprising:

receiving a first selection of the first visual object returned in response to a query;

receiving a second selection of the second visual object returned in response to the query, wherein the first visual object and the second visual object were determined to be responsive to the first query based on keywords associated with the first visual object and the second visual object;

creating a training set including a first identifier for the first visual object as an input and a second identifier for the second visual object as an output, wherein the training set is used to adjust a plurality of similarity coefficients associated with nodes in a neural network so that when the training set input is submitted to the neural network the neural network produces the training set output in response;

inputting the training set into the neural network; and

updating a similarity coefficient from a first node in the neural network that corresponds to the first visual object to a second node in the neural network that corresponds to the second visual object, wherein the similarity coefficient is initially determined by using relationships identified by the keywords associated with the first visual object and the second visual object, wherein the similarity coefficient is updated using a reverse propagation algorithm, wherein a sum of the similarity coefficients associated with the first node equals a number, and wherein the sum of the similarity coefficients associated with all other nodes in the neural network equal the number or zero.