Joint embedding for item association
First Claim
1. A method for associating semantically-related items of a plurality of item types, comprising:
- (a) embedding, by one or more computers, training items of the plurality of item types in a joint embedding space having more than two dimensions configured in a memory coupled to at least one processor, wherein each of the dimensions is defined by a real-valued axis, wherein each embedded training item corresponds to a respective location in the joint embedding space, wherein said each embedded training item is represented by a respective vector of real numbers corresponding to the respective location, and wherein each of the real numbers of the vector corresponds to a mapping of the respective location to one of the dimensions;
(b) learning, by the one or more computers, one or more mappings into the joint embedding space for each of the plurality of item types to create a trained joint embedding space and one or more learned mappings, the learning including;
selecting, based on known relationships between the training items, a related pair of the training items that includes a first item and a second item, wherein the first item and the second item are separated by a first distance in the joint embedding space;
selecting a third item that is less related to the first item than the second item, but is closer to the first item than the second item in the joint embedding space; and
adjusting a mapping function to increase a distance between the first item and the third item relative to a distance between the first item and the second item; and
(c) associating, by the one or more computers, one or more of the embedded training items with a first item based upon a distance in the trained joint embedding space from the first item to each said associated embedded training items, wherein each said distance is determined based upon a first vector of real numbers corresponding to the first item and a second vector of real numbers corresponding to a respective one of the associated embedded training items.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems to associate semantically-related items of a plurality of item types using a joint embedding space are disclosed. The disclosed methods and systems are scalable to large, web-scale training data sets. According to an embodiment, a method for associating semantically-related items of a plurality of item types includes embedding training items of a plurality of item types in a joint embedding space configured in a memory coupled to at least one processor, learning one or more mappings into the joint embedding space for each of the item types to create a trained joint embedding space and one or more learned mappings, and associating one or more embedded training items with a first item based upon a distance in the trained joint embedding space from the first item to each said associated embedded training items. Exemplary item types that may be embedded in the joint embedding space include images, annotations, audio and video.
-
Citations
25 Claims
-
1. A method for associating semantically-related items of a plurality of item types, comprising:
-
(a) embedding, by one or more computers, training items of the plurality of item types in a joint embedding space having more than two dimensions configured in a memory coupled to at least one processor, wherein each of the dimensions is defined by a real-valued axis, wherein each embedded training item corresponds to a respective location in the joint embedding space, wherein said each embedded training item is represented by a respective vector of real numbers corresponding to the respective location, and wherein each of the real numbers of the vector corresponds to a mapping of the respective location to one of the dimensions; (b) learning, by the one or more computers, one or more mappings into the joint embedding space for each of the plurality of item types to create a trained joint embedding space and one or more learned mappings, the learning including; selecting, based on known relationships between the training items, a related pair of the training items that includes a first item and a second item, wherein the first item and the second item are separated by a first distance in the joint embedding space; selecting a third item that is less related to the first item than the second item, but is closer to the first item than the second item in the joint embedding space; and adjusting a mapping function to increase a distance between the first item and the third item relative to a distance between the first item and the second item; and (c) associating, by the one or more computers, one or more of the embedded training items with a first item based upon a distance in the trained joint embedding space from the first item to each said associated embedded training items, wherein each said distance is determined based upon a first vector of real numbers corresponding to the first item and a second vector of real numbers corresponding to a respective one of the associated embedded training items. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 22, 23, 24, 25)
-
-
14. A system for associating semantically-related items of a plurality of item types, comprising:
-
at least one processor; a memory coupled to the at least one processor; a joint embedding space configurator configured to embed training items of the of the plurality of item types in a joint embedding space in the memory, wherein the joint embedding space has more than two dimensions, wherein each of the dimensions is defined by a real-valued axis, wherein each embedded training item corresponds to a respective location in the joint embedding space, wherein said each embedded training item is represented by a respective vector of real numbers corresponding to the respective location, and wherein each of the real numbers of the vector corresponds to a mapping of the respective location to one of the dimensions; a mapper configured to learn one or more mappings into the joint embedding space for each of the item types to create a trained joint embedding space and one or more learned mappings, wherein the mapper learns the one or more mappings by performing operations including; selecting, based on known relationships between the training items, a related pair of the training items that includes a first item and a second item, wherein the first item and the second item are separated by a first distance in the joint embedding space; selecting a third item that is less related to the first item than the second item, but is closer to the first item than the second item in the joint embedding space; and adjusting a mapping function to increase a distance between the first item and the third item relative to a distance between the first item and the second item; and an item associator configured associate one or more of the embedded training items with a first item based upon a distance in the trained joint embedding space from the first item to each associated embedded training items, wherein each said distance is determined based upon a first vector of real numbers corresponding to the first item and a second vector of real numbers corresponding to a respective one of the associated embedded training items. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
-
21. A non-transitory computer readable medium storing instructions wherein said instructions when executed cause at least one processor to associate semantically-related items of a plurality of item types using a method comprising:
-
embedding training items of the plurality of item types in a joint embedding space configured in a memory coupled to at least one processor, wherein the joint embedding space has more than two dimensions, wherein each of the dimensions is defined by a real-valued axis, and wherein each embedded training item corresponds to a respective location in the joint embedding space, wherein said each embedded training item is represented by a respective vector of real numbers corresponding to the respective location, and wherein each of the real numbers of the vector corresponds to a mapping of the respective location to one of the dimensions; learning one or more mappings into the joint embedding space for each of the item types to create a trained joint embedding space and one or more learned mappings, the learning including; selecting, based on known relationships between the training items, a related pair of the training items that includes a first item and a second item, wherein the first item and the second item are separated by a first distance in the joint embedding space; selecting a third item that is less related to the first item than the second item, but is closer to the first item than the second item in the joint embedding space; and adjusting a mapping function to increase a distance between the first item and the third item relative to a distance between the first item and the second item; and associating one or more of the embedded training items with a first item based upon a distance in the trained joint embedding space from the first item to each said associated embedded training items, wherein each said distance is determined based upon a first vector of real numbers corresponding to the first item and a second vector of real numbers corresponding to a respective one of the associated embedded training items.
-
Specification