IDENTIFYING IMAGE COMMENTS FROM SIMILAR IMAGES

• US 20170185236A1
• Filed: 12/28/2015
• Published: 06/29/2017
• Est. Priority Date: 12/28/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Rui, Yong, Ma, Wei-Ying, Liu, Chunyan, Li, Di, Xu, Pengfei, Wang, Baoxun, Yang, Kuiyuan
• Granted Patent: US 10,732,783 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

An image chat application generates comments to images based on features of the images. In one example, the image chat application searches through a repository of stored image-comment pairs to identify a stored image that is similar to the image, and generates a comment to the image based on an identified stored image-comment pair. In another example, the image chat application may identify and tag particular objects that dominate an image, and may generate a comment to the image based on characteristics of those particular objects. In this second example, the image chat application further generates a comment to the image based on comments previously associated with the identified tag.

Claims

1. A system comprising:

one or more processors;

a computer-readable medium coupled to the one or more processors, including one or more modules that are executable by the one or more processors to;

receive data indicating an image;

extract, by the one or more processors, a first set of feature representations from the image;

determine a similarity between the image and individual stored images of a dataset of stored images, based at least in part on the first set of feature representations of the image and a second set of feature representations of the individual stored images;

identify a particular stored image from the dataset of stored images, based at least in part on the similarity;

identify a comment, based at least in part on the particular stored image; and

cause, a display of the comment on a user interface of a client device, the comment being displayed as a response to a display of the image.

2. The system of claim 1, wherein the first set of feature representations from the image correspond to a dominant object of the image.

3. The system of claim 1, wherein the one or more modules are further executable by the one or more processors to:

identify a tag that is associated with the particular stored image; and

wherein the comment is identified further based at least in part on the tag.

4. The system of claim 3, wherein the dataset of stored images is a first dataset of stored images, the tag of the particular stored image is a first tag of a first particular stored image;

and

wherein the one or more modules are further executable by the one or more processors to;

identify a second dataset of stored images, based at least in part on the first tag, wherein individual stored images of the second dataset of stored images include one or more tags that describe a dominant object associated with the individual stored images;

determine a similarity between the image and individual stored images of the second dataset of stored images;

identify a second particular stored image from the second dataset of stored images, based at least in part on the similarity; and

wherein the comment is identified further based at least in part on a second tag of the second particular stored image.

5. The system of claim 4, wherein the one or more modules are further executable by the one or more processors to identify the second dataset of stored images further based at least in part on a dataset index, wherein the dataset index identifies at least one association between the first tag and the second tag.

6. The system of claim 1, wherein the client device is associated with a user or a user group, and wherein the individual stored images of the dataset of stored images include at least one tag that identifies a dominant object of the individual stored images, the at least one tag being based at least in part on particular tags that are commonly associated with the user or the user group.

7. The system of claim 1, wherein the one or more modules are further executable by the one or more processors to:

determine that the image is displayed on a web-based social media platform, based at least in part on the data indicating the image; and

retrieve the image from the web-based social media platform.

8. One or more computer-readable storage media including one or more modules that are executable by one or more processors to cause a computing system to:

receive data indicating an image;

determine a similarity between the image and individual stored images of a dataset of stored images, based at least in part on a first set of feature representations of the image and a second set of feature representations of the individual stored images; and

identify a particular stored image from the dataset of stored images, based at least in part on the similarity; and

identify a particular comment from a plurality of comments associated with the particular stored image; and

cause a display of the particular comment on a user interface of a client device, the particular comment being displayed as a response to a display of the image.

9. The one or more computer-readable storage media of claim 8, wherein the plurality of comments are based at least in part on one or more of:

comments previously posted by users on a social media platform;

or

stock comments.

10. The one or more computer-readable storage media of claim 8, wherein the one or more modules are further executable by the one or more processors to select the particular comment randomly from the plurality of comments.

11. The one or more computer-readable storage media of claim 8, wherein the one or more modules are further executable by the one or more processors to identify a tag associated with the particular stored image, and the particular comment is identified from the plurality of comments based at least in part on the tag.

12. A computer-implemented method, comprising:

receiving, via a client device, data indicating an image;

extracting, from the image, a first set of feature representations having a first feature vector;

determining a similarity between the image and individual stored images of a dataset of stored images, based at least in part on the first feature vector and a second feature vectors extracted from the individual stored images;

identifying a particular stored image from the dataset of stored images, based at least in part on the similarity;

identifying a plurality of comments based at least in part on the particular stored image; and

causing, a display of at least a particular comment of the plurality of comments on a user interface of the client device as a response to a display of the image.

13. The computer-implemented method of claim 12, wherein the plurality of comments are associated with the individual stored images or are associated with one or more tags, the one or more tags being associated with the individual stored images.

14. The computer-implemented method of claim 12, further comprising selecting the particular comment randomly from the plurality of comments.

15. The computer-implemented method of claim 12, wherein the individual stored images of the dataset of stored images include one or more tags that identify a dominant object of the individual stored images, the method further comprising identifying a tag from the one or more tags that is associated with the particular stored image, wherein identifying the plurality of comments is further based at least in part on the tag.

16. The computer-implemented method of claim 15, wherein the tag is a first tag, the particular stored image is a first particular stored image, and the dataset of stored images is a first dataset of stored images, and further comprising:

determining that a number of the plurality of comments is greater than a predetermined threshold number of comments;

identifying a second tag that is associated with the first tag, based at least in part on a dataset index that identifies an association between the first tag and the second tag;

identifying a second dataset of stored images that is associated with the second tag;

determining a similarity between the image and stored images of the second dataset of stored images;

identifying a second particular stored image from the second dataset of stored images, based at least in part on the similarity; and

wherein identifying the plurality of comments is further based at least in part on additional comments associated with the second tag.

17. The computer-implemented method of claim 16, wherein the first tag identifies the dominant object of the image as a tangible entity, and wherein the second tag further describes features commonly associated with the tangible entity.

18. The computer-implemented method of claim 12, further comprising reducing a first length of the first feature vector and a second length of the second feature vector to a same predetermined feature vector length.

19. The computer-implemented method of claim 12, further comprising reducing a first length of the first feature vector and a second length of the second feature vector progressively through a plurality of non-linear mapping layers associated with the dataset of stored images and the image.

20. The computer-implemented method of claim 12, wherein reducing a first length of the first feature vector and a second length of the second feature vector is performed using at least one of a Top-K most frequent feature dimensionality reduction technique, a principle component analysis technique, or a local sensitive hashing technique.