Image-based information providing device and method using object recognition

• US 9,495,592 B2
• Filed: 07/03/2013
• Issued: 11/15/2016
• Est. Priority Date: 07/10/2012
• Status: Active Grant

• Current Assignee: Yahoo Japan Corporation (A Holdings Corp.)
• Original Assignee: Yahoo Japan Corporation (A Holdings Corp.)
• Inventors: Sakamoto, Ryuki
• Primary Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US13/934,903
• Publication Number:
  US 20140016822A1
• Time in Patent Office: 1,231 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user

Abstract

An information providing device according to the exemplary embodiment includes an object recognizing unit, a retrieving unit, an obtaining unit, and a transmitting unit. The object recognizing unit extracts an image of a specific object which appears in an image of a moving image content to be distributed to a terminal device. The retrieving unit requests a retrieval device to retrieve a similar image with the image of the specific object as a retrieval key and obtains a retrieval result from the retrieval device. The obtaining unit obtains recommend information related with an image of the specific object which appears in an image of a moving image content based on the retrieval result. The transmitting unit transmits the recommend information to the terminal device.

Claims

1. An information providing device that communicates with a terminal device via a network, the information providing device comprising:

a processor operatively coupled to a memory, wherein the processor is configured to;

recognize an image of a specific object which appears in an image of a moving image content to be distributed to the terminal device via the network;

request a retrieval device to retrieve similar images that are similar to the image of the specific object as a retrieval key;

obtain a retrieval result from the retrieval device, the retrieval result including at least two or more different web addresses, each of the two or more different web addresses being an address of a webpage in which an image that is similar to the image of the specific object is posted;

determine content that is common between webpages of the two or more different web addresses;

obtain recommend candidates based on the content that is common between the webpages containing at least one similar image;

transmit, via the network, selection information including information on the recommend candidates to the terminal device;

obtain a selection result with respect to the selection information from the terminal device via the network, the selection result including information on one or more recommend candidates selected from the recommend candidates;

obtain recommend information corresponding to the selected one or more recommend candidates; and

transmit the recommend information to the terminal device via the network in accordance with a temporal reproducing position.

2. The information providing device according to claim 1, wherein the processor is further configured to extract the image of the specific object from an image of a person who appears in the moving image content after extracting the image of the person who appears in the image of the moving image content.

3. The information processing device according to claim 2, wherein the processor is further configured to:

recognize a specific person among persons appearing in the moving image content, and

obtain the image of the specific object from the image of the specific person.

4. The information providing device according to claim 1, wherein the processor is further configured to transmit the recommend information to the terminal device so as to be displayed in the terminal device at a timing when the image of the specific object appears in a reproducing image of the moving image content in the terminal device or at a timing when the image of the specific object disappears from the reproducing image.

5. The information providing device according to claim 1, wherein the processor is further configured to:

transmit the moving image content to the terminal device in a streaming manner; and

recognize the specific object from the image of the moving image content which is being distributed to the terminal device.

6. The information providing device according to claim 1, wherein

the memory is configured to store the image of the specific object and a temporal reproducing position of the moving image content from which the image of the specific object is recognized, so as to be associated with each other, and

the processor is further configured to:

obtain the image of the specific object in accordance with the temporal reproducing position stored in the memory.

7. The information processing device according to claim 1, wherein the temporal reproducing position indicates an order of appearance of the image included in the moving image content.

8. The information processing device according to claim 1, wherein the processor is further configured to transmit the recommend information to the terminal device before the image of the specific object appears in a reproducing image of the moving image content in the terminal device.

9. The information processing device according to claim 1, wherein the processor is further configured to recognize the specific object in accordance with attribute information of a user of the terminal device.

10. The information processing device according to claim 1, wherein the processor is further configured to:

(i) collect web pages of the two or more different web addresses, and

(ii) determine the recommend information based on the collected web pages.

11. The information processing device according to claim 1, wherein the processor is further configured to retrieve the similar image with a plurality of images among images of specific objects which continuously appear in the moving image content as the retrieval key.

12. An information providing method which is executed by a computer, the method comprising:

recognizing an image of a specific object which appears in an image of a moving image content to be distributed to a terminal device via a network;

requesting a retrieval device to retrieve similar images that are similar to the extracted image of the specific object as a retrieval key;

obtaining a retrieval result from the retrieval device, the retrieval result including at least two or more different web addresses, each of the two or more different web addresses being an address of a webpage containing an image that is similar to the image of the specific object;

determining content that is common between webpages of the two or more different web addresses;

obtaining recommend candidates based on the content that is common between the webpages containing at least one similar image;

transmitting, via the network, selection information including information on the recommend candidates to the terminal device;

obtaining, via the network, a selection result with respect to the selection information from the terminal device, the selection result including information on one or more recommend candidates selected from the recommend candidates;

obtaining recommend information corresponding to the selected one or more recommend candidates; and

transmitting the obtained recommend information via the network in accordance with a temporal reproducing position to the terminal device.

13. A non-transitory computer readable storage medium containing program instructions, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to:

recognize an image of a specific object which appears in an image of a moving image content to be distributed to a terminal device via a network;

request a retrieval device to retrieve similar images that are similar to the extracted image of the specific object as a retrieval key;

obtain a retrieval result from the retrieval device, the retrieval result including at least two or more different web addresses, each of the two or more different web addresses being an address of a webpage in which an image that is similar to the image of the specific object is posted;

determine content that is common between webpages of the two or more different web addresses;

obtain recommend candidates based on the content that is common between the webpages containing at least one similar image;

transmit, via the network, selection information including information on the recommend candidates to the terminal device;

obtain, via the network, a selection result with respect to the selection information from the terminal device, the selection result including information on one or more recommend candidates selected from the recommend candidates;

obtain recommend information corresponding to the selected one or more recommend candidates; and

transmit the obtained recommend information via the network in accordance with a temporal reproducing position to the terminal device.

14. An information providing device that communicates with a terminal device via a network, the information providing device comprising:

a processor operatively coupled to a memory, wherein the processor is configured to;

recognize an image of a specific object which appears in an image of a moving image content to be distributed to the terminal device via the network;

request a retrieval device to retrieve similar images that are similar to the image of the specific object as a retrieval key;

obtain a retrieval result from the retrieval device, the retrieval result including at least two or more different web addresses, each of the two or more different web addresses being an address of a webpage where an image that is similar to the image of the specific object is posted;

determine content that is common between webpages of the two or more different web addresses;

obtain recommend information based on the content that is common between the webpages containing at least one similar image; and

transmit the recommend information to the terminal device via the network in accordance with a temporal reproducing position.