Mobile device, image reproducing device and server for providing relevant information about image captured by image reproducing device, and method thereof

• US 9,652,659 B2
• Filed: 07/16/2015
• Issued: 05/16/2017
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Seong-min, Ahn, Kyung-ho, Han, Heung-woo
• Primary Examiner(s): YEH, EUENG NAN
• Application Number:
  US14/801,452
• Publication Number:
  US 20160078059A1
• Time in Patent Office: 670 Days
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content

Abstract

A method executed by a server to provide a mobile device with relevant information about an object in a captured image received from an image reproducing device, the method including: receiving, from the image reproducing device, an image captured by the image reproducing device; receiving, from the mobile device, information corresponding to an application usage history of the mobile device; searching, based on the application usage history, for the relevant information about the object in the captured image; and transmitting to the mobile device the relevant information found based on the searching.

Claims

1. A method executed by a server to provide a mobile device with relevant information about an object in a captured image received from an image reproducing device, the method comprising:

receiving, from the image reproducing device, an image captured by the image reproducing device;

receiving, from the mobile device, information corresponding to an application usage history of the mobile device;

determining an interest category based on the information corresponding to the application usage history;

obtaining a plurality of relevant keywords corresponding to an object in the received image captured by the image reproducing device;

selecting an interest keyword included in the interest category from among the plurality of obtained relevant keywords; and

transmitting to the mobile device the relevant information about the object in the captured image based on the selected interest keyword.

2. The method of claim 1, wherein the information corresponding to the application usage history comprises use information about at least one selected from among a name of an application executed on the mobile device, information about an operation performed by the application, and information about when and where the application was executed.

3. The method of claim 1, further comprising:

searching for the relevant information using the selected interest keyword.

4. The method of claim 1, further comprising searching for the relevant information using an identifier for the interest keyword and the interest category.

5. The method of claim 1, further comprising:

receiving, from the mobile device, an update request to update the relevant information;

receiving, from the mobile device information, updated application usage history; and

updating the relevant information about the object in the captured image based on the updated application usage history.

6. A non-transitory computer readable storage medium having recorded thereon a program for running the method of claim 1 on a computer.

7. A method executed by a server to provide a mobile device with relevant information about an object in a captured image received from an image reproducing device, the method comprising:

receiving, from the image reproducing device, an image captured by the image reproducing device;

receiving, from the mobile device, information corresponding to an application usage history of the mobile device;

searching, based on the application usage history, for the relevant information about the object in the captured image; and

transmitting to the mobile device the relevant information found based on the searching,

wherein the captured image is an image captured from content being reproduced by the image reproducing device, and

wherein the method further comprises;

receiving a content identifier from the image reproducing device;

obtaining broadcast information corresponding to the content from a broadcast server that provides the content based on the content identifier; and

transmitting the obtained broadcast information to the image reproducing device or the mobile device.

8. A method executed in a mobile device to output relevant information about an object in an image captured by an image reproducing device, the method comprising:

receiving a user input;

transmitting, to the image reproducing device, in response to the user input, an image capture request for capturing an image from content being reproduced by the image reproducing device;

transmitting, to a server, information corresponding to an application usage history of the mobile device;

receiving, from the server, relevant information about the object in the image captured by the image reproducing device according to the image capture request; and

outputting the received relevant information,

wherein the server obtains relevant information by;

determining an interest category based on the information corresponding to the application usage history;

obtaining a plurality of relevant keywords corresponding to the object in the image captured by the image reproducing device;

selecting an interest keyword included in the interest category from among the plurality of obtained relevant keywords; and

searching, based on the selected interest keyword, for the relevant information about the object in the captured image.

9. The method of claim 8, wherein the information corresponding to the application usage history comprises use information about at least one selected from among a name of an application executed on the mobile device, information about an operation performed by the application, and information about when and where the application was executed.

10. The method of claim 8, further comprising:

transmitting, to the server, a relevant information update request for updated relevant information;

transmitting, to the server, information corresponding to an updated application usage history of the user; and

receiving, from the server, the updated relevant information updated based on the updated application usage history.

11. A server comprising:

a transceiver configured to communicate with an image reproducing device and a mobile device; and

a controller configured to control the transceiver to receive an image captured by the image reproducing device from the image reproducing device, to control the transceiver to receive information corresponding to an application usage history of the mobile device from the mobile device, to determine an interest category based on the information corresponding to the application usage history, to search for relevant information about an object in the captured image based on the determined interest category, and to control the transceiver to transmit the relevant information to the mobile device,

wherein the controller is further configured to obtain a plurality of relevant keywords corresponding to the object in the captured image, and to select an interest keyword included in the interest category from among the plurality of obtained relevant keywords.

12. The server of claim 11, wherein the information corresponding to the application usage history comprises use information corresponding to at least one selected from among a name of the application executed on the mobile device, an operation performed by the application, and information about when and where the application was executed.

13. The server of claim 11, wherein the controller is further configured to search for the relevant information using the selected interest keyword.

14. The server of claim 11, wherein the controller is further configured to search for the relevant information based on the interest keyword and the interest category.

15. The server of claim 11, wherein the controller is further configured to control the transceiver to receive, from the mobile device, an update request to update the relevant information, to control the transceiver to receive, from the mobile device, information on an updated application usage history, and to update the relevant information based on the updated application usage history.

16. A server comprising:

a transceiver configured to communicate with an image reproducing device and a mobile device; and

a controller configured to control the transceiver to receive an image captured by the image reproducing device from the image reproducing device, to control the transceiver to receive information corresponding to an application usage history of the mobile device from the mobile device, to search for relevant information about an object in the captured image based on the application usage history, and to control the transceiver to transmit the relevant information to the mobile device,

wherein the image captured by the image reproducing device is an image captured from content being reproduced in the image reproducing device, and

wherein the controller is further configured to control the transceiver to receive a content identifier from the image reproducing device, to control the transceiver to obtain, from a broadcasting server that provides the content, broadcast information corresponding to the content based on the identifier for the content, and to control the transceiver to transmit, to the image reproducing device or the mobile device, the obtained broadcast information.

17. A mobile device comprising:

an input interface configured to receive a user input;

a transceiver; and

a controller configured to control the transceiver to transmit, to an image reproducing device, an image capture request for capturing an image from content being reproduced by the image reproducing device in response to the user input, to control the transceiver to transmit, to a server, information corresponding to an application usage history, and to control the transceiver to receive, from the server, relevant information about an object in the image captured by the image reproducing device according to the image capture request,

wherein the server is configured to obtain the relevant information by;

determining an interest category based on the information corresponding to the application usage history;

obtaining a plurality of relevant keywords corresponding to the object in the captured image;

selecting an interest keyword included in the interest category from among the plurality of obtained relevant keywords; and

searching, based on the selected interest keyword, for the relevant information about the object in the captured image.

18. A mobile device comprising:

a transceiver; and

a controller configured to control the transceiver to transmit, to an image reproducing device, an image capture request to capture an image of content reproduced by the image reproducing device, to determine an interest category based on application usage history of the mobile device, to search for relevant information corresponding to the image captured according to the image capture request based on the determined interest category, and to output the relevant information,

wherein the controller is further configured to extract an object from the captured image, to obtain a plurality of relevant keywords corresponding to the object in the captured image, and to select an interest keyword included in the interest category from among the plurality of obtained relevant keywords.

19. The mobile device of claim 18, wherein the controller is further configured to search for the relevant information corresponding to the extracted object based on the selected interest keyword.