MOBILE DEVICE, IMAGE REPRODUCING DEVICE AND SERVER FOR PROVIDING RELEVANT INFORMATION ABOUT IMAGE CAPTURED BY IMAGE REPRODUCING DEVICE, AND METHOD THEREOF

• US 20160078059A1
• Filed: 07/16/2015
• Published: 03/17/2016
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kang, Seong-Min, AHN, Kyung-ho, HAN, Heung-woo
• Granted Patent: US 9,652,659 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content

Abstract

A method executed by a server to provide a mobile device with relevant information about an object in a captured image received from an image reproducing device, the method including: receiving, from the image reproducing device, an image captured by the image reproducing device; receiving, from the mobile device, information corresponding to an application usage history of the mobile device; searching, based on the application usage history, for the relevant information about the object in the captured image; and transmitting to the mobile device the relevant information found based on the searching.

Claims

1. A method executed by a server to provide a mobile device with relevant information about an object in a captured image received from an image reproducing device, the method comprising:

receiving, from the image reproducing device, an image captured by the image reproducing device;

receiving, from the mobile device, information corresponding to an application usage history of the mobile device;

searching, based on the application usage history, for the relevant information about the object in the captured image; and

transmitting to the mobile device the relevant information found based on the searching.

2. The method of claim 1, wherein the information corresponding to the application usage history comprises use information about at least one selected from among a name of an application executed on the mobile device, information about an operation performed by the application, and information about when and where the application was executed.

3. The method of claim 1, wherein the searching for the relevant information comprises:

obtaining a keyword corresponding to the object in the captured image;

obtaining a relevant keyword corresponding to the obtained keyword; and

searching for the relevant information using the obtained relevant keyword.

4. The method of claim 3, wherein the searching for the relevant information using the relevant keyword comprises:

determining an interest category based on the information corresponding to the application usage history; and

selecting an interest keyword included in the interest category from among a plurality of obtained relevant keywords.

5. The method of claim 4, wherein the searching for the relevant information using the obtained relevant keyword further comprises searching for the relevant information using an identifier for the interest keyword and the interest category.

6. The method of claim 1, wherein the captured image is an image captured from content being reproduced by the image reproducing device, and

wherein the method further comprises:

receiving a content identifier from the image reproducing device;

obtaining broadcast information corresponding to the content from a broadcast server that provides the content based on the content identifier; and

transmitting the obtained broadcast information to the image reproducing device or the mobile device.

7. The method of claim 1, further comprising:

receiving, from the mobile device, an update request to update the relevant information;

receiving, from the mobile device information, updated application usage history; and

updating the relevant information about the object in the captured image based on the updated application usage history.

19. A non-transitory computer readable storage medium having recorded thereon a program for running the method of claim 1 on a computer.

8. A method executed in a mobile device to output relevant information about an object in an image captured by an image reproducing device, the method comprising:

receiving a user input;

transmitting, to the image reproducing device, in response to the user input, an image capture request for capturing an image from content being reproduced by the image reproducing device;

transmitting, to a server, information corresponding to an application usage history of the mobile device;

receiving, from the server, relevant information about the object in the image captured by the image reproducing device according to the image capture request; and

outputting the received relevant information.

9. The method of claim 8, wherein the information corresponding to the application usage history comprises use information about at least one selected from among a name of an application executed on the mobile device, information about an operation performed by the application, and information about when and where the application was executed.

10. The method of claim 8, further comprising:

transmitting, to the server, a relevant information update request for updated relevant information;

transmitting, to the server, information corresponding to an updated application usage history of the user; and

receiving, from the server, the updated relevant information updated based on the updated application usage history.

11. A server comprising:

a transceiver configured to communicate with an image reproducing device and a mobile device; and

a controller configured to control the transceiver to receive an image captured by the image reproducing device from the image reproducing device, to control the transceiver to receive information corresponding to an application usage history of the mobile device from the mobile device, to search for relevant information about an object in the captured image based on the application usage history, and to control the transceiver to transmit the relevant information to the mobile device.

12. The server of claim 11, wherein the information corresponding to the application usage history comprises use information corresponding to at least one selected from among a name of the application executed on the mobile device, an operation performed by the application, and information about when and where the application was executed.

13. The server of claim 11, wherein the controller is further configured to obtain a keyword corresponding to the object in the captured image, to obtain a relevant keyword corresponding to the keyword, and to search for the relevant information using the obtained relevant keyword.

14. The server of claim 13, wherein the controller is further configured to determine an interest category based on the information corresponding to the application usage history, and to select an interest keyword included in the interest category from among a plurality of obtained relevant keywords.

15. The server claim 14, wherein the controller is further configured to search for the relevant information based on the interest keyword and the interest category.

16. The server claim 11, wherein the image captured by the image reproducing device is an image captured from content being reproduced in the image reproducing device, and

wherein the controller is further configured to control the transceiver to receive a content identifier from the image reproducing device, to control the transceiver to obtain, from a broadcasting server that provides the content, broadcast information corresponding to the content based on the identifier for the content, and to control the transceiver to transmit, to the image reproducing device or the mobile device, the obtained broadcast information.

17. The server of claim 11, wherein the controller is further configured to control the transceiver to receive, from the mobile device, an update request to update the relevant information, to control the transceiver to receive, from the mobile device, information on an updated application usage history, and to update the relevant information based on the updated application usage history.

18. A mobile device comprising:

an input interface configured to receive a user input;

a transceiver; and

a controller configured to control the transceiver to transmit, to an image reproducing device, an image capture request for capturing an image from content being reproduced by the image reproducing device in response to the user input, to control the transceiver to transmit, to a server, information corresponding to an application usage history, and to control the transceiver to receive, from the server, relevant information about an object in the image captured by the image reproducing device according to the image capture request.

20. A mobile device comprising:

a transceiver; and

a controller configured to control the transceiver to transmit, to an image reproducing device, an image capture request to capture an image of content reproduced by the image reproducing device, to search for relevant information corresponding to the image captured according to the image capture request, and to output the relevant information.

21. The mobile device of claim 20, wherein the controller is further configured to extract an object from the captured image, and to search for the relevant information corresponding to the extracted object.

22. The mobile device of claim 20, wherein the controller is further configured to search for relevant information based on an application usage history of the mobile device.

23. The mobile device of claim 20, wherein the relevant information comprises information corresponding to a plurality of searches based on a plurality of relevant keywords.