Presentation of content on companion display device based on content presented on primary display device

• US 9,496,922 B2
• Filed: 04/21/2014
• Issued: 11/15/2016
• Est. Priority Date: 04/21/2014
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Candelore, Brant
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): Khalid, Omer
• Application Number:
  US14/257,618
• Publication Number:
  US 20150304589A1
• Time in Patent Office: 939 Days
• Field of Search: 725/42, 725/39, 725/60, 348/500, 348/565, 348/564, 348/563
• US Class Current: 1/1
• CPC Class Codes: H04B 5/72 for local intradevice communication
• CPC Class Codes: H04N 21/42209 for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/485 End-user interface for client configuration
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

In one aspect, a first device includes a display, at least one computer readable storage medium bearing instructions executable by a processor, and at least one processor configured for accessing the computer readable storage medium to execute the instructions. The instructions configure the processor to acquire data pertaining to first content presented on a second device, determine information pertaining to the first content based on the data, identify second content related to but different from the first content at least in part based on the information, and present a user interface (UI) on the display manipulable to issue a command to at least one of the first and second devices to present the second content responsive to the identification of the second content.

Claims

1. A first device, comprising:

at least one computer display, the first device being a mobile communication device (MCD);

at least one computer memory comprising instructions executable by at least one processor of the MCD to;

receive a user command;

responsive to the user command, trigger an on-screen-display (OSD) to be presented on a second device, the second device being a primary display device (PDD) adapted to present first content;

search for second content related to but different from the first content;

present the second content on the MCD, the second content being ancillary content to the first content;

present at least one first user interface (UI) on the MCD including a listing of one or more pieces of second content that have been located during the search, second content denoted by respective selector elements in the listing being selectable to automatically without further user input cause the MCD to begin presenting the second content associated with a selected element from the listing, the first UI also including a device selection section including a prompt for a user to select one or more of the selector elements in the listing, the first UI also including a fling selector element selectable to cause the MCD to, responsive to a fling command or gesture, second content selected from the listing to a target device such that the second content automatically without further user input is presented on the target device;

wherein when the target device is the PDD, the PDD presents the first content simultaneously with presenting second content in a second UI, the second UI including functionality to cause the PDD to cease presenting the second content, the second UI including functionality to present a listing from which another second content can be selected for presentation;

the instructions being further executable to present;

a prompt for whether the MCD should automatically without user input be selected as a companion screen device to the PDD such that the MCD can be configured to automatically present second content based on detection of first content being presented on another device.

2. The first device of claim 1, wherein the instructions are further executable to:

responsive to receiving the command at the second device based at least in part on manipulation of a user interface (UI) presented responsive to identification of the second content, present the second content on the first device or the second device.

3. The first device of claim 1, wherein the command is a first command, and wherein the instructions are further executable to:

responsive to receiving a second command at the second device, present the second content on a third device different from the first device and different from the second device.

4. The first device of claim 2, wherein the UI includes at least a first selector element associated with content unrelated to the first content and unrelated to the second content that is selectable to present the content associated with the first selector element, the UI further including the at least a second selector element selectable to issue the command to at least one of the first and second devices to present the second content.

5. The first device of claim 1, wherein the instructions are executable to:

capture an image of an on-screen-display (OSD) to be presented on the PDD;

send the image to a computer network server for executing optical character recognition (OCR) on the image to identify information pertaining to first content;

at least in part based on the information, identify the second content related to but different from the first content; and

wherein the data is acquired at least in part based on at least one image of the first content detected by a camera on the first device.

6. The first device of claim 5, wherein the determination of information pertaining to the first content includes determining the identity of the first content based on the data.

7. The first device of claim 5, wherein the determination of information pertaining to the first content is at least in part based on an Internet search using at least a portion of the data.

8. The first device of claim 7, wherein the Internet search using at least a portion of the data is an Internet search of at least one image of the first content gathered by a camera on the first device.

9. The first device of claim 5, wherein the acquisition of data includes acquiring data pertaining to a watermark, the watermark included on at least one image of the first content presented on the second device.

10. The first device of claim 5, wherein the acquisition of data includes acquiring data pertaining to an audiomark in audio of the first content.

11. The first device of claim 5, wherein the data pertains to text associated with the first content, wherein the text is presented on the second device, and wherein the text is acquired based at least in part on optical character recognition.

12. The first device of claim 11, wherein the text is presented on an on screen display (OSD) of the second device.

13. The first device of claim 11, wherein the text is of closed captioning of the first content.

14. A method, comprising:

initiating a data collection application at a companion device (CD);

executing the data collection application to receive from the CD a command;

responsive to the command, triggering an on-screen-display (OSD) to be presented on a second device, the second device being a primary display (PDD) adapted to present first content, the PDD being different from the CD;

searching for second content related to but different from the first content;

presenting the second content on the CD, the second content being ancillary content to the first content;

presenting at least one first user interface (UI) on the CD including a listing of one or more pieces of second content that have been located during the searching for second content, second content denoted by respective selector elements in the listing being selectable to automatically without further user input cause the CD to begin presenting the second content associated with a selected element from the listing, the first UI also including a device selection section including a prompt for a user to select one or more of the selector elements in the listing, the first UI also including a fling selector element selectable to cause the CD to, responsive to a fling command or gesture, second content selected from the listing to a target device such that the second content automatically without further user input is presented on the target device;

wherein when the target device is the PDD, the method comprises presenting on the PDD the first content simultaneously with presenting second content in a second UI, the second UI including functionality to cause the PDD to cease presenting the second content, the second UI including functionality to present a listing from which another second content can be selected for presentation; and

prompting for whether the CD should automatically without user input be selected as a companion screen device to the PDD such that the CD can be configured to automatically present second content based on detection of first content being presented on another device.

15. The method of claim 14, wherein a user interface (UI) presented responsive to identification of the second content is manipulable to present the second content on the first device.

16. The method of claim 14, comprising:

using the MCD to capture an image of the OSD;

executing optical character recognition (OCR) on the image to identify information pertaining to first content;

at least in part based on the data, identifying the second content; and

without input from a user, presenting the second content, the second content being ancillary content to the first content, wherein the data pertaining to the first content is data pertaining to a first portion of audio of the first content, of video of the first content, or of audio and video of the first content; and

wherein the second content contains information regarding the first portion but not information regarding a second portion of the first content that is different from the first portion.

17. A computer memory that is not a transitory signal and that comprises instructions executable by at least one processor of a companion device (CD) to:

receive from a user input element on the first device a command;

responsive to the command, trigger an on-screen-display (OSD) to be presented on a second device, the second device being a primary display device (PDD);

present at least one first user interface (UI) on the CD including a listing of one or more pieces of second content that have been located during a search, second content denoted by respective selector elements in the listing being selectable to automatically without further user input cause the CD to begin presenting the second content associated with a selected element from the listing, the first UI also including a device selection section including a prompt for a user to select one or more of the selector elements in the listing, the first UI also including a fling selector element selectable to cause the CD to, responsive to a fling command or gesture, second content selected from the listing to a target device such that the second content automatically without further user input is presented on the target device;

wherein when the target device is the PDD, the PDD presents the first content simultaneously with presenting second content in a second UI, the second UI including functionality to cause the PDD to cease presenting the second content, the second UI including functionality to present a listing from which another second content can be selected for presentation; and

prompt for whether the CD should automatically without user input be selected as a companion screen device to the PDD such that the CD can be configured to automatically present second content based on detection of first content being presented on another device.

18. The computer readable storage medium of claim 17, wherein the instructions when executed by the processor configure the processor to identify the second content based at least in part on an Internet search of at least a portion of the data.

19. The computer readable storage medium of claim 17, wherein the second content is ancillary content to the first content, and the instructions are executable to:

capture an image of the OSD;

based on executing optical character recognition (OCR) on the image, presenting the listing.