Second screen methods and arrangements

• US 9,462,341 B2
• Filed: 02/28/2014
• Issued: 10/04/2016
• Est. Priority Date: 11/07/2008
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F.
• Primary Examiner(s): Featherstone, Mark D
• Application Number:
  US14/193,935
• Publication Number:
  US 20140282735A1
• Time in Patent Office: 949 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04H 60/372 Programme
• CPC Class Codes: H04H 60/61 Arrangements for services using the result of monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04H 60/74 using programme related information, e.g. title, composer or interpreter
• CPC Class Codes: H04H 60/80 characterised by transmission among terminal devices
• CPC Class Codes: H04H 60/85 which are mobile communication networks
• CPC Class Codes: H04N 2005/91335 the copy protection signal being a watermark
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/4227 Providing Remote input by a user located remotely from the client device, e.g. at work
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/8358 : involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 5/76 : Television signal recording
• CPC Class Codes: H04N 5/765 : Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/775 : between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/913 : for scrambling ; for copy protection
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04N 9/8205 : involving the multiplexing of an additional signal and the colour video signal
• CPC Class Codes: H04N 9/8211 : the additional signal being a sound signal H04N9/835 takes precedence
• CPC Class Codes: H04N 9/835 : involving processing of the sound signal

Abstract

The present technology concerns cell phones and similar devices, and their use in conjunction with media content (electronic and physical) and other systems (e.g., televisions, digital video recorders, and electronic program directories). Some aspects of the technology particularly concern “second screen” applications that sense a television program being watched by a user, and present menus of complementary content on the phone touchscreen from which the user can select. This complementary content can include other video content, associated web pages, opportunities to buy merchandise related to the program, etc. This complementary content can be identified by a provider of the television program, or can be identified otherwise (e.g., by crowd-sourcing). In some embodiments, the phone instructs a remote DVR to record content of interest for later viewing. The technology also provides features for making TV watching a social experience—involving remote friends. A great number of other arrangements and details are also disclosed.

Claims

1. A method comprising the acts:

transforming data sampled from an audio-visual program content of a first type, rendered by a first device to a user, by a watermark extraction process executing on a hardware processor, to yield watermark payload data, said watermark extraction process including the acts;

discerning a watermark calibration signal in the sampled data;

adjusting the sampled data in accordance with the discerned watermark calibration signal;

extracting a series of raw bits by demodulating the adjusted sample data with a carrier signal, said raw bits redundantly encoding a watermark payload; and

decoding the watermark payload data from the extracted series of raw bits;

submitting said watermark payload data to a database and, in response, obtaining metadata about said audio-visual program content;

using said metadata, identifying plural alternative sources for audio-visual program content of said first type;

selecting between said identified plural alternative sources based on stored rule data; and

arranging for user viewing of the audio-visual program content of said first type, from said selected source.

2. The method of claim 1 in which the stored rule data expresses a preference of the user regarding at least one of:

(a) streaming media content vs. downloaded;

(b) free content vs. paid;

(c) commercials;

(d) video resolution;

(e) delivery from sources that may be unauthorized; and

(f) source.

3. The method of claim 1 that includes searching electronic program guide information using said metadata, to identify said plural sources for the audio-visual program content of the first type, wherein the user needs neither to recognize the rendered audio-visual program content so that it can be located in electronic program guide information, nor select from among said alternative sources, both such acts being performed otherwise.

4. The method of claim 1 in which said arranging act comprises instructing a content recording system to record audio-visual content, so that the user can later view the audio-visual content recorded using said system.

5. The method of claim 1 in which the act of obtaining metadata comprises receiving information corresponding to ambient audio sampled by a user device from the rendered audio-visual program content, said user device being different than the first device, and using the received information—

in conjunction with fingerprint or watermark data in a first database—

to identify metadata about the rendered audio-visual program content.

6. The method of claim 1 in which the program content of the first type comprises an episode from a program series that includes plural episodes, and the method includes:

using said metadata to identify plural sources for another episode in said series;

selecting between said plural sources for said another episode, based on the stored rule data; and

arranging for the viewer to view said another episode, from said selected source.

7. The method of claim 1 in which the program content of the first type comprises a sports game of a first type, and the method includes:

using said metadata to identify plural sources for another sports game of said first type;

selecting between said plural sources for said another sports game of said first type, based on the stored rule data; and

arranging for the viewer to view said another sports game of said first type, from said selected source.

8. The method of claim 1 in which the program content of the first type comprises a sports game of a first type, involving two competing teams, and the method includes:

using said metadata to identify plural sources for another sports game of said first type involving one of said teams;

selecting between said plural sources for said another sports game of said first type involving one of said teams, based on the stored rule data; and

arranging for the viewer to view said another sports game of said first type involving one of said teams, from said selected source.

9. The method of claim 1 in which the program content of the first type comprises a sporting event involving a competitor, and the method includes:

using said metadata to identify plural sources for another sporting event involving said competitor;

selecting between said plural sources for said another sporting event involving said competitor, based on the stored rule data; and

arranging for the viewer to view said another sporting event involving said competitor, from said selected source.

10. The method of claim 2 in which the stored rule data expresses a preference of the user regarding at least two of (a)-(f).

11. The method of claim 2 in which the stored rule data expresses a preference of the user regarding at least three of (a)-(f).

12. The method of claim 2 in which the stored rule data indicates one of (a)-(f) that should be given priority over another of (a)-(f) in selecting the source.

13. A method in which a user device samples audio from audio-visual program content being delivered from a first networked source to a first rendering device, and rendered by said first device to a user, the method comprising the acts:

transforming the sampled audio data by a watermark extraction process to yield watermark payload data, using a hardware processor configured to perform such transforming, said watermark extraction process including the acts;

discerning a watermark calibration signal in the sampled audio data;

adjusting the sampled audio data in accordance with the discerned watermark calibration signal;

extracting a series of raw bits by demodulating the adjusted sample data with a carrier signal, said raw bits redundantly encoding a watermark payload; and

decoding the watermark payload data from the extracted series of raw bits;

submitting said watermark payload data to a database and, in response, obtaining metadata about said audio-visual program content, based on the sample audio; and

using the metadata in arranging delivery of said audio-visual content to the user from a second networked source different than the first networked source, said second networked source being selected from among plural networked sources, based on user preference data, the user preference data concerning a preference of the user regarding at least one of;

(a) streaming media content vs. downloaded;

(b) free content vs. paid;

(c) commercials;

(d) video resolution;

(e) delivery from sources that may be unauthorized; and

(f) source.

14. The method of claim 13 in which the user preference data concerns a preference of the user regarding at least two of (a)-(f).

15. The method of claim 13 in which the user preference data concerns a preference of the user regarding at least three of (a)-(f).

16. The method of claim 13 in which the user preference data indicates one of (a)-(f) that should be given priority over another of (a)-(f) in selecting from among the plural networked sources.

17. The method of claim 13 in which said act of arranging delivery comprises arranging delivery of said audio-visual program content to said user device.

18. A method comprising the acts:

transforming audio data sampled from first audio-visual program content that is rendered by a first device to a user and that comprises a sporting event of a first type involving first and second competitors, by a watermark extraction process to yield watermark payload data, using a hardware processor configured to perform such transforming, said watermark extraction process including the acts;

discerning a watermark calibration signal in the audio data;

adjusting the audio data in accordance with the discerned watermark calibration signal;

extracting a series of raw bits by demodulating the adjusted audio data with a carrier signal, said raw bits redundantly encoding a watermark payload; and

decoding the watermark payload data from the extracted series of raw bits;

submitting said watermark payload data to a database and, in response, obtaining metadata about said audio-visual program content; and

based on said metadata, identifying related second audio-visual program content, said second content comprising;

another sporting event of said first type involving one of said competitors;

a sporting event involving said first and second competitors, but being of an event type different than said first type;

wherein said identifying is performed by a hardware processor configured to perform such act.

19. The method of claim 18 that further includes:

identifying plural alternative sources for the related second audio-visual program content;

selecting between said identified plural alternative sources based on stored rule data, said selecting being performed by a hardware processor configured to perform such act; and

arranging for the user to view the related second audio-visual program content, from said selected source.