Content interaction methods and systems employing portable devices

• US 9,788,043 B2
• Filed: 11/14/2008
• Issued: 10/10/2017
• Est. Priority Date: 11/07/2008
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F., MacIntosh, Brian T., Conwell, William Y.
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): PEREZ FUENTES, LUIS M
• Application Number:
  US12/271,772
• Publication Number:
  US 20100119208A1
• Time in Patent Office: 3,252 Days
• Field of Search: 386291
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/372 Programme
• CPC Class Codes: H04H 60/61 Arrangements for services using the result of monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04H 60/91 Mobile communication networks for accessing the Internet H04H60/85
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4227 Providing Remote input by a user located remotely from the client device, e.g. at work
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

A portable device, such as a cell phone, is used to “forage” media content from a user'"'"'s environment. For example, it may listen to a television viewed by a traveler in an airport lounge. By reference to digital watermark or other data extracted from the content, the device can identify the television program, and enable a variety of actions. For example, the device may instruct a DVR to record the remainder of the television program—or determine when the program will be rebroadcast, and instruct the DVR to record the program in its entirety at that later time. The device may also identify content that preceded (or follows) the foraged content. Thus, a user who tunes-in just at the end of an exciting sporting event can capture one of the following commercials, identify the preceding program, and download same for later viewing. In other aspects, a cell phone can be used as a “second screen,” through which a user can interact with ambient content—such as reviewing electronic program guide data, or enjoying interactive television features. A great variety of other functions and arrangements are also detailed.

Claims

1. A method for remotely programming a video recorder, comprising the acts:

with a user-carried, microphone-equipped portable device, capturing ambient audio;

decoding plural-bit auxiliary information added to the audio as a steganographic digital watermark prior to its capture;

querying a data structure with at least part of the decoded information;

as a result of said querying, receiving information identifying an audio-visual program to which the captured audio corresponds;

with a user interface associated with the portable device, presenting information to the user identifying the audio-visual program, based on the received information; and

determining whether said audio-visual program is scheduled for availability at some point in the future on a content distribution system to which the video recorder has access;

presenting information about upcoming availability via the user interface; and

programming the video recorder to record the audio-visual program at said point in the future;

the method further including sending data corresponding to the captured ambient audio to a remote server, and decoding the plural-bit auxiliary information at the remote server.

2. The method of claim 1 wherein the decoding comprises decoding using a processor in the portable device.

3. The method of claim 1 that further includes receiving a user command entered through the user interface, and programming the video recorder to record the audio-visual program at said point in the future in accordance with the received user command.

4. A method comprising the acts:

with a user-carried, microphone-equipped portable device, capturing ambient audio;

decoding plural-bit auxiliary information added to the audio as a steganographic digital watermark prior to its capture;

querying a first data structure with at least part of the decoded information;

as a result of querying the first data structure, identifying metadata associated with content to which the captured audio corresponds;

querying a second data structure using certain of the identified metadata, the first and second data structures comprising the same data structure;

as a result of querying the second data structure, receiving information about availability of an audio-visual program to which the captured ambient audio relates;

with a user interface associated with the portable device, presenting information to the user based on at least some of the received information; and

arranging for the audio-visual program to be made available to the user;

wherein the method further includes presenting information to the user indicating that the audio-visual program is or will be available for delivery to the user on a physical medium, and the arranging comprises requesting such delivery.

5. The method of claim 4 that includes capturing the ambient audio from a promo for the audio-visual content.

6. The method of claim 4 that includes capturing the ambient audio from a rendering of the audio-visual content.

7. The method of claim 4 that includes presenting information to the user indicating that the audio-visual program will, at a future point, be available on a content distribution system to which a digital video recorder has access, and the arranging comprises programming the digital video recorder to record the program at said future point.

8. The method of claim 4 that includes presenting information to the user indicating that the audio-visual program is or will be available for delivery to the user, and the arranging comprises requesting such delivery.

9. A method comprising the acts:

with a user-carried, microphone-equipped portable device, capturing ambient audio;

decoding plural-bit auxiliary information added to the audio as a steganographic digital watermark prior to its capture;

querying a data structure with at least part of the decoded information;

as a result of said querying, receiving information identifying an audio-visual program to which the captured audio corresponds;

with a user interface associated with the portable device, presenting information to the user identifying the audio-visual program, based on the received information;

identifying a source from which at least a portion of the audio-visual program is, or will be, available;

presenting an index of the audio-visual program to the user as a series of informational panes, displayed in cover-flow fashion.

10. The method of claim 9 that further includes the user navigating to a desired point in the program using the series of informational panes, and selecting same;

and delivering the program to the user starting at the selected point.

11. A method involving a television, a laptop computer, and a mobile communications device, the method comprising the acts:

with the mobile communications device, capturing an excerpt of a program being rendered by the television;

processing the excerpt to derive a corresponding identifier;

querying a database with the derived identifier to identify a program identifier associated with the program being rendered by the television; and

by reference to the identified program identifier, presenting on the laptop a display of information related to the program being rendered by the television;

wherein the communications device helps to identify the television program, and thus cooperates with the laptop computer to display information related to the program.

12. The method of claim 11 that includes:

receiving the program identifier at the communications device;

transferring the program identifier from the communications device to the laptop computer;

sending a request for information from the laptop computer, the sent request including the program identifier; and

as a consequence of the sent request, receiving information at the laptop, and presenting said display based thereon.

13. A method involving a television, a first mobile communications device, and a second mobile communications device, the method comprising the acts:

with the first mobile communications device, capturing an excerpt of a program being rendered by the television;

processing the excerpt to derive a corresponding identifier;

querying a database with the derived identifier to identify a program identifier associated with the program being rendered by the television; and

by reference to the identified program identifier, presenting on the first device a first display of information related to the program being rendered by the television;

with the second mobile communications device, capturing an excerpt of the same program being rendered by the television;

processing the excerpt to derive a corresponding identifier;

querying a database with the derived identifier to identify a program identifier associated with the program being rendered by the television; and

by reference to the identified program identifier, presenting on the second device a second, different display of information related to the program being rendered by the television;

wherein users of the first and second devices are presented different information related to the program being rendered by the television.

14. A method involving a television system and a first mobile communications device, the television system and the device each including a screen, the method comprising the acts:

with the first mobile communications device, capturing audio or video data from the television system;

by reference to the captured data, determining an identifier; and

by reference to the identifier, presenting information to a user on a screen other than the television system screen, the presented information relating to operation of that particular television system.

15. The method of claim 14 in which the identifier is associated with the television system, and serves to identify same.

16. The method of claim 15 wherein the presented information comprises program guide information for programs delivered to the television system from a content distribution system.

17. The method of claim 15 wherein the presented information relates to operation of a digital video recorder portion of the television system.

18. The method of claim 15 that includes presenting the information to the user on a screen of a laptop computer.

19. The method of claim 15 that includes presenting the information to the user on a screen of the first mobile communication device.

20. A method for interacting with audio-visual content, the method comprising the acts:

with a mobile communications device, capturing an excerpt of a first program item being rendered by a television;

processing the excerpt to derive a corresponding identifier;

querying a database with the derived identifier to identify the first program item;

also identifying from a data structure at least one different second program item that preceded or follows the first program item; and

presenting to a user at least some of the information identified from the database, wherein said presenting comprises presenting to the user a graphical depiction of plural program items and their relative ordering.

21. The method of claim 20 in which the presenting comprises presenting at least the title of one of said program items on a screen of the mobile communications device.

22. The method of claim 20 in which the processing comprises processing an audio excerpt to decode plural-bit auxiliary data added to the audio as a steganographic digital watermark prior to its capture.

23. The method of claim 20 in which the processing comprises deriving fingerprint data from a captured audio excerpt.

24. The method of claim 20 in which the graphical depiction comprises a cover flow-like presentation of the program items and their sequence.

25. The method of claim 20 wherein said also identifying comprises identifying at least one different program item that preceded the first program item.

26. The method of claim 20 wherein said also identifying comprises identifying at least one different program item that follows the first program item.

27. A non-transitory computer readable medium containing software instructions for configuring a mobile communications device to interact with audio-visual content by acts including:

capturing an excerpt of a first program item being rendered by a television;

processing the excerpt to derive a corresponding identifier;

querying a database with the derived identifier to identify the first program item;

also identifying from a data structure at least one different second program item that preceded or follows the first program item; and

presenting to a user at least some of the information identified from the database, wherein said presenting comprises presenting to the user a graphical depiction of plural program items and their relative ordering.