Watermarking and signal recognition for managing and sharing captured content, metadata discovery and related arrangements

• US 9,454,789 B2
• Filed: 05/05/2014
• Issued: 09/27/2016
• Est. Priority Date: 05/03/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Lord, John D.
• Primary Examiner(s): Nakhjavan, Shervin
• Application Number:
  US14/270,163
• Publication Number:
  US 20150016661A1
• Time in Patent Office: 876 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 2201/005 Image watermarking
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

Content played at an event is identified using watermarking and/or other content recognition combined with contextual metadata, which facilitates identification and correlation with other content and metadata when it is posted to a network.

Claims

1. A method comprising:

in a network system, receiving items of content captured by audio or visual sensors of electronic devices of users, the items being uploaded to the network system by the electronic devices of the users, and the network system is configured to stream or download the received items to other users of the network system;

with one or more processors, executing instructions to perform the acts of;

obtaining information extracted from the received items of captured content, the information comprising an auxiliary payload signal encoded in a digital watermark extracted from audio or visual signals of the received items;

by reference to the obtained information, correlating the received items of captured content with each other based on the digital watermark to produce correlated received items; and

providing a network service that links together the correlated received items, the network service providing an aggregated presentation of the correlated received items, the aggregated presentation providing links to obtain by download or stream the correlated received items.

2. The method of claim 1, wherein the received items of captured content comprises audio content captured by microphones of the users at a live event, wherein each of the electronic devices captures instances of ambient audio at the live event, the instances each being embedded with the digital watermark, which is associated with the live event.

3. The method of claim 1, wherein the received items of captured content comprises imagery.

4. The method of claim 1, wherein the obtained information comprises a watermark ID encoded in the auxiliary payload signal of the digital watermark, which is embedded in audio content;

the watermark ID being embedded in captured live audio, which is captured from a live performance in electronic audio processing equipment;

the captured live audio being processed by the electronic audio processing equipment to embed the watermark ID and then output audio with the watermark ID embedded in the output to the users at the live performance with low latency; and

the watermark ID being linked to metadata identifying the live performance.

5. The method of claim 1, wherein the obtained information comprises information describing a characteristic of the received items of captured content, the characteristic being encoded in the auxiliary payload signal of the digital watermark.

6. The method of claim 1, wherein the obtained information comprises content fingerprints derived from the received items of captured content, the content fingerprints being used by the network system to identify the received items, as a distinct means to identify the received items from the digital watermark.

7. The method of claim 1, further comprising extracting the information from the received items of captured content.

8. The method of claim 1, wherein the act of correlating the received items of captured content is based on the obtained information, the obtained information comprising watermark IDs embedded in the received content items, the watermark IDs being associated with an event at which the received items are captured.

9. The method of claim 8, further comprising identifying at least one item of metadata associated with the watermark IDs from the received items, wherein the act of correlating the received items of captured content comprises correlating the received items with each other based on the at least one identified item of metadata.

10. The method of claim 1, further comprising synchronizing the received items of captured content with each other, and presenting the received items synchronized within the aggregated presentation, the aggregated presentation organizing the received items for access via download or stream based on the synchronizing.

11. The method of claim 1, further comprising, by reference to the obtained information, aggregating metadata associated with the obtained information to produce aggregated metadata, the aggregated metadata being aggregated from metadata later linked to the digital watermark after the received items are uploaded to the network site.

12. The method of claim 11, wherein the aggregated metadata comprises at least one item of content, the at least one item of content being associated with the received items by metadata indexed by a watermark ID in the digital watermark.

13. The method of claim 12, wherein the aggregated metadata comprises at least one item of metadata that is associated with the at least one item of content.

14. The method of claim 11, further comprising aggregating data that is associated with the aggregated metadata.

15. The method of claim 11, further comprising providing user device access to the aggregated metadata.

16. The method of claim 11, further including providing a site at which user devices obtain a graphical view of the aggregated metadata.

17. The method of claim 11, further including providing a site at which user devices obtain a graphical view of aggregated items of content.

18. The method of claim 17, wherein the aggregated items of content are synchronized by watermarks extracted from the aggregated items of content.