METADATA ACQUISITION USING EMBEDDED WATERMARKS

• US 20150324947A1
• Filed: 07/21/2015
• Published: 11/12/2015
• Est. Priority Date: 03/13/2014
• Status: Active Grant

• Current Assignee: Verance Corporation
• Original Assignee: Verance Corporation
• Inventors: Winograd, Joseph M., Zhao, Jian, Petrovic, Rade
• Granted Patent: US 10,504,200 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: H04L 2101/35 containing special prefixes
• CPC Class Codes: H04L 61/302 Administrative registration, e.g. for domain names at internet corporation for assigned names and numbers [ICANN]
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/561 Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/5683 Storage of data provided by user terminals, i.e. reverse caching
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82

Abstract

Methods, devices and computer program products are described that improve acquisition of metadata that is associates with a multimedia content, such an audio or video content. In one exemplary method, the multimedia content is received at a first device that is equipped with a watermark detector and watermark extraction operations are conducted to detect a plurality of watermark messages. Each watermark message includes a server code and an interval code. In this method, a server Internet Protocol (IP) address is obtained that corresponds to a server that stores at least a portion of the metadata associated with the multimedia content. The IP address corresponds to at least one of the server codes detected from the plurality of watermark messages. According to the above method, the metadata is requested from the server having the server IP address, and a response including the metadata associated with the multimedia content is received.

Claims

1. A method for acquiring metadata for a multimedia content, comprising:

receiving the multimedia content at a first device equipped with a watermark detector that is implemented at least partially using electronic circuitry;

conducting watermark extraction operations using the watermark detector to detect a plurality of watermark messages from the multimedia content, each watermark message including a server code and an interval code;

obtaining a server Internet Protocol (IP) address corresponding to a server that stores at least a portion of the metadata associated with the multimedia content, the IP address corresponding to at least one of the server codes detected from the plurality of watermark messages;

requesting the metadata from the server having the server IP address; and

receiving a response including the metadata associated with the multimedia content.

2. The method of claim 1, wherein obtaining the sever IP address and requesting the metadata are carried out using a combined lookup and query request that includes a first section and a second section,

the first section including a lookup portion of the combined lookup and query request and including the at least one of the server codes detected from the plurality of watermark messages, and

the second section including a request portion of the combined lookup and query request formed as a hypertext transfer protocol (HTTP) request and including the at least one of the interval codes detected from the plurality of watermark messages.

3. The method of claim 1, wherein each watermark message further includes a trigger symbol, and wherein a change in at least one trigger symbol in the plurality of detected watermark messages causes a new request for metadata to be initiated.

4. The method of claim 1, wherein the response includes all of the following:

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries the at least one of the interval codes detected from the plurality of watermark messages;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a next query time value indicative of an earliest time at which a new query for an event or service is permitted to occur; and

a next query slack indicative of a time duration during which a new query is allowed to occur.

5. The method of claim 1, wherein the response includes one or more of:

a specific server code that is identical to one of the server codes detected from the plurality of watermark messages;

a specific interval code that is identical to one of the interval codes detected from the plurality of watermark messages;

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries one of the interval codes or server codes detected from the plurality of watermark messages;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a deactivation time value indicative of time at which an application or service associated with at least one segment of the multimedia content is terminated;

a next query time indicative of an earliest time at which a new query for an event or service is permitted to occur;

or

a next query slack indicative of a time duration during which a new query is allowed to occur.

6. The method of claim 1, further comprising:

using the watermark detector to detect an additional set of plurality of watermark messages embedded in the content, the additional set of plurality of watermark messages corresponding to a second watermarking layer that is distinct from a first watermarking layer that includes the plurality of watermark messages, the second plurality of watermark messages having performance requirements or properties that are different from the plurality of watermark messages in the first watermarking layer.

7. The method of claim 6, wherein the additional set of plurality of watermark messages include a second server code and a second interval code that differ respectively from at least one of the server codes and the interval codes of the plurality of watermarks of the first watermarking layer.

8. The method of claim 6, wherein the performance requirements or properties that differ between the plurality of the watermarks of the first watermarking layer and the additional set of plurality of watermarks of the second watermarking layer include one or more of:

a payload size, a frequency, a throughput, a robustness, an error rate, an audio quality, or a metadata protocol.

9. A device, comprising:

a processor, and

a memory including processor executable code, the processor executable code upon execution by the processor configures the device to;

receive a multimedia content at the device;

conduct watermark extraction operations to detect a plurality of watermark messages from the multimedia content, each watermark message including a server code and an interval code;

obtain a server Internet Protocol (IP) address corresponding to a server that stores at least a portion of the metadata associated with the multimedia content, the IP address corresponding to at least one of the server codes detected from the plurality of watermark messages;

request the metadata from the server having the server IP address; and

receive a response including the metadata associated with the multimedia content.

10. The device of claim 9, wherein the sever IP address and request for the metadata are carried out through a combined lookup and query request that includes a first section and a second section,

the first section including a lookup portion of the combined lookup and query request and including the at least one of the server codes detected from the plurality of watermark messages, and

the second section including a request portion of the combined lookup and query request formed as a hypertext transfer protocol (HTTP) request and including the at least one of the interval codes detected from the plurality of watermark messages.

11. The device of claim 9, wherein each watermark message further includes a trigger symbol, and wherein the processor executable code upon execution by the processor configures the device to initiate a new request for metadata upon detection of a change in at least one trigger symbol in the plurality of detected watermark messages.

12. The device of claim 9, wherein the response includes all of the following:

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries the at least one of the interval codes detected from the plurality of watermark messages;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a next query time value indicative of an earliest time at which a new query for an event or service is permitted to occur, and

a next query slack indicative of a time duration during which a new query is allowed to occur.

13. The device of claim 9, wherein the processor executable code upon execution by the processor further configures the device to detect an additional set of plurality of watermark messages embedded in the content, the additional set of plurality of watermark messages corresponding to a second watermarking layer that is distinct from a first watermarking layer that includes the plurality of watermark messages, the second plurality of watermark messages having performance requirements or properties that are different from the plurality of watermark messages in the first watermarking layer.

14. The device of claim 13, wherein the additional set of plurality of watermark messages include a second server code and a second interval code that differ from at least on of the server codes and the interval codes of the plurality of watermarks of the first watermarking layer.

15. The device of claim 13, wherein the performance requirements or properties that differ between the plurality of the watermarks of the first watermarking layer and the additional set of plurality of watermarks of the second watermarking layer include one or more of:

a payload size, a frequency, a throughput, a robustness, an error rate, an audio quality, or a metadata protocol.

16. A computer program product, embodied on one or more non-transitory computer readable media, comprising:

program code for receiving the multimedia content at a first device equipped with a watermark detector that is implemented at least partially using electronic circuitry;

program code for conducting watermark extraction operations using the watermark detector to detect a plurality of watermark messages from the multimedia content, each watermark message including a server code and an interval code;

program code for obtaining a server Internet Protocol (IP) address corresponding to a server that stores at least a portion of the metadata associated with the multimedia content, the IP address corresponding to at least one of the server codes detected from the plurality of watermark messages;

program code for requesting the metadata from the server having the server IP address; and

program code for receiving a response including the metadata associated with the multimedia content.

17. A method for providing metadata associated with a multimedia content, comprising:

receiving a query at a metadata server, the query having been initiated by a requesting device coupled to a network, the query including an interval code having been obtained from a watermark message embedded in the multimedia content, the metadata server uniquely identifiable by an Internet Protocol (IP) address obtained based on a server code of the watermark message embedded in the multimedia content; and

transmitting a response to the query based on information in the received query, the response comprising at least a portion of metadata including all of the following;

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries the interval code;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a next query time value indicative of an earliest time at which a new query for an event or service is permitted to occur; and

a next query slack indicative of a time duration during which a new query is allowed to occur.

18. The method of claim 17, further comprising, in response to the query, providing an interactive content to the requesting device that is based on the received interval code.

19. The method of claim 17, further comprising receiving at least a portion of the metadata from an additional server that is associated with a server registrant device or a content provider device, and storing the at least a portion of the metadata at the metadata server.

20. The method of claim 19, further comprising receiving an interactive content from the additional server that is associated with the content provider device and storing the received interactive content at the metadata server.

21. A server device, comprising:

at least one processor;

a non-transitory data storage device configured to store one or more of metadata or interactive multimedia content; and

a memory, comprising processor executable code, the processor executable code when executed by one of the at least one processor configures to server device to;

receive a query at a metadata server, the query having been initiated by a requesting device coupled to a network, the query including an interval code having been obtained from a watermark message embedded in the multimedia content, the metadata server uniquely identifiable by an Internet Protocol (IP) address obtained based on a server code of the watermark message embedded in the multimedia content; and

transmit a response to the query based on information in the received query, the response comprising at least a portion of metadata including all of the following;

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries the interval code;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a next query time value indicative of an earliest time at which a new query for an event or service is permitted to occur; and

a next query slack indicative of a time duration during which a new query is allowed to occur.

22. The server device of claim 21, the processor executable code when executed by the at least one processor further configures to server device to, in response to the query, provide an interactive content to the requesting device that is based on the received interval code.

23. The server device of claim 21, wherein the processor executable code when executed by the at least one processor further configures to server device to receive at least a portion of the metadata from an additional server that is associated with a server registrant device or a content provider device, and store the at least a portion of the metadata at the metadata server.

24. The server device of claim 23, wherein the processor executable code when executed by the at least one processor further configures to server device to receive an interactive content from the additional server that is associated with the content provider device and store the received interactive content at the metadata server.

25. A computer program product, embodied on one or more non-transitory computer readable media, comprising:

program code for receiving a query at a metadata server, the query having been initiated by a requesting device coupled to a network, the query including an interval code having been obtained from a watermark message embedded in the multimedia content, the metadata server uniquely identifiable by an Internet Protocol (IP) address obtained based on a server code of the watermark message embedded in the multimedia content; and

program code for transmitting a response to the query based on information in the received query, the response comprising at least a portion of metadata including all of the following;

an Interval Start Time indicative of a starting time of a segment of the multimedia content that carries the interval code;

an identification value uniquely identifying a program that at least one watermarked segment of the multimedia content is part of;

a service identification value indicative of a service associated with at least one watermarked segment of the multimedia content;

an activation time value indicative of a stating time at which an application or service associated with the multimedia content is available for activation;

a next query time value indicative of an earliest time at which a new query for an event or service is permitted to occur; and

a next query slack indicative of a time duration during which a new query is allowed to occur.

26. A method for acquiring metadata for a multimedia content, comprising:

receiving the multimedia content at a first device equipped with a watermark detector that is implemented at least partially using electronic circuitry;

conducting watermark extraction operations using the watermark detector to detect a plurality of watermark messages from the multimedia content, each watermark message including a server code and an interval code;

obtaining a server Internet Protocol (IP) address corresponding to a server that stores at least a portion of the metadata associated with the multimedia content, the IP address corresponding to at least one of the server codes detected from the plurality of watermark messages;

requesting the metadata from the server having the server IP address;

at the server, receiving a request including the at least one of the detected interval codes and transmitting a response based on information in the received in the request to the first device; and

receiving, at the first device, a response from the server, the response including the metadata associated with the multimedia content that is formed based in-part on the at least one of the interval codes detected from the plurality of watermark messages.