Annotation of metadata through capture infrastructure

• US 9,576,473 B2
• Filed: 09/04/2013
• Issued: 02/21/2017
• Est. Priority Date: 11/26/2008
• Status: Active Grant

• Current Assignee: Samba TV Incorporated
• Original Assignee: Free Stream Media Corporation
• Inventors: Navin, Ashwin, Harrison, David, Smallman, Josh
• Primary Examiner(s): Ibrahim, Mohamed
• Application Number:
  US14/018,396
• Publication Number:
  US 20140007156A1
• Time in Patent Office: 1,266 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 2221/033 Test or assess software
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 2101/668 Internet protocol [IP] address subnets
• CPC Class Codes: H04L 61/2514 between local and global IP addresses
• CPC Class Codes: H04L 61/2517 using port numbers
• CPC Class Codes: H04L 61/256 NAT traversal
• CPC Class Codes: H04L 61/2575 using address mapping retrieval, e.g. simple traversal of user datagram protocol through session traversal utilities for NAT [STUN]
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 61/4541 Directories for service discovery
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/61 : for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/141 : Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04N 21/23424 : involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2541 : Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/278 : Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/4126 : The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41407 : embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4147 : PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4415 : using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/466 : Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/6175 : involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/835 : Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 21/8352 : involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/8358 : involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A method, apparatus and system annotation of meta-data through a capture infrastructure are disclosed. In one embodiment, a method of a client device includes applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data. The client device then associates the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and/or the content identifier. A capture infrastructure annotates the audio-visual data with a brand name and/or a product name by comparing entries in the master database with a closed captioning data of the audio-visual data and/or through an application of an optical character recognition algorithm in the audio-visual data. The content identifier may involve a music identification, an object identification, a facial identification, and/or a voice identification. A minimal functionality including accessing a tuner and/or a stream decoder that identifies a channel and/or a content may be found in the networked media device. The networked media device may produce an audio fingerprint and/or a video fingerprint that is communicated with the capture infrastructure.

Claims

1. A method of a client device comprising:

applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data; and

associating the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier,

wherein a capture infrastructure annotates the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;

wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,

wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked media device,

wherein the networked media device produces at least one of an audio fingerprint and a video fingerprint that are communicated with the capture infrastructure,

wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,

wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,

wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data,

wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,

wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,

wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,

wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,

wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on the client device is paired with the networked media device when the client device is communicatively coupled with the networked media device,

wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked media device communicates each public IP address with any paired entity to the capture infrastructure, and

wherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement in the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.

2. The method of claim 1 further comprising:

referencing an additional metadata comprising at least one of the content identifier and the advertisement based on a video processing algorithm,

wherein the additional meta data is at least one of a title, a description, a thumbnail, a name of an individual, and a historical data, and

wherein the additional metadata is determined from a browser history captured from the client device based on a capture policy, and correlating a relevance of the browser history with at least one of the content identifier and the advertisement;

constraining an executable environment in a security sandbox;

executing a sandboxed application in the executable environment using a processor and a memory;

automatically instantiating a connection between the sandboxed application and the unannounced device associated with the networked media device based on the determination that the internet protocol address of the port from the unannounced device is associated with the networked media device;

processing an identification data associated with the sandbox reachable service sharing a public address with the client device;

determining a private address pair of the sandbox reachable service based on the identification data;

establishing a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of the security sandbox; and

appending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.

3. The method of claim 2:

wherein the client device and the networked media device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device communicate with each other through a relay service employed by a pairing server having a discovery module and a relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

4. The method of claim 3:

wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked media device when;

a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receive communications from a public IP address of a different network on which the sandbox reachable service operates, and

wherein a second NAT device coupled with the different network on which the networked media device operates to translate the private IP address of the networked media device to the public IP address visible to the sandboxed application.

5. The method of claim 4:

wherein the networked media device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked media device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

wherein the networked media device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, and a multidimensional visual presentation device.

6. The method of claim 5 further comprising:

utilizing at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service; and

optimizing a polling period between polling such that it is less than a timeout period of a session through the relay service.

7. The method of claim 6 further comprising:

initiating the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server,

and a form having a confirmation dialog that is submitted back to the pairing server, and

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.

8. The method of claim 2 further comprising:

extending the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox;

and bypassing a pairing server having the discovery algorithm and the relay algorithm when establishing the connection between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module and

the relay module added to the security sandbox.

9. The method of claim 8 further comprising:

applying the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device;

and applying the relay algorithm of the security sandbox to establish the connection between the sandboxed application and the sandbox reachable service of the networked media device.

10. The method of claim 9:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.

11. The method of claim 2 further comprising:

wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,

wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, and an Asynchronous Javascript® and

a XML (AJAX) application, and

wherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked media device.

12. The method of claim 11 further comprising:

utilizing an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the connection between the sandboxed application and the sandbox reachable service.

13. The method of claim 2 further comprising:

accessing a paring server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,

wherein the pairing server performs a discovery lookup of any device that has announced that it shares the public address associated with the client device,

and wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

14. The method of claim 13 further comprising at least one of:

wherein the sandbox reachable service announces an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,

wherein the range of public addresses is known by the pairing server so that the announcement of the availability of the sandbox reachable service across the range of public addresses is unnecessary,

wherein the sandbox reachable service communicates at least one of a global unique identifier and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service, and

wherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.

15. The method of claim 2 further comprising:

eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device when the connection is established, wherein the shared network is at least one of the local area network, a multicast network, an anycast network, and a multilan network;

minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established; and

improving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the connection is established.

16. The method of claim 2:

wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,

wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, and

wherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.

17. A method of a networked device comprising:

applying an automatic content recognition algorithm to determine a content identifier of an audio-visual data; and

associating the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier,

wherein a capture infrastructure annotates the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;

wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,

wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked device,

wherein the networked device produces at least one of an audio fingerprint and a video fingerprint that are communicated with the capture infrastructure,

wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,

wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,

wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions m the audio-visual data,

wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,

wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,

wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,

wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,

wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on a client device is paired with the networked device when the client device is communicatively coupled with the networked device,

wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked device communicates each public IP address with any paired entity to the capture infrastructure, and

wherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement in the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.

18. The method of claim 17 further comprising:

accessing a pairing server when processing an identification data associated with a sandbox reachable service of the networked device that shares a public address with a client device,

wherein the pairing server performs a discovery lookup of any device that has announced that it shares the public address associated with the client device,

and wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service,

appending a header of a hypertext transfer protocol to permit the networked device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm, and

wherein the client device to operate in at least one manner such that the client device;

to process an identification data associated with the sandbox reachable service sharing a public address with the client device;

to determine a private address pair of the sandbox reachable service based on the identification data; and

to establish a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of a security sandbox.

19. The method of claim 18 wherein the client device:

to extend a security sandbox with a discovery algorithm and a relay algorithm through the discovery module and the relay module added to the security sandbox; and

to bypass a paring server having the discovery algorithm and the relay algorithm when establishing the communication between the sandboxed application and the sandbox reachable service when the security is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

20. The method of claim 19 wherein the client device:

to apply the discovery algorithm of the security sandbox to determine that the networked device having the sandbox reachable service communicates in a shared network common to the client device and the networked device; and

to apply the relay algorithm of the security sandbox to establish the communication between the sandboxed application and the sandbox reachable service of the networked device.

21. The method of claim 20:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked device.

22. The method of claim 21:

wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,

wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, and

wherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.

23. The method of claim 22:

wherein the client device and the networked device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

24. The method of claim 23:

wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked device when;

a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receives communications from a public IP address of a different network on which the sandbox reachable service operates, and

wherein a second NAT device coupled with the different network on which the networked device operates to translates the private IP address of the networked device to the public IP address visible to the sandboxed application.

25. The method of claim 24:

wherein the networked device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

wherein the networked device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, and

a multidimensional visual presentation device.

26. The method of claim 25 wherein the client device:

to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service; and

to optimize a polling period between polling such that it is less than a timeout period of a session through the relay service.

27. The method of claim 26 wherein the client device:

to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server,

and a form having a confirmation dialog that is submitted back to the pairing server, and

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.

28. The method of claim 18 wherein the client device:

to access a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,

wherein the pairing server performs a discovery lookup of any devices that have announced that they share the public address associated with the client device, and

wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

29. The method of claim 28 further comprising at least one of:

announcing an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses; and

communicating at least one of a global unique identifier, a hardware address, and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service, and

wherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.

30. The method of claim 28 further comprising:

eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked device when the communication is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;

minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the communication is established; and

improving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the communication is established.

31. The method of claim 18:

wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in a security sandbox,

wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, and an Asynchronous Javascript® and

a XML (AJAX) application, and

wherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked device.

32. The method of claim 31 wherein the client device:

to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the communication between the sandboxed application and the sandbox reachable service.

33. A system comprising:

a networked device and a client device to apply an automatic content recognition algorithm to determine a content identifier of an audio-visual data and to associate the content identifier with an advertisement data based on a semantic correlation between a meta-data of the advertisement provided by a content provider and the content identifier; and

a capture infrastructure to annotate the audio-visual data with at least one of a brand name and a product name by comparing entries in the master database with at least one of a closed captioning data of the audio-visual data and through an application of an optical character recognition algorithm in the audio-visual data;

wherein the content identifier is at least one of a music identification, an object identification, a facial identification, and a voice identification,

wherein a minimal functionality comprising accessing at least one of a tuner and a stream decoder that identifies at least one of a channel and a content is found in the networked media device,

wherein the networked media device produces at least one of an audio fingerprint and a video fingerprint that are communicated with a capture infrastructure,

wherein the capture infrastructure compares at least one of the audio fingerprint and the video fingerprint with a master database,

wherein the capture infrastructure annotates the audio-visual data with a logo name by comparing entries in the master database with a logo data of the audio-visual data identified using a logo detection algorithm,

wherein the capture infrastructure automatically divides the audio-visual data into a series of scenes based on a sematic grouping of actions in the audio-visual data,

wherein the audio-visual data is analyzed in advance of a broadcast to determine content identifiers associated with each commercial in the audio-visual data such that advertisements are pre-inserted into the audio-visual data prior to broadcast,

wherein the capture infrastructure applies a time-order algorithm to automatically match advertisements to the audio-visual data when a correlation pattern is identified by the capture infrastructure with other audio-visual content previously analyzed,

wherein the capture infrastructure includes a buffer that is saved to a persistent storage and for which a label is generated to facilitate identification of reoccurring sequences,

wherein a post processing operation is at least one of automated through a post-processing algorithm and a crowd-sourced operation using a plurality of users in which a turing test is applied to determine a veracity of an input,

wherein a device pairing algorithm is used in which a cookie data associated with a web page visited by the user stored on a browser on the client device is paired with the networked media device when the client device is communicatively coupled with the networked media device,

wherein a transitive public IP matching algorithm is utilized in which at least one of the client device and the networked media device communicates each public IP address with any paired entity to the capture infrastructure, and

wherein a tag that is unconstrained from a same-origin policy is used to automatically load the advertisement m the browser, wherein the tag is at least one of an image tag, a frame, a iframe, and a script tag.

34. The system of claim 33 wherein the client device:

to extend the security sandbox with a discovery algorithm and a relay algorithm through the discovery module and a relay module added to the security sandbox,

and to bypass a paring server having the discovery algorithm and the relay algorithm when establishing the connection between the sandboxed application and the sandbox reachable service when the security is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

35. The system of claim 34 wherein the client device:

to apply the discovery algorithm of the security sandbox to determine that the networked device having the sandbox reachable service communicates in a shared network common to the client device and the networked device, and

to apply the relay algorithm of the security sandbox to establish the connection between the sandboxed application and the sandbox reachable service of the networked device.

36. The system of claim 35:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour* protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network (LAN) based protocol that discovers services in a LAN based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked device.

37. The system of claim 36:

wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,

wherein the remote access token identifies at least one of a set of communicable private Internet Protocol (IP) addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, and

wherein the client device can communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.

38. The system of claim 37:

wherein the client device and the networked device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

39. The system of claim 38:

wherein the trusted communication is facilitated in a manner such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked device when;

a first Network Address Translator (NAT) device coupled with a network on which the client device operates to receives communications from a public IP address of a different network on which the sandbox reachable service operates, and

wherein a second NAT device coupled with the different network on which the networked device operates to translates the private IP address of the networked device to the public IP address visible to the sandboxed application.

40. The system of claim 39:

wherein the networked device comprises a plurality of sandbox reachable applications including the sandbox reachable application, and wherein a service agent module of the networked device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

wherein the networked device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, and

a multidimensional visual presentation device.

41. The system of claim 40 wherein the client device:

to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service, and

to optimize a polling period between polling such that it is less than a timeout period of a session through the relay service.

42. The system of claim 41 wherein the client device:

to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server, and a form having a confirmation dialog that is submitted back to the pairing server, and

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.

43. The system of claim 33 wherein the client device:

to access a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,

wherein the pairing server performs a discovery lookup of any device that have announced that they share the public address associated with the client device, and

wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

44. The system of claim 43 wherein the networked device to at least one of:

announce a sandbox reachable service of the networked device to a discovery module using a processor and memory,

announce an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,

communicate at least one of a global unique identifier and an alphanumeric name to the pairing server along with at least one of a hardware address associated with the networked device, a public address pair associated with a sandbox reachable service of the networked device, and a private address pair associated with the sandbox reachable service of the networked device, and

wherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.

45. The system of claim 44 wherein the client device:

to eliminate a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked device when the connection is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;

to minimize a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the connection is established;

and to improve privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked device when the connection is established.

46. The system of claim 33:

wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,

wherein the sandboxed application comprises at least one of a markup language application such as a HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, and an Asynchronous Javascript® and

a XML (AJAX) application, and

wherein an access control algorithm governs a policy through which a secondary authentication is required when establishing a communication between the sandboxed application and the networked device.

47. The system of claim 46 wherein the client device:

to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the connection between the sandboxed application and the sandbox reachable service.

48. The system of claim 33:

wherein the communication session is established by appending a header of a hypertext transfer protocol to permit the networked device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is either one of a origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.