Remotely control devices over a network without authentication or registration

• US 10,419,541 B2
• Filed: 12/07/2016
• Issued: 09/17/2019
• Est. Priority Date: 11/26/2008
• Status: Active Grant

• Current Assignee: Samba TV Incorporated
• Original Assignee: Free Stream Media Corporation
• Inventors: Harrison, David, Jantz-Sell, Chris
• Primary Examiner(s): Zand, Davoud A
• Application Number:
  US15/371,220
• Publication Number:
  US 20170085651A1
• Time in Patent Office: 1,014 Days
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 2221/2119 Authenticating web pages, e.g. with suspicious links
• CPC Class Codes: G06F 2221/2133 Verifying human interaction, e.g., Captcha
• CPC Class Codes: H04L 63/029 Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A method, apparatus and system related to zero-configuration remote control of device(s) coupled to a networked media device through a client side device communicatively coupled with the networked media device are disclosed. In one embodiment, a method of a client device includes determining that an internet protocol address and a port from an unannounced device is associated with a networked media device. The client device constrains an executable environment in a security sandbox. Then, the client device executes a sandboxed application in the executable environment using a processor and a memory. Next, the client device automatically instantiates a connection between the sandboxed application and the unannounced device associated with the networked media device based on the determination that the internet protocol address of the port from the unannounced device is associated with the networked media device. The unannounced device may utilize a web services interface and/or an infrared remote control interface. The networked media device may utilize an InfraRed (IR) blaster to associate with the unannounced device instead of the internet protocol address and the port when the unannounced device utilizes an infrared remote control interface.

Claims

1. A method comprising:

determining, through a client device, association between an unannounced device and a media device networked with the client device;

constraining an executable environment in a security sandbox within the client device;

executing a sandboxed application in the executable environment using a processor and a memory; and

enabling, through the sandboxed application, at least one of;

control of and communication with the unannounced device determined to be associated with the networked media device.

2. The method of claim 1, wherein at least one of:

the unannounced device is at least one of a set-top box (STB) that functions as a tuner for a content distribution service, a stereo, an HDMI-CEC protocol device, an AV-link, and a media player,

the sandboxed application operates as a remote control device through the communication between the sandboxed application and the unannounced device associated with the networked media device based on the determination of the association between the unannounced device and the networked media device,

wherein the client device and the unannounced device are on different networks and not directly communicatively coupled,

wherein the unannounced device utilizes at least one of a web services interface and an infrared remote control interface, and

wherein the networked media device utilizes an InfraRed (IR) blaster to associate with the unannounced device when the unannounced device utilizes the infrared remote control interface.

3. The method of claim 1, further comprising:

processing an identification data associated with a sandbox reachable service of the networked media device sharing a public address with the client device;

determining a private address pair of the sandbox reachable service based on the identification data;

establishing a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of the security sandbox; and

appending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is one of;

an origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.

4. The method of claim 3, further comprising:

accessing a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,

wherein the pairing server performs a discovery lookup of any device that has announced that it shares the public address associated with the client device, and

wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

5. The method of claim 4, further comprising associating the client device with the networked media device based on at least one of:

the sandbox reachable service announcing an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,

the pairing server knowing the range of public addresses so that the announcement of the availability of the sandbox reachable service across the range of public addresses is unnecessary,

the sandbox reachable service communicating at least one of a global unique identifier and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service, and

the private address pair including a private IP address and a port number associated with the sandbox reachable service.

6. The method of claim 3, further comprising:

eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;

minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device; and

improving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device.

7. The method of claim 3, wherein at least one of:

the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,

the sandboxed application comprises at least one of;

an HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, an Asynchronous Javascript® and

an XML (AJAX) application, and

an access control algorithm governs a policy through which a secondary authentication is required when establishing the communication session between the sandboxed application and the sandbox reachable service of the networked media device.

8. The method of claim 7, further comprising:

utilizing an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the communication session between the sandboxed application and the sandbox reachable service.

9. The method of claim 3, further comprising:

extending the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox; and

bypassing a pairing server having the discovery algorithm and the relay algorithm when establishing the communication session between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

10. The method of claim 9, further comprising:

applying the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device; and

applying the relay algorithm of the security sandbox to establish the communication session between the sandboxed application and the sandbox reachable service of the networked media device.

11. The method of claim 9:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour®

protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network based protocol that discovers services in a local area network based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.

12. The method of claim 3, further comprising at least one of:

utilizing a cookie associated with the security sandbox to store a remote access token on a storage of the client device;

identifying, through the remote access token, at least one of a set of communicable private IP addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device; and

enabling the client device to communicate with the sandbox reachable services that previously operated on the common shared network through the remote access token.

13. The method of claim 3:

wherein the client device and the networked media device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, and an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device communicate with each other through a relay service employed by a pairing server having a discovery module and a relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

14. The method of claim 13:

wherein the trusted communication is facilitated such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked media device when;

a first Network Address Translator (NAT) device coupled with a network on which the client device operates receives communications from a public IP address of a different network on which the sandbox reachable service operates, and

a second NAT device coupled with the different network on which the networked media device operates translates the private IP address of the networked media device to the public IP address visible to the sandboxed application.

15. The method of claim 13, wherein at least one of:

the networked media device comprises a plurality of sandbox reachable applications including the sandbox reachable service,

a service agent module of the networked media device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

the networked media device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, and a multidimensional visual presentation device.

16. The method of claim 13, further comprising:

utilizing at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service; and

optimizing a polling period between polling to be less than a timeout period of a session through the relay service.

17. The method of claim 13, further comprising:

initiating the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server, and a form having a confirmation dialog that is submitted back to the pairing server,

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.

18. A method comprising:

associating, through a networked media device, an unannounced device with the networked media device;

enabling, through a sandbox reachable service of the networked media device, association between the networked media device and a client device by way of a sandboxed application executing on the client device, the client device;

constraining an executable environment in a security sandbox therein; and

executing the sandboxed application in the executable environment using a processor and a memory; and

enabling, through the association of the sandbox reachable service and the sandboxed application and the association of the unannounced device with the networked media device, the client device to at least one of;

control the unannounced device and communicate therewith through the sandboxed application.

19. The method of claim 18:

wherein the unannounced device is at least one of a set-top box (STB) that functions as a tuner for a content distribution service, a stereo, an HDMI-CEC protocol device, an AV-link, and a media player,

wherein the sandboxed application operates as a remote control device through the communication between the sandboxed application and the unannounced device associated with the networked media device,

wherein the client device and the unannounced device are on different networks and not directly communicatively coupled,

wherein the unannounced device utilizes at least one of a web services interface and an infrared remote control interface, and

wherein the networked media device utilizes an InfraRed (IR) blaster to associate with the unannounced device when the unannounced device utilizes the infrared remote control interface.

20. The method of claim 18, further comprising:

appending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is one of;

an origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm,

wherein the client device operates;

to process an identification data associated with the sandbox reachable service sharing a public address with the client device,

to determine a private address pair of the sandbox reachable service based on the identification data, and

to establish a communication session between the sandboxed application and the sandbox reachable service using a cross-site scripting technique of the security sandbox.

21. The method of claim 20, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is configured:

to access a pairing server when processing the identification data associated with the sandbox reachable service sharing the public address with the client device,

wherein the pairing server performs a discovery lookup of any devices that have announced that they share the public address associated with the client device, and

wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

22. The method of claim 21, further comprising at least one of:

announcing an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses; and

communicating at least one of a global unique identifier, a hardware address, and an alphanumeric name to the pairing server along with the private address pair of the sandbox reachable service,

wherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.

23. The method of claim 20, further comprising:

eliminating a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device when the communication session is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network;

minimizing a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the communication session is established; and

improving privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the communication session is established.

24. The method of claim 20:

wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in a security sandbox,

wherein the sandboxed application comprises at least one of a HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, an Asynchronous Javascript® and

an XML (AJAX) application, and

wherein an access control algorithm governs a policy through which a secondary authentication is required when establishing the communication session between the sandboxed application and the networked media device.

25. The method of claim 24, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is further configured:

to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the communication session between the sandboxed application and the sandbox reachable service.

26. The method of claim 20, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is further configured:

to extend the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox, and

to bypass a pairing server having the discovery algorithm and the relay algorithm when establishing the communication session between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

27. The method of claim 26, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is further configured:

to apply the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device, and

to apply the relay algorithm of the security sandbox to establish the communication between the sandboxed application and the sandbox reachable service of the networked media device.

28. The method of claim 26:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour®

protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network based protocol that discovers services in a local area network based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.

29. The method of claim 20, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service:

a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,

the remote access token identifies at least one of a set of communicable private IP addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, and

the client device is capable of communicating with the sandbox reachable services that previously operated on the common shared network through the remote access token.

30. The method of claim 26:

wherein the client device and the networked media device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, and an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

31. The method of claim 30:

wherein the trusted communication is facilitated such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked media device when;

a first NAT device coupled with a network on which the client device operates receives communications from a public IP address of a different network on which the sandbox reachable service operates, and

a second NAT device coupled with the different network on which the networked media device operates translates the private IP address of the networked media device to the public IP address visible to the sandboxed application.

32. The method of claim 30, wherein at least one of:

the networked media device comprises a plurality of sandbox reachable applications including the sandbox reachable service,

a service agent module of the networked media device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

the networked media device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, and a multidimensional visual presentation device.

33. The method of claim 30, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is further configured:

to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service, and

to optimize a polling period between polling to be less than a timeout period of a session through the relay service.

34. The method of claim 30, wherein, in accordance with the communication session established between the sandboxed application and the sandbox reachable service, the client device is further configured:

to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server, and a form having a confirmation dialog that is submitted back to the pairing server,

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.

35. A system comprising:

a client device constraining an executable environment in a security sandbox therewithin and executing a sandboxed application in the executable environment; and

a media device networked with the client device, the networked media device associating an unannounced device therewith,

wherein at least one of;

the client device and the networked media device is configured to enable, through the sandboxed application, the client device to at least one of;

control the unannounced device and communicate therewith in accordance with the association of the unannounced device with the networked media device and association of the client device with the networked media device by way of the sandboxed application.

36. The system of claim 35:

wherein the unannounced device is at least one of a set-top box (STB) that functions as a tuner for a content distribution service, a stereo, an HDMI-CEC protocol device, an AV-link, and a media player,

wherein the sandboxed application operates as a remote control device through the communication between the sandboxed application and the unannounced device associated with the networked media device,

wherein the client device and the unannounced device are on different networks and not directly communicatively coupled,

wherein the unannounced device utilizes at least one of a web services interface and an infrared remote control interface, and

wherein the networked media device utilizes an InfraRed (IR) blaster to associate with the unannounced device when the unannounced device utilizes the infrared remote control interface.

37. The system of claim 35:

wherein a communication session is established between the sandboxed application and a sandbox reachable service of the networked media device by appending a header of a hypertext transfer protocol to permit the networked media device to communicate with the sandboxed application as a permitted origin domain through a Cross-origin resource sharing (CORS) algorithm, wherein the header is one of;

an origin header when the CORS algorithm is applied and a referrer header in an alternate algorithm.

38. The system of claim 37, wherein the client device is further configured:

to access a pairing server when processing identification data associated with the sandbox reachable service sharing a public address with the client device,

wherein the pairing server performs a discovery lookup of any device that has announced sharing of the public address associated with the client device, and

wherein the sandbox reachable service announces itself to the pairing server prior to the establishment of the communication session between the sandboxed application and the sandbox reachable service.

39. The system of claim 38, wherein the networked media device is configured to at least one of:

announce the sandbox reachable service of the networked media device to a discovery module using a processor and memory,

announce an availability of the sandbox reachable service across a range of public addresses such that the sandboxed application communicates with the sandbox reachable service in any one of the range of the public addresses,

communicate at least one of a global unique identifier and an alphanumeric name to the pairing server along with at least one of a hardware address associated with the networked media device, a public address pair associated with the sandbox reachable service of the networked media device, and a private address pair associated with the sandbox reachable service of the networked media device,

wherein the private address pair includes a private IP address and a port number associated with the sandbox reachable service.

40. The system of claim 37, wherein the client device is configured:

to eliminate a communication through a centralized infrastructure when the sandboxed application and the sandbox reachable service communicate in a shared network common to the client device and the networked media device when the communication session is established, wherein the shared network is at least one of a local area network, a multicast network, an anycast network, and a multilan network,

to minimize a latency in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the communication session is established; and

to improve privacy in the communication session when the sandboxed application and the sandbox reachable service communicate in the shared network common to the client device and the networked media device when the communication session is established.

41. The system of claim 37:

wherein the sandboxed application is at least one of a web page, a script, a binary executable, an intermediate bytecode, an abstract syntax tree, and an executable application in the security sandbox,

wherein the sandboxed application comprises at least one of a HyperText Markup Language 5 (HTML5) application, a Javascript®

application, an Adobe®

Flash®

application, a Microsoft®

Silverlight®

application, a JQuery®

application, an Asynchronous Javascript® and

an XML (AJAX) application, and

wherein an access control algorithm governs a policy through which a secondary authentication is required when establishing the communication session between the sandboxed application and the networked media device.

42. The system of claim 41, wherein the client device is further configured:

to utilize an exception to a same origin policy through a use of at least one of a hyperlink, a form, the script, a frame, a header, and an image when establishing the communication session between the sandboxed application and the sandbox reachable service.

43. The system of claim 37, wherein the client device is further configured:

to extend the security sandbox with a discovery algorithm and a relay algorithm through a discovery module and a relay module added to the security sandbox, and

to bypass a pairing server having the discovery algorithm and the relay algorithm when establishing the communication session between the sandboxed application and the sandbox reachable service when the security sandbox is extended with the discovery algorithm and the relay algorithm through the discovery module and the relay module added to the security sandbox.

44. The system of claim 43, wherein the client device is further configured:

to apply the discovery algorithm of the security sandbox to determine that the networked media device having the sandbox reachable service communicates in a shared network common to the client device and the networked media device, and

to apply the relay algorithm of the security sandbox to establish the communication session between the sandboxed application and the sandbox reachable service of the networked media device.

45. The system of claim 43:

wherein the discovery algorithm utilizes a protocol comprising at least one of a Bonjour®

protocol, a SSDP protocol, a LSD uTorrent®

protocol, a multicast protocol, an anycast protocol, and another Local Area Network based protocol that discovers services in a local area network based on a broadcast from any one of an operating system service, the security sandbox, the client device, the sandbox reachable service, and the networked media device.

46. The system of claim 35:

wherein a cookie associated with the security sandbox is used to store a remote access token on a storage of the client device,

wherein the remote access token identifies at least one of a set of communicable private IP addresses and hardware addresses associated with sandbox reachable services that previously operated on a common shared network with the client device, and

wherein the client device is capable of communicating with the sandbox reachable services that previously operated on the common shared network through the remote access token.

47. The system of claim 43:

wherein the client device and the networked media device reside on networks that are incommunicable with each other comprising at least one of a firewall separation, a different network separation, a physical separation, and an unreachable connection separation, and

wherein the sandboxed application of the security sandbox of the client device and the sandbox reachable service of the networked media device communicate with each other through a relay service employed by the pairing server having the discovery module and the relay module to facilitate a trusted communication between the sandboxed application and the sandbox reachable service.

48. The system of claim 47:

wherein the trusted communication is facilitated such that the sandboxed application never learns at least one of a private IP address and a hardware address of the networked media device when;

a first NAT device coupled with a network on which the client device operates receives communications from a public IP address of a different network on which the sandbox reachable service operates, and

a second NAT device coupled with the different network on which the networked media device operates translates the private IP address of the networked media device to the public IP address visible to the sandboxed application.

49. The system of claim 47:

wherein the networked media device comprises a plurality of sandbox reachable applications including the sandbox reachable service,

wherein a service agent module of the networked media device coordinates communications with the discovery module of at least one of the security sandbox and the pairing server,

wherein the security sandbox is at least one of an operating system on which the sandboxed application is hosted and a browser application of the operating system, and

wherein the networked media device is at least one of a television, a projection screen, a multimedia display, a touchscreen display, an audio device, a weather measurement device, a traffic monitoring device, a status update device, a global positioning device, a geospatial estimation device, a tracking device, a bidirectional communication device, a unicast device, a broadcast device, and a multidimensional visual presentation device.

50. The system of claim 47, wherein the client device is further configured:

to utilize at least one of a WebSocket and a long polling service message query interface to reduce a latency of message delivery during the trusted communication between the sandboxed application and the sandbox reachable service, and

to optimize a polling period between polling to be less than a timeout period of a session through the relay service.

51. The system of claim 47, wherein the client device is further configured:

to initiate the relay service through at least one of a series of web pages where information is communicated using hyperlinks that point at the pairing server, and a form having a confirmation dialog that is submitted back to the pairing server,

wherein a global unique identifier is masked through the pairing server when the confirmation dialog is served from the pairing server.