Voice control of endpoint devices through a multi-services gateway device at the user premises

• US 10,530,598 B2
• Filed: 12/08/2015
• Issued: 01/07/2020
• Est. Priority Date: 12/29/2006
• Status: Active Grant

• Current Assignee: Kip Prod P1 LP
• Original Assignee: Kip Prod P1 LP
• Inventors: Ansari, Amir, Cowgill, George A., Nicholls, Leon E., Raissyan, Atousa, Ramayya, Jude P., Masin, Ramprakash, McQuarters, Alvin R.
• Primary Examiner(s): Potratz, Daniel B
• Application Number:
  US14/962,165
• Publication Number:
  US 20160226823A1
• Time in Patent Office: 1,491 Days
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06F 16/64 Browsing; Visualisation therefor generation of a list or set of audio data G06F16/638
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2807 Exchanging configuration information on appliance services in a home automation network arrangements for maintenance or administration involving network analysis for automatically determining the actual topology of a network H04L41/12; hardware or software tools for network management using graphical user interfaces H04L41/22; address allocation H04L61/50
• CPC Class Codes: H04L 12/2812 describing content present in a home automation network, e.g. audio video content retrieval from the Internet G06F16/95
• CPC Class Codes: H04L 12/2814 Exchanging control software or macros for controlling appliance services in a home automation network arrangements for maintenance or administration involving configuration of the network and network elements H04L41/08
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00
• CPC Class Codes: H04L 12/2818 from a device located outside both the home and the home network access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 47/80 : Actions related to the user profile or the type of traffic
• CPC Class Codes: H04L 47/83 : based on usage prediction
• CPC Class Codes: H04L 49/25 : Routing or path finding in a switch fabric
• CPC Class Codes: H04L 61/4552 : Lookup mechanisms between a plurality of directories; Synchronisation of directories, e.g. metadirectories
• CPC Class Codes: H04L 63/02 : for separating internal from external traffic, e.g. firewalls
• CPC Class Codes: H04L 63/08 : for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0876 : based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
• CPC Class Codes: H04L 63/10 : for controlling access to devices or network resources
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 65/102 : Gateways arrangements for connecting between networks having differing types of switching systems, e.g. gateways, H04L12/66
• CPC Class Codes: H04L 65/1101 : Session protocols
• CPC Class Codes: H04L 65/1108 : Web based protocols, e.g. webRTC
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/104 : Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/141 : Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/53 : using third party service providers
• CPC Class Codes: H04L 69/325 : in the network layer [OSI layer 3], e.g. X.25 H04L69/16 takes precedence
• CPC Class Codes: H04N 21/00 : Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/40 : Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 12/00 : Security arrangements; Authentication; Protecting privacy or anonymity
• CPC Class Codes: H04W 12/033 : of the user plane, e.g. user's traffic
• CPC Class Codes: H04W 12/0431 : Key distribution or pre-distribution; Key agreement
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 12/065 : Continuous authentication
• CPC Class Codes: H04W 12/08 : Access security
• CPC Class Codes: H04W 12/35 : Protecting application or service provisioning, e.g. securing SIM application provisioning
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: Y10S 370/911 : Bridge, e.g. brouter, bus extender

Abstract

A gateway device providing and managing interactive user voice-controlled home automation services over at least endpoint devices associated with the gateway device is described. Interfaces enable communications of the gateway device with at least one endpoint device located within a use premises and data networks. A processor coupled to the interfaces and programming in storage in the server operate a communications program for configuring the gateway to communicate with endpoint devices, associate endpoint devices with the gateway, and communicate with a remote serve to access service management center applications. A home automation control program provides operation via a home automation controller configured in the gateway with voice automation messaging protocols based on the voice commands to enable interactive voice-controlled control of endpoint devices using voice commands an automate functions associated with the endpoint devices.

Claims

1. A communication device for use at a user premises to provide interactive user voice-controlled home automation, comprising;

(a) a first interface for enabling communications with at least one endpoint device located within the user premises;

(b) a second interface for enabling communications of the communication device at the user premises via a wide area network;

(c) a local processor coupled to the interfaces;

(d) local storage coupled to the processor; and

(e) programming in the storage including;

(1) a communications program for configuring the communication device to communicate via the first interface with the at least one endpoint device, associate the at least one endpoint device with the communication device, and communicate with at least one remote server each associated with at least one of a plurality of third-party applications that are stored locally on the communication device, wherein each third-party application is configured for authentication with a respective one of a plurality of third-party service providers, wherein each of the third-party applications is capable of being accessed and controlled via the at least one endpoint device, and wherein the communication device;

stores authentication information in the local storage;

executes one of the locally stored third-party applications to authenticate the communication device with the respective one of the plurality of third-party service providers based on the locally stored authentication information, wherein at least a portion of the locally stored authentication information is obtained in response to an initial registration with the respective one of the plurality of third-party service provider, wherein the authenticating is independent of authenticating the communication device with other ones of the plurality of third-party service providers;

in response to successful authentication of the communication device,

establishes bi-directional communication with the respective one of the plurality of third-party service providers that has authenticated the communication device, independently from other ones of the plurality of third-party service providers, wherein the at least a portion of the locally stored authentication information is used for renewing key-based authentication for the bi-directional communication with the respective one of the plurality of third-party service provider,

receives, via the wide area network, media from the respective one of the plurality of third-party service providers,

sends, via the one of the locally stored third-party applications, the received media to the at least one endpoint device such that the media is displayed on a display of the at least one endpoint device, and

causes the display of the at least one endpoint device to display an interface for controlling at least one of displaying of the media or media control functions; and

(2) a control program providing operations via a controller associated with the communication device having voice protocols, wherein in response to successful authentication of the communication device,

the control program enables the user to provide voice commands to control, via the one of the locally stored third-party applications, displaying of the media by the at least one endpoint device, wherein the voice commands are provided through the communication device and/or a remote client.

2. The communication device of claim 1, further including in the storage at least one of:

IP protocol standards, non-IP protocol standards, and vendor specific protocols, and also configured to enable in-premise protocol messaging with the at least one endpoint device, retrieval of presence and a status information from the at least one endpoint device, and remote monitoring and control of vendor diverse devices located within the premises.

3. The communication device of claim 2, wherein the remote monitoring and control of vendor diverse devices is via endpoint devices associated with the communication device.

4. The communication device of claim 1, wherein the wide area network provides communications of the communication device with the remote server to access a service management center.

5. The communication device of claim 1, wherein the programming, when executed by the processor, causes the communication device to provide enforcement regarding authorization, authentication, configuration, or use of the communication device via the remote client to access and control the at least one endpoint device.

6. The communication device of claim 1, wherein the programming, executed by the processor, causes the communication device to further provide management of application services based upon communications of the communication device with a service management center via the wide area network through the second interface.

7. The communication device of claim 1, wherein the programming enables the first or second interface of the communication device to establish peer communications between the communication device and the remote server via the wide area network.

8. The communication device of claim 1, wherein the remote client is provided in the form of at least one of:

a personal computer, a cell phone, a personal digital assistant, a remote control, a television, a set top box, or any combination thereof, and includes a user interface to access the third-party applications.

9. The communication device of claim 1, wherein the programming in the storage includes configuration data for management, responses, and interactions with the at least one endpoint device in response to communications between the communication device and the remote server via the wide area network.

10. The communication device of claim 1, wherein the voice protocols for communications between the communication device and the remote server comprise an instant messaging protocol.

11. The communication device of claim 8, wherein the user interface is enabled by at least one of a driver and a communications protocol accessible from at least one of the communication device or the remote server and enables the user to determine at least one of status, change, configuration, event log, or any combination thereof for the at least one endpoint device.

12. The communication device of claim 1, wherein the programming in the communication device further enables communications with a second communication device via the remote server.

13. The communication device of claim 12, wherein the programming in the communication device configures the communication device to communicate over the wide area network and receive network address information from the second communication device such that the address information enables the communication device to establish peer to peer communication with the second communication device.

14. The communication device of claim 1, wherein the programming in the communication device further enables communications with a second communication device via the remote server.

15. The communication device of claim 14, wherein the programming in the communication device configures the communication device to communicate over the wide area network and receive network address information from the second communication device such that the address information enables the communication device to establish peer to peer communication with the second communication device.

16. The communication device of claim 1, wherein media content is displayed on the endpoint device.

17. The communication device of claim 1, wherein control of the plurality of third-party applications via the at least one endpoint device includes setting limitations configured to prevent access to one or more of the third-party applications and/or a portion of media content associated with the plurality of third-party applications.

18. The communication device of claim 1, wherein the wide area network enabling communications of the communication device can be enabled and/or disabled via a display of the at least one endpoint device.

19. The communication device of claim 1, wherein the controller is part of the communication device.

20. A system for enabling the operation and management of interactive user voice-control, the system comprising:

a communication device configured for operation at a user premises and including;

(a) a first interface for enabling communications with at least one endpoint device located within the user premises;

(b) a second interface for enabling communications of the communication device at the user premises via a wide area network;

(c) a local processor coupled to the interfaces;

(d) local storage coupled to the processor and configured to store a plurality of third-party programming each configured for authentication by a respective one of a plurality of third-party service providers, and

(e) programming in the storage including;

(1) programming for executing one of the third-party programming in the local storage to authenticate the communication device with the respective one of the plurality of third-party service providers using authentication information locally stored on the local storage, wherein the at least a portion of the locally stored authentication information is obtained based on an initial registration with the respective one of the plurality of third-party service providers,

(2) a communications program for configuring the communication device to associate the at least one endpoint device and enable communications by communication device via the interfaces with the at least one endpoint device and with a remote server to establish bi-directional communication with the respective one of the plurality of third-party service providers in response to successful authentication of the communication device, wherein the communication device receives media transmitted via the wide area network and sends the received media to the at least one endpoint device, wherein the third-party programming associated with the respective third-party service provider provides control of displaying of the media by a display of the at least one endpoint device, and wherein the at least a portion of the locally stored authentication information is used for renewing key-based authentication for the bi-directional communication with the respective one of the plurality of third-party service provider; and

(3) in response to the successful authentication of the communication device and via the one of the third-party programming, a control program having voice automation protocols capable of enabling interactive voice-enabled control of the media displayed by the at least one endpoint device through at least one of the communication device, a remote client, or the at least one endpoint device.

21. The system of claim 20, further including in the storage at least one of IP protocol standards, non-IP protocol standards, or vendor specific protocols, and also configured to enable in-premise protocol messaging with the at least one endpoint device, retrieval of presence and a status information from the at least one endpoint device, and remote monitoring and control of vendor diverse devices located within the premises.

22. The system of claim 21, wherein the remote monitoring and control of vendor diverse home automation devices is via endpoint devices associated with the communication device.

23. The system of claim 20, wherein the wide area network provides communications of the communication device with the remote server to access a service management center.

24. The system of claim 20, wherein execution of the programming by the processor causes the communication device to provide enforcement regarding authorization, authentication, configuration, or use of the communication device via the client to access and control the at least one endpoint device.

25. The system of claim 20, wherein the execution of the programming by the processor causes the communication device to further provide management of application services based upon communications of the communication device with the service management center via the wide area network through the second interface.

26. The system of claim 20, wherein the programming enables the first or second interface of the communication device to establish peer communications between the communication device and the remote server via the wide area network.

27. The system of claim 20, wherein the remote client is provided in the form of at least one of a personal computer, a cell phone, a personal digital assistant, a remote control, a television, a set top box or any combination thereof, and includes a user interface to access the third-party programming.

28. The system of claim 27, wherein the user interface is enabled by at least one of a driver and a communications protocol accessible from at least one of the communication device or the remote server and enables the user to determine at least one of status, change, configuration, event log, or any combination thereof for the at least one endpoint device.

29. The system of claim 20, wherein the programming in the storage includes configuration data for management, responses, and interactions with the at least one endpoint device in response to communications between the communication device and the remote server via the wide area network.

30. The system of claim 20, wherein the voice automation protocols for communications between the communication device and the remote server comprise an instant messaging type protocol.

31. The system of claim 20, wherein the communication device transmits a list of functionalities to a second communication device via peer to peer communications between the communication device and the second communication device.

32. The system of claim 31, further comprising a first associated endpoint device that is a remote control, wherein the communication device is enabled to receive a signal from the remote control via a driver communications protocol of a driver program, and wherein the communication device controls and manages a second associated endpoint device via the driver program based on the received signal from the remote control.

33. A method for operating a communication device at a user premises to provide and manage interactive user voice-controlled services and enable the provision of a service from at least one endpoint device associated with the communication device, the method comprising;

(a) providing a communication device including a first interface for enabling communications with at least one endpoint device located within the user premises, a second interface for enabling communications of the communication device at the user premises via a wide area network, a processor coupled to the interfaces, and programming in a storage of the communication device to utilize protocols and enable communications, voice recognition, endpoint association, and/or endpoint management, the storage having a plurality of locally stored third-party applications each configured to perform authentication with a respective one of a plurality of third-party service providers;

(b) configuring the communication device for bi-directional communication with the respective one of the third-party service providers to provide information for authorization and/or authentication by the respective one of the third-party service providers based, at least in part, on one or more locally stored credentials that are obtained in response to an initial registration with the respective one of the third-party service providers, wherein the one or more locally stored credentials are used for renewing key-based authorization and/or authentication for the bi-directional communication with the respective one of the plurality of third-party service provider;

(c) after authentication, via one of the locally stored third-party applications, by the respective one of the plurality of third-party service providers, the one of the locally stored third-party applications manages delivery of media from the respective one of the plurality of third-party service providers to the at least one endpoint device and displaying of the delivered media by the at least one endpoint device, wherein the media includes at least one of video or music; and

(d) accessing a control program to configure the communication device with voice automation protocols to enable interactive voice-enabled control of one or more endpoint devices via voice commands provided from a user through at least one of the communication device, a remote client, and/or the at least one endpoint device such that the user is capable of providing voice commands to control, via the one of the locally stored third-party applications and in response to successful authentication, automate functions associated with the one or more endpoint devices providing the media.

34. The method of claim 33, wherein at least one of the communication device and the storage contains at least one of:

IP protocol standards, non-IP protocol standards, and vendor specific protocols, and the communication device is further configured to enable in-premise protocol messaging with the at least one endpoint device, retrieval of presence and a status information from the at least one endpoint device, and remote monitoring and control of vendor diverse devices located within the premises.

35. The method of claim 34, wherein the remote monitoring and control of vendor diverse home automation devices is conducted via endpoint devices associated with the communication device.

36. The method of claim 33, wherein the wide area network enables communications of the communication device with a remote server to access a service management center.

37. The method of claim 33, wherein execution of the programming by the processor further enables the communication device to enforce at least one of authorization, authentication, configuration, and use of the communication device to access and control the at least one endpoint device.

38. The method of claim 33, wherein execution of the programming by the processor causes the communication device to further provide management of the application services based upon communications of the communication device with a service management center via the wide area network.

39. The method of claim 33, wherein execution of the programming enables the first or second interface of the communication device to establish peer communications between the communication device and the remote server via the wide area network.

40. The method of claim 33, wherein the remote client is provided in the form of at least one of:

a personal computer, a cell phone, a personal digital assistant, a remote control, a television, a set top box, or any combination thereof, and includes a user interface to access the plurality of locally stored third-party applications.

41. The method of claim 40, wherein the user interface is enabled by at least one of a driver and a communications protocol accessible from at least one of the communication device or the remote server and enables the user to determine at least one of status, change, configuration, event log, or any combination thereof for the at least one endpoint device.

42. The method of claim 33, wherein the programming in the storage includes configuration data for management, responses, and interactions with the at least one endpoint device in response to communications between the communication device and the remote server via the wide area network.

43. The method of claim 33, wherein a controller further enables the user to provide voice commands that enable messaging protocols for communications between the communication device and the remote server via at least one instant messaging protocol.

44. The method of claim 33, wherein the programming in the communication device further enables communications with a second communication device via the remote server.

45. The method of claim 44, wherein the programming in the communication device configures the communication device to communicate over the wide area network and receive network address information from the second communication device such that the address information enables the communication device to establish peer to peer communication with the second communication device.

46. A communication system for use at a user premises to manage interactive user voice-controlled services, comprising:

a communication device including;

a first interface configured to enable communications of the communication device with at least one television located within the user premises;

a second interface configured to enable communications of the communication device via a wide area network;

a processor coupled to the first and second interfaces; and

memory coupled to the processor and including;

a communications program configured to allow the communication device to communicate via the first and second interfaces with the at least one television and at least one remote server,

a plurality of locally stored third-party applications, wherein a respective one of the locally stored third-party applications is configured to establish communication with a respective third-party service provider to enable authentication by the respective third-party service provider based, at least in part, on authentication information locally stored in the memory, wherein at least a portion of the locally stored authentication information is obtained in response to an initial registration with the respective third-party service provider, wherein the communication is independent of the other third-party service providers associated with other ones of the locally stored third-party applications, wherein in response to the authentication, the respective one of the locally stored third-party applications enables media to be streamed to the television, and wherein the at least a portion of the locally stored authentication information is used to renew authentication during streaming of the media; and

a control program having voice protocols configured to enable, in response to the authentication, interactive voice-enabled control of the respective one of the locally stored third-party applications by a user within the user premises.

47. The communication system of claim 46, wherein the voice commands provided from the user are received via a controller associated with the communication device, wherein the controller is configured to control functions associated with the at least one television or the communication device.

48. The communication system of claim 46, further comprising a controller configured to communicate with and control the communication device.

49. A communication device for use at a user premises, the communication device comprising:

at least one interface that enables communication with at least one endpoint device at the user premises and communication via a wide area network;

one or more processors; and

local storage that stores authentication information, wherein the local storage includes

authentication information;

a plurality of third-party applications for enabling communication with a respective one of a plurality of third-party service providers;

wherein the communication device

executes one of the locally stored third-party applications to authenticate the communication device with the respective one of the plurality of third-party service providers based on at least a portion of the locally stored authentication information, wherein the authentication is independent of authentication of the communication device by other ones of the plurality of third-party service providers and wherein the at least a portion of the locally stored authentication information is obtained based, at least in part, on an initial registration with the respective one of the plurality of third-party service providers;

in response to successful authentication of the communication device, the communication device

communicates with the respective one of the plurality of third-party service providers that has authenticated the communication device to receive media from the respective one of the plurality of third-party service providers, wherein the at least a portion of the locally stored authentication information is used to renew key-based authentication for communicating with the respective one of the plurality of third-party service providers; and

enables a user to provide voice commands to the communication device to control displaying of the received media by the at least one endpoint device.

50. The communication device of claim 49, wherein the voice commands are provided through the communication device and/or a remote client.