Method for communication between users and smart appliances

• US 10,003,683 B2
• Filed: 03/19/2015
• Issued: 06/19/2018
• Est. Priority Date: 02/27/2015
• Status: Active Grant

• Current Assignee: Samsung Eletrônica da Amazônia Ltda (Samsung Electronics Co. Ltd.)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kaneblai Martins Costa, Gustavo, Barbosa Silveira, Alexandre, Jose Tobias, Rodrigo, Zilse Pereira Borges, Renata, Ortiz Delvequio Franco, Viviane, Victor Motta, Paulo, Rogrio Piva, Fbio, Gustavo Pellacani Petrini, Daniel, Lopes Ramos, Thiago, Kim, Taehwan
• Primary Examiner(s): Hussain, Imad
• Application Number:
  US14/662,714
• Publication Number:
  US 20160255186A1
• Time in Patent Office: 1,188 Days
• Field of Search: 709206
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

The present invention provides a method (200) that allows users (211) of smart appliances to interact in a similar way with objects (202, 207) such as devices, smart appliances, Web services, software applications (THINGS/objects) and people (212) and also the communication of an object (202) with another object (207). The provided method comprises mapping the entire object (207) connected as a contact. These contacts (objects) will act as real instant messaging contacts, receive user commands and entries in a natural language fashion and provide response using the IM interface. Such objects (202, 207) must register as an instant messaging user on a common instant messaging server (210). Furthermore, an application wrapper (208) is also used to map an instant messaging contact and an object allowing an object not supporting this solution to be used by a device implementing the proposed method.

Claims

1. A method comprising:

preparing a message to send from the user device to an instant messaging (IM) contact in a contact list of the user device;

establishing whether the IM contact is a person or an object; and

determining, based on a result of the establishing, whether to send the message to the IM contact or load a natural language interface (NLI), wherein the determining includes;

if the IM contact is determined to be a person, sending the message in natural language format to the IM contact;

if the IM contact is determined to be an object, loading the NLI in the user device, including;

detecting a marker and a command of the message;

analyzing a context of the message and an ontology of the object to identify a purpose of the message;

evaluating possible commands for the object and identifying a command among the possible commands that most likely reflects and intention of the user;

submitting the identified command to the object; and

receiving a response from the object and translating the response into a natural language response.

2. The method according to claim 1, characterized in that a registration of a new IM contact in the contact list includes:

pairing the user device with another device that can be registered as a new IM contact in the contact list.

3. The method according to claim 2, characterized in that the pairing the user device includes pairing the user device using at least one of Wi-Fi communication, Bluetooth communication, and near field communication (NFC).

4. The method according to claim 1, characterized in that the message is prepared using text to speech technology.

5. The method according to claim 1, characterized in that the object includes at least one of a device, smart appliance, web service, and software application.

6. The method according to claim 1, characterized in that the message includes at least of digital audio, digital image, digital video, and compressed text.

7. The method according to claim 1, characterized in that a service wrapper is used to establish a connection between the user device and a web service that does not support direct communication with the user device.

8. The method according to claim 1, characterized in that a device wrapper is used to establish a connection between the user device and a smart appliance that does not support direct communication with the user device.

9. A non-transitory computer-readable recording medium storing a program to implement a method comprising:

preparing a message to send from a user device to an instant messaging (IM) contact in a contact list of the user device;

establishing whether the IM contact is a person or an object; and

determining, based on a result of the establishing, whether to send the message to the IM contact or load a natural language interface (NLI), wherein the determining includes;

if the IM contact is determined to be a person, sending the message in natural language format to the IM contact;

if the IM contact is determined to be an object, loading the NLI, including;

detecting a marker and a command of the message;

analyzing a context of the message and an ontology of the object to identify a purpose of the message;

evaluating possible commands for the object and identifying a command among the possible commands that most likely reflects and intention of the user;

submitting the identified command to the object; and

receiving a response from the object and translating the response into a natural language response.

10. An apparatus comprising:

at least one memory configured to store instructions; and

at least one processor configured to execute the stored instructions to implement a method comprising;

preparing a message to send from the apparatus to an instant messaging (IM) contact in a contact list of the user device;

establishing whether the IM contact is a person or an object; and

determining, based on a result of the establishing, whether to send the message to the IM contact or load a natural language interface (NLI), wherein the determining includes;

if the IM contact is determined to be a person, sending the message in natural language format to the IM contact;

if the IM contact is determined to be an object, loading the NLI, including;

detecting a marker and a command of the message;

analyzing a context of the message and an ontology of the object to identify a purpose of the message;

evaluating possible commands for the object and identifying a command among the possible commands that most likely reflects and intention of the user;

submitting the identified command to the object; and

receiving a response from the object and translating the response into a natural language response.