×

Electronic device for processing multi-modal input, method for processing multi-modal input and server for processing multi-modal input

  • US 10,191,718 B2
  • Filed: 11/28/2017
  • Issued: 01/29/2019
  • Est. Priority Date: 11/28/2016
  • Status: Active Grant
First Claim
Patent Images

1. An electronic device comprising:

  • a housing;

    a touchscreen display exposed through a first portion of the housing;

    a microphone disposed at a second portion of the housing;

    a speaker disposed at a third portion of the housing;

    a memory disposed inside the housing; and

    at least one processor disposed inside the housing, and electrically connected to the display, the microphone, the speaker, and the memory,wherein the memory is configured to store a plurality of application programs, each of the application programs including a graphic user interface (GUI), andwherein the memory is further configured to store instructions that, when executed, cause the at least one processor to;

    control the microphone to receive an utterance from a user;

    generate or receive text data based on the utterance;

    generate or receive a sequence of states in which the electronic device performs a task based on the text data, wherein the sequence of states includes at least two states of at least one of the application programs;

    select one of the application programs, based at least partly on the sequence of states;

    launch the selected application program;

    control to display the GUI of the selected application program on the display;

    control to display at least part of a sequence of automatically providing an input into the GUI on the display, based at least partly on the sequence of states;

    cause the selected application program to perform an operation that utilizes at least one parameter, using the input as the at least one parameter, depending on the sequence of states; and

    control to display a result of the operation on the display.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×