SERVER, CONTROL METHOD THEREOF, IMAGE PROCESSING APPARATUS, AND CONTROL METHOD THEREOF

• US 20150012280A1
• Filed: 05/07/2014
• Published: 01/08/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: SHIN, Yong-wook, CHUNG, Ji-hye, SHIN, Seung-min, LEE, Hee-ran, CHOI, Sung-wook
• Granted Patent: US 10,140,985 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

A system includes a server and an image processing apparatus, and the server is provided that includes a communication interface, a storage, and a processor. The communication interface is configured to communicate with the image processing apparatus. The storage is configured to store data. The processor may provide a result of processing a first event that includes a speech of a user to the image processing apparatus in response to the first event being received from the image processing apparatus, store a record of the first event in the storage according to processing of the first event, determine a relation between the first and second events that includes a user input by a non-speech method in response to the second event being received from the image processing apparatus, and process the second event based on the record of the first event stored in the storage in response to the relation.

Claims

1. A system comprising:

a server; and

at least one image processing apparatus,

the server comprises;

a communication interface configured to communicate with the at least one image processing apparatus;

a storage configured to store data; and

a processor configured to provide, to the image processing apparatus, a result of processing a first event that includes a speech of a user in response to the first event being received from the image processing apparatus, store, in the storage, a record of the first event that corresponds to the processing of the first event, determine a relation between the first event and a second event that includes a user input by a non-speech method in response to the second event being received from the image processing apparatus, and process the second event based on the record of the first event stored in the storage in response to the relation.

2. The system of claim 1, wherein the processor determines whether to process the second event independently of the first event or to process the second event based on the result of processing the first event when the second event is received using the record of the first event stored in the storage.

3. The system of claim 1, wherein the processor is further configured to perform retrieval from a preset database based on the first event when the first event is received, and transmit, to the image processing apparatus, one final result selected based on the second event from among a plurality of retrieval results with respect to the first event in response to the second event being received with the plurality of retrieval results being derived from the first event.

4. The system of claim 3, wherein the processor is further configured to transmit, to the image processing apparatus, the plurality of retrieval results and a request command to select one of the retrieval results when there is the plurality of retrieval results with respect to the first event, and receive, from the image processing apparatus, the second event corresponding to the request command.

5. The system of claim 1, wherein the second event is converted into a virtual voice command recognizable and received by the server.

6. A control method of a server, the control method comprising:

receiving, from an image processing apparatus, a first event based on a speech of a user;

providing a result of processing the first event to the image processing apparatus and storing a record of the first event according to processing of the first event;

receiving, from the image processing apparatus, a second event based on a user input by a non-speech method;

determining a relation between the first event and the second event; and

processing the second event based on a pre-stored record of the first event in response to the relation.

7. The control method of claim 6, wherein the determining of the relation comprises:

determining whether to process the second event independently of the first event or to process the second event based on the result of processing the first event using the pre-stored record stored in the storage.

8. The control method of claim 6,

wherein the providing of the result of processing the first event to the image processing apparatus comprises:

performing retrieval from a preset database based on the first event, and

wherein the processing of the second event based on the pre-stored record of the first event comprises;

transmitting, to the image processing apparatus, one final result selected based on the second event among a plurality of retrieval results with respect to the first event in response to the second event being received with the plurality of retrieval results being derived from the first event.

9. The control method of claim 8, wherein the receiving of the second event comprises:

transmitting, to the image processing apparatus, the plurality of retrieval results and a request command configured to select one of the retrieval results when there is the plurality of retrieval results with respect to the first event; and

receiving, from the image processing apparatus, the second event corresponding to the request command.

10. The control method of claim 6, wherein the second event is converted into a virtual voice command recognizable and received by the server.

11. An image processing apparatus comprising:

a storage configured to store data;

a user interface comprising;

a speech input interface configured to receive an input of a speech of a user; and

a non-speech input interface configured to receive a non-speech user input;

a processor configured to perform processing on an operation corresponding to a first event that corresponds to the input of the speech at the speech input interface; and

a controller configured to store a record of generating and processing the first event in the storage, determine a relation between the first event and a second event, wherein the second event occurs at the non-speech input interface after the first event occurs, and control the processing of the second event based on the record of the first event stored in the storage in response to the relation.

12. The image processing apparatus of claim 11, wherein the controller determines whether to process the second event independently of the first event or to process the second event based on a result of processing the first event when the second event occurs using the record of the first event stored in the storage.

13. The image processing apparatus of claim 11, wherein the controller is further configured to display a plurality of retrieval results configured to be selected on a display, wherein the plurality of retrieval results with respect to the operation correspond to the first event, and determine whether the second event gives an instruction to select one of the retrieval results when the second event occurs when the retrieval results are being displayed.

14. The image processing apparatus of claim 13, further comprising:

a communication interface configured to communicate with a server that performs retrieval processing on a voice command corresponding to the speech,

wherein the controller transmits the first event to the server, receives a retrieval result with respect to the first event from the server, and displays the retrieval result on the display.

15. The image processing apparatus of claim 11, wherein the controller is further configured to convert the second event into a virtual voice command such that the server recognizes the second event, and transmits the virtual voice command to the server.

16. A control method of an image processing apparatus, the control method comprising:

generating a first event based on an input of a speech of a user, wherein a speech input interface is configured to receive the input; and

performing an operation corresponding to the first event,

wherein the performing of the operation corresponding to the first event comprises;

storing a record of generating and processing the first event in a storage;

generating a second event based on a non-speech user input, wherein a non-speech input interface is configured to receive a non-speech user input;

determining a relation between the first event and the second event; and

processing the second event based on the record of the first event in response to the relation.

17. The control method of claim 16, wherein the determining of the relation comprises:

determining whether to process the second event independently of the first event or to process the second event based on a result of processing the first event using the record of the first event.

18. The control method of claim 16,

wherein the performing of the operation corresponding to the first event further comprises:

displaying a plurality of retrieval results configured to be selected, wherein the plurality of retrieval results with respect to the operation correspond to the first event, and

wherein the determining of the relation between the first event and the second event comprises;

determining whether the second event gives an instruction to select one of the retrieval results.

19. The control method of claim 18, wherein the image processing apparatus is further configured to:

communicates with a server that performs retrieval processing on a voice command corresponding to the speech,

transmits the first event to the server, and

displays a retrieval result with respect to the first event received from the server.

20. The control method of claim 19, wherein the image processing apparatus is further configured to:

convert the second event into a virtual voice command that the server recognizes, and

transmits the virtual voice command to the server.