Reusable multimodal application

• US 9,489,441 B2
• Filed: 06/30/2015
• Issued: 11/08/2016
• Est. Priority Date: 04/10/2002
• Status: Active Grant

• Current Assignee: Gula Consulting Limited Liability Company (Intellectual Ventures LLC)
• Original Assignee: Gula Consulting Limited Liability Company (Intellectual Ventures LLC)
• Inventors: Anderl, Ewald C.
• Primary Examiner(s): Batista, Marcos
• Application Number:
  US14/755,105
• Publication Number:
  US 20150379102A1
• Time in Patent Office: 497 Days
• Field of Search: 455/414.4, 370/338
• US Class Current: 1/1
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04M 2201/42 Graphical user interfaces
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 3/42204 Arrangements at the exchange for service or number selection by voice at the terminal H04M1/27
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method and system are disclosed herein for accepting multimodal inputs and deriving synchronized and processed information. A reusable multimodal application is provided on the mobile device. A user transmits a multimodal command to the multimodal platform via the mobile network. The one or more modes of communication that are inputted are transmitted to the multimodal platform(s) via the mobile network(s) and thereafter synchronized and processed at the multimodal platform. The synchronized and processed information is transmitted to the multimodal application. If required, the user verifies and appropriately modifies the synchronized and processed information. The verified and modified information are transferred from the multimodal application to the visual application. The final result(s) are derived by inputting the verified and modified results into the visual application.

Claims

1. A method comprising:

receiving, by a first computing device, data indicative of input information from a second computing device via a communications network, the input information resulting from an input received by the second computing device while executing an application in a first mode of communication;

generating, by the first computing device, synchronized information by synchronizing the input information with a second mode of communication; and

transmitting, by the first computing device, the synchronized information to the second computing device,

wherein the synchronized information, after being received by the second computing device, facilitates the second computing device switching from the first mode of communication to the second mode of communication and rendering the synchronized information for verification and modification in the second mode of communication while continuing execution of the application.

2. The method of claim 1, wherein the first and second modes of communication comprise one of audio, visual, and haptic modes.

3. The method of claim 1, wherein the input information comprises a search request.

4. The method of claim 3, further comprising:

receiving, by the first computing device, search grammar associated with the search request from the second computing device; and

identifying, by the first computing device, grammar elements associated with the search request and the search grammar.

5. The method of claim 4, further comprising:

determining, by the first computing device, search words based at least in part upon the identified grammar elements; and

transmitting, by the first computing device, the search words to the second computing device.

6. The method of claim 1, wherein the synchronized information comprises a plurality of search terms.

7. The method of claim 6, wherein the synchronized information, after being received by the second computing device, facilitates the second computing device rendering the plurality of search terms in the second mode of communication and permitting user selection of one of the plurality of search terms.

8. The method of claim 1, wherein the input information comprises one or more of text information or a uniform resource locator address.

9. The method of claim 1, wherein the synchronized information, after being received by the second computing device, facilitates the second computing device switching from the first mode of communication to the second mode of communication in response to receiving a user selection of the second mode of communication.

10. A computing device comprising a processor and memory, the memory comprising executable instructions that, in response to being executed by the processor, cause the computing device to:

receive data indicative of input information from a remote computing device via a communications network, the input information resulting from an input received by the remote computing device while executing an application in a first mode of communication;

generate synchronized information by synchronizing the input information with a second mode of communication; and

transmit the synchronized information to the remote computing device,

wherein the synchronized information, after being received by the remote computing device, facilitates the remote computing device switching from the first mode of communication to the second mode of communication and rendering the synchronized information for verification and modification in the second mode of communication while continuing execution of the application.

11. The computing device of claim 10, wherein the input information comprises a search request.

12. The computing device of claim 11, wherein the memory further comprises executable instructions that, in response to being executed by the processor, cause the computing device to:

receive search grammar associated with the search request from the remote computing device; and

identify grammar elements associated with the search request and the search grammar.

13. The computing device of claim 12, wherein the memory further comprises executable instructions that, in response to being executed by the processor, cause the computing device to:

determine search words based at least in part upon the identified grammar elements; and

transmit the search words to the remote computing device.

14. The computing device of claim 10, wherein the synchronized information comprises a plurality of search terms.

15. The computing device of claim 14, wherein the synchronized information, after being received by the remote computing device, facilitates the remote computing device rendering the plurality of search terms in the second mode of communication and permitting user selection of one of the plurality of search terms.

16. A non-transitory, computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions comprising instructions that, in response to being executed by a computing device, cause the computing device to:

receive data indicative of input information from a mobile computing device via a communications network, the input information resulting from an input received by the mobile computing device while executing an application in a first mode of communication;

generate synchronized information by synchronizing the input information with a second mode of communication; and

transmit the synchronized information to the mobile computing device,

wherein the synchronized information, after being received by the mobile computing device, facilitates the mobile computing device switching from the first mode of communication to the second mode of communication and rendering the synchronized information for verification and modification in the second mode of communication while continuing execution of the application.

17. The computer-readable storage medium of claim 16, wherein the input information comprises a search request, and wherein the computer-executable instructions further comprise instructions that, in response to being executed by the computing device, cause the computing device to:

receive search grammar associated with the search request from the mobile computing device; and

identify grammar elements associated with the search request and the search grammar.

18. The computer-readable storage medium of claim 17, wherein the computer-executable instructions further comprise instructions that, in response to being executed by the computing device, cause the computing device to:

determine search words based at least in part upon the identified grammar elements; and

transmit the search words to the mobile computing device.

19. The computer-readable storage medium of claim 16, wherein the synchronized information comprises a plurality of search terms.

20. The computer-readable storage medium of claim 19, wherein the synchronized information, after being received by the mobile computing device, facilitates the mobile computing device rendering the plurality of search terms in the second mode of communication and permitting user selection of one of the plurality of search terms.