Interactive audio communication system

• US 9,412,394 B1
• Filed: 03/09/2015
• Issued: 08/09/2016
• Est. Priority Date: 03/09/2015
• Status: Expired due to Fees

• Current Assignee: Go Yuasa, Tadahiro Suzuki
• Original Assignee: Jigen Labs, LLC
• Inventors: Suzuki, Tadahiro, Yuasa, Go
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US14/642,419
• Time in Patent Office: 519 Days
• Field of Search: 704/235, 704/270, 704/270.1, 704/275, 704/260, 704/251, 379/88.01, 379/88.08, 379/88.11
• US Class Current: 1/1
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/6083 by interfacing with the vehicle audio system
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72442 for playing music files

Abstract

An interactive audio communication system includes a server for operating the interactive audio communications, a first computing device for operating a first application program, and a second computing device for operating a second application program, wherein the second application program includes the steps of receiving audio inputs from a user via the second microphone, transforming the audio inputs to text strings and transmitting the data of the audio inputs and/or the text strings to the first computing device, wherein the first application program running on the first computing device includes the steps of, receiving the data of the audio inputs and/or the text strings, displaying the text strings together with an audio mark if the text strings are received together with the data of the audio inputs on the first display.

Claims

1. A system for operating interactive audio communications using a communication network, comprising:

at least one server for providing an application programming interface (API) for operating the interactive audio communications on the communication network, said at least one server being connected to the communication network;

a first computing device for operating a first application program using the API via the communication network, the first computing device comprising;

a first processor;

a first memory;

a first speaker; and

a first display;

a second computing device for operating a second application program using the API via the communication network, the second computing device comprising;

a second processor;

a second memory;

a second microphone;

a second speaker; and

a second display for displaying information transmitted from the first computing device via the communication network using the second application program using the API via the communication network;

wherein the second memory stores instructions in the second application program that, when executed by the second processor, causes the second processor to perform a second operation comprising the steps of;

starting the second application program using the API via the communication network;

downloading the information from the first computing device via the server;

displaying the downloaded information on the second display;

receiving audio inputs from a user of the second computing device via the second microphone;

memorizing data of the audio inputs into the second memory;

transforming the audio inputs to text strings; and

transmitting the data of the audio inputs in the second memory and/or the text strings to the first computing device;

wherein the first memory stores instructions in the first application program that, when executed by the first processor, causes the first processor to perform a first operation comprising the steps of;

receiving the data of the audio inputs and/or the text strings;

displaying the text strings together with an audio mark which shows that the data of the audio inputs are transmitted together with the text strings if the text strings are received together with the data of the audio inputs on the first display; and

reproducing the audio inputs via the first speaker if a user of the first computing device touches the audio mark on the first display.

2. The system for operating interactive audio communications using the communication network of claim 1,

wherein the second operation of the second processor further comprises the steps of:

displaying a talk button on the second display; and

starting recording of the audio inputs from the user of the second computing device via the second microphone into the second memory for a predetermined time period when the talk button is touched.

3. The system for operating interactive audio communications using the communication network of claim 1,

wherein the second operation of the second processor further comprises the steps of:

displaying a talk button on the second display;

starting recording of the audio inputs from the user via the second microphone into the second memory when the talk button is touched; and

stopping the recording of the audio inputs from the user when the talk button is next touched.

4. The system for operating interactive audio communications using the communication network of claim 1,

wherein the second operation of the second processor further comprises the steps of:

displaying a talk button on the second display;

starting recording of the audio inputs from the user via the second microphone into the second memory together with geographical location information of the user of the second computing device and time information into the second memory when the talk button is touched for a predetermined time period.

5. The system for operating interactive audio communications using the communication network of claim 4,

wherein the predetermined time period is set to be not less than 0.3 seconds and not more than three seconds.

6. The system for operating interactive audio communications using the communication network of claim 1,

wherein the second operation of the second processor further comprises the steps of:

changing frequency characteristics of signals of the audio inputs from the user via the second microphone; and

memorizing the data of the audio inputs having changed frequency characteristics thereof into the second memory.

7. The system for operating interactive audio communications using the communication network of claim 1,

wherein the second operation of the second processor further comprises the steps of:

downloading pre-recorded audio from said at least one server;

selecting one of the pre-recorded audio by outputting the selected pre-recorded audio via the second speaker and selecting the pre-recorded audio;

memorizing the selected pre-recorded audio into the second memory when responding to a message sent via the communication network; and

transmitting the selected pre-recorded audio when responding to a message from a user of the first computing device via the communication network.

8. A system for operating interactive audio communications using a communications network, comprising:

at least one server for providing an application programming interface (API) for operating the interactive audio communications on the communication network, said at least one server being connected to the communication network;

a first computing device for operating web-site programs using the API, the first computing device being arranged to communicate with said at least one server via the communication network, the first computing device comprising;

a first processor;

a first memory; and

a first display;

a second computing device for operating application programs using the API via the communication network, the second computing device comprising;

a second processor;

a second memory;

a second display; and

a second microphone

wherein the second memory stores instructions in the application programs that, when executed by the second processor, causes the second processor to perform a second operation comprising the steps of;

receiving an inquiry in audio form via the second microphone or an inquiry in text form from a user of the second computing device;

transforming the inquiry in the audio form into text strings; and

transmitting data of the inquiry in the audio form and text strings or data of the inquiry in the text form, geographical information of the user and time information showing a location and a time where and when the transmission to the first computing device is performed;

wherein the first memory stores instructions in the web-site programs that, when executed by the first processor, causes the first processor to perform a first operation comprising the steps of;

receiving the data of the inquiry in the audio form and the text strings or the data of the inquiry in the text form, the geographical information and the time information from the second computing device via the communication network;

extracting at least a keyword from the text string and associating said at least a keyword with advertisements to be broadcast;

displaying an audio mark for reproducing the inquiry in the audio form together with the text strings with said at least a keyword on the first display if the inquiry in the audio form is received;

displaying a time table including the advertisements including said at least a keyword on the first display;

displaying a map associated with the geographical location of the user on the first display; and

transmitting answers or response in a text form to the inquiry.

9. The system for operating interactive audio communications using the communication network of claim 8,

wherein the second operation of the second processor further comprises:

downloading the answers provided by the first computing device;

transforming the answers in text form into audio form if the answer is received in the text form;

outputting the answers in the audio form via the second speaker; and

memorizing the inquiry in the audio form or the inquiry in the text form and the downloaded answers provided by the first computing device.

10. The system for operating interactive audio communications using the communication network of claim 9,

wherein the second operation of the second processor further comprises:

downloading a rating factor of an operator who is providing the answers or responses; and

memorizing the rating factor to the second memory.

11. The system for operating interactive audio communications using the communication network of claim 8,

wherein the first operation of the first processor further comprises:

displaying a remaining time for responding to the inquiry from the user of the second computing device on the first display in a first color until a predetermined time has elapsed, and displaying an elapsed time in a second color from a point when the remaining time becomes zero.

12. The system for operating interactive audio communications using the communication network of claim 11,

wherein the first operation of the first processor further comprises:

displaying a reject button to be used when the inquiry from the user of the second computing device is not within parameters or guidelines of acceptable inquiries; and

responding to the inquiry from the user of the second computing device using a template message.

13. The system for operating interactive audio communications using the communication network of claim 9,

wherein the first operation of the first processor further comprises:

storing the answer in a text form from the operator who provides the answer to the inquiry from the user of the second computing device;

displaying the answers in the text form on the first display, wherein a word being difficult to transform into an audio form is made visible on the first display; and

storing the word into said at least one server.

14. A system for operating interactive audio communications using a communication network, comprising:

at least one server for providing an application programming interface (API) for operating the interactive audio communications on the communication network, said at least one server being connected to the communication network;

a first computing device for operating a first application program using the API via the communication network, the first computing device comprising;

a first display;

a first processor; and

a first memory,

a second computing device for operating a second application program using the API via the communication network, the second computing device comprising;

a second processor,

a second memory;

a second microphone; and

a second speaker

wherein the first memory stores instructions in the first program that, when executed by the first processor, causes the first processor to perform a first operation comprising the steps of;

starting the first application program to display information provided by said at least one server;

selecting items from the information provided by said at least one server and/or creating to-do items;

transmitting the selected items and/or the to-do items to the second computing device via the communication network;

wherein the second memory stores instructions in the second program that, when executed by the second processor, causes the second processor to perform second operation comprising the steps of;

receiving the selected items and/or the to-do items transmitted from the first computing device via the communication network;

performing a function related to the selected items and/or the to-do items;

receiving audio inputs from a user of the second computing device via the second microphone;

transforming the audio inputs to text strings; and

transmitting data of the audio inputs and/or the text strings to the first computing device.

15. The system for operating interactive audio communications using the communication network of claim 14,

wherein the second computing device further comprises an interface device being designed to communicate with the second computing device via a wireless network, the interface device comprising:

a sub-microphone;

a sub-talk button for starting recording of the audio inputs from the user via the sub-microphone into the second memory for a first predetermined time period when the sub-talk button is pushed;

wherein the second operation of the second processor further comprises;

receiving the audio inputs from the user of the second computing device via the sub-microphone for the first predetermined time period when the sub-talk button is pushed.

16. The system for operating interactive audio communications using the communication network of claim 15,

wherein the sub-talk button has a function for memorizing the data of the audio inputs from the user via the sub-microphone into the second memory together with the geographical location information of the user of the second computing device and time information when the sub-talk button is pushed for a second predetermined time period.

17. The system for operating interactive audio communications using the communication network of claim 16, wherein the second predetermined time period is not less than 0.3 seconds and not more than 3.0 seconds.

18. The system for operating interactive audio communications using the communication network of claim 14,

wherein the second computing device further comprises an interface device being designed to communicate with the second computing device via a wireless network, the interface device comprising:

a sub-microphone;

a sub-talk button for starting recording of the audio inputs from the user via the sub-microphone into the second memory when the sub-talk button is pushed and for stopping recording of the audio inputs when the sub-talk button is next pushed.

19. The system for operating interactive audio communications using the communication network of claim 18,

wherein the interface device further comprises:

a bookmark button for bookmarking an item from the information provided by the first computing device, the tag information including a text string constituting a resource associated with the bookmarked item; and

a sub-speaker,

wherein the second operation of the second processor further comprises the steps of;

downloading the information provided by said the first computing device;

reading out loud the information via the sub-speaker;

determining whether the bookmark button is pushed while or after the item to be bookmarked from the information is read out loud via the sub-speaker; and

extracting the tag information from the bookmarked item and memorizing the tag information into said at least one server if the bookmark button bookmarks the item from the information while or after the information is being read out loud via the sub-speaker.