Method, device, and system for obtaining information based on audio input

• US 10,354,307 B2
• Filed: 05/31/2016
• Issued: 07/16/2019
• Est. Priority Date: 05/29/2014
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Ye, Wa, Xu, Zhenyu, Sun, Bo, Zang, Jing, Tang, Zhuo, Wang, Hongyang, Li, Keren
• Primary Examiner(s): Airapetian, Mila
• Application Number:
  US15/168,595
• Publication Number:
  US 20160275588A1
• Time in Patent Office: 1,141 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 21/14 by displaying frequency domain information
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/375 Commercial
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/63 for services of sales data processing systems or methods specially adapted for electronic commerce G06Q30/00
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2542 for selling goods, e.g. TV shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8545 for generating interactive applications

Abstract

A Method and a device for obtaining information based on an audio input are disclosed. The method is performed at an electronic device having one or more processors and memory. The method includes: while running a social networking application, sending a first audio input to a server hosting the social networking application, wherein the first audio input is captured in response to a triggering event; retrieving, from the server, a product information item associated with a product identified based on the first audio input, wherein the product information item is obtained by comparing at least one audio feature of the first audio input with signature audio features extracted from audio of TV or radio broadcast within a predetermined time span; and displaying the retrieved product information item to a user of the electronic device within the social networking application.

Claims

1. A method for obtaining information based on an audio input, comprising:

at an electronic device having one or more processors and memory;

while running a social networking application;

sending a first audio input to a server hosting the social networking application, wherein the first audio input is captured in response to a triggering event;

retrieving, from the server, a product information item associated with a product identified based on the first audio input, wherein the product information item is obtained by comparing at least one audio feature of the first audio input with signature audio features extracted from audio of TV or radio broadcast within a predetermined time span; and

displaying the retrieved product information item to a user of the electronic device within the social networking application;

in response to an interaction from the user indicating an intent to order the product, sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application.

2. The method of claim 1, wherein:

the first audio input comprises an audio recording of at least a portion of a first advertisement for a product broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first advertisement for the product broadcast on TV on a first channel at the first time; and

the product information item is identified based on advertisement information associated with the first channel and the first time.

3. The method of claim 1, wherein:

the trigger event includes receiving a voice instruction from the user of the electronic device, the voice instruction including one or more keywords;

the first audio input comprises an audio recording of at least a portion of a first program broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first program broadcast on TV on a first channel at the first time; and

the product information item is identified based on the first channel, the first time, and the one or more keywords.

4. The method of claim 1, wherein:

the product information item includes a phone number for calling a seller to order the product; and

the method further comprises;

sending a request to initiate a telephone call to the seller using the telephone number; and

in response to establishing a live connection with the seller on the phone, sending a notification to the user to join the telephone call to the seller on the electronic device.

5. The method of claim 1, further comprising:

retrieving seller information items corresponding to multiple sellers identified based on the product information item;

presenting a listing of the multiple sellers and respective selling prices of the product offered by the multiple sellers; and

in response to a selection from the user, sending a request to the server for ordering the product from the selected seller.

6. The method of claim 1, further comprising:

receiving an alert from the server in response to the server detecting an activity from a user'"'"'s contact within the social networking application, the activity being associated with the product; and

in response to receiving the alert, presenting a notification to the user asking the user whether to establish a chat session between the user and the user'"'"'s contact within the social networking application.

18. The method of claim 1, wherein:

the interaction from the user indicating an intent to order the product comprises a numerical value transfer instruction;

the transaction account associated with the user within the social networking application is a payment account;

the sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application comprises;

sending a payment request carrying a payment amount, an identity of the payment account, and an identity of a payment receiving account to the server according to the numerical value transfer instruction, so that the server, according to the payment request, subtracts the payment amount from the payment account and adds the payment amount to the payment receiving account according to the payment request.

7. An electronic device, comprising:

one or more processors; and

memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for;

while running a social networking application;

sending a first audio input to a server hosting the social networking application, wherein the first audio input is captured in response to a triggering event;

retrieving, from the server, a product information item associated with a product identified based on the first audio input, wherein the product information item is obtained by comparing at least one audio feature of the first audio input with signature audio features extracted from audio of TV or radio broadcast within a predetermined time span; and

displaying the retrieved product information item to a user of the electronic device within the social networking application;

in response to an interaction from the user indicating an intent to order the product, sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application.

8. The electronic device of claim 7, wherein:

the first audio input comprises an audio recording of at least a portion of a first advertisement for a product broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first advertisement for the product broadcast on TV on a first channel at the first time; and

the product information item is identified based on advertisement information associated with the first channel and the first time.

9. The electronic device of claim 7, wherein:

the trigger event includes receiving a voice instruction from the user of the electronic device, the voice instruction including one or more keywords;

the first audio input comprises an audio recording of at least a portion of a first program broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first program broadcast on TV on a first channel at the first time; and

the product information item is identified based on the first channel, the first time, and the one or more keywords.

10. The electronic device of claim 7, wherein:

the product information item includes a phone number for calling a seller to order the product; and

the one or more programs further comprise instructions for;

sending a request to initiate a telephone call to the seller using the telephone number; and

in response to establishing a live connection with the seller on the phone, sending a notification to the user to join the telephone call to the seller on the electronic device.

11. The electronic device of claim 7, wherein the one or more programs further comprise instructions for:

retrieving seller information items corresponding to multiple sellers identified based on the product information item;

presenting a listing of the multiple sellers and respective selling prices of the product offered by the multiple sellers; and

in response to a selection from the user, sending a request to the server for ordering the product from the selected seller.

12. The electronic device of claim 7, wherein the one or more programs further comprise instructions for:

receiving an alert from the server in response to the server detecting an activity from a user'"'"'s contact within the social networking application, the activity being associated with the product; and

in response to receiving the alert, presenting a notification to the user asking the user whether to establish a chat session between the user and the user'"'"'s contact within the social networking application.

19. The electronic device of claim 7, wherein:

the interaction from the user indicating an intent to order the product comprises a numerical value transfer instruction;

the transaction account associated with the user within the social networking application is a payment account;

the sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application comprises;

sending a payment request carrying a payment amount, an identity of the payment account, and an identity of a payment receiving account to the server according to the numerical value transfer instruction, so that the server, according to the payment request, subtracts the payment amount from the payment account and adds the payment amount to the payment receiving account according to the payment request.

13. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors, cause the electronic device to perform operations comprising:

while running a social networking application;

sending a first audio input to a server hosting the social networking application, wherein the first audio input is captured in response to a triggering event;

retrieving, from the server, a product information item associated with a product identified based on the first audio input, wherein the product information item is obtained by comparing at least one audio feature of the first audio input with signature audio features extracted from audio of TV or radio broadcast within a predetermined time span; and

displaying the retrieved product information item to a user of the electronic device within the social networking application;

in response to an interaction from the user indicating an intent to order the product, sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application.

14. The non-transitory computer readable storage medium of claim 13, wherein:

the first audio input comprises an audio recording of at least a portion of a first advertisement for a product broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first advertisement for the product broadcast on TV on a first channel at the first time; and

the product information item is identified based on advertisement information associated with the first channel and the first time.

15. The non-transitory computer readable storage medium of claim 13, wherein:

the trigger event includes receiving a voice instruction from the user of the electronic device, the voice instruction including one or more keywords;

the first audio input comprises an audio recording of at least a portion of a first program broadcast on TV at a first time;

the at least one audio feature of the first audio input matches a first signature audio feature extracted from the first program broadcast on TV on a first channel at the first time; and

the product information item is identified based on the first channel, the first time, and the one or more keywords.

16. The non-transitory computer readable storage medium of claim 13, wherein:

the product information item includes a phone number for calling a seller to order the product; and

the one or more programs further comprise instructions which cause the electronic device to perform operations comprising;

sending a request to initiate a telephone call to the seller using the telephone number; and

in response to establishing a live connection with the seller on the phone, sending a notification to the user to join the telephone call to the seller on the electronic device.

17. The non-transitory computer readable storage medium of claim 13, wherein the one or more programs further comprise instructions which cause the electronic device to perform operations comprising:

retrieving seller information items corresponding to multiple sellers identified based on the product information item;

presenting a listing of the multiple sellers and respective selling prices of the product offered by the multiple sellers; and

in response to a selection from the user, sending a request to the server for ordering the product from the selected seller.

20. The non-transitory computer readable storage medium of claim 13, wherein:

the interaction from the user indicating an intent to order the product comprises a numerical value transfer instruction;

the transaction account associated with the user within the social networking application is a payment account;

the sending a request to the server for ordering the product using a transaction account associated with the user within the social networking application comprises;

sending a payment request carrying a payment amount, an identity of the payment account, and an identity of a payment receiving account to the server according to the numerical value transfer instruction, so that the server, according to the payment request, subtracts the payment amount from the payment account and adds the payment amount to the payment receiving account according to the payment request.