Assisted shopping

• US 10,296,953 B2
• Filed: 08/15/2016
• Issued: 05/21/2019
• Est. Priority Date: 06/29/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hertschuh, Fabien G., St-Pierre, Benoit, Ford, James L., Cier, Sean P., Hall, VI, Samuel P., Wong, Peter C.
• Primary Examiner(s): Pham, Michael
• Application Number:
  US15/237,569
• Publication Number:
  US 20160350825A1
• Time in Patent Office: 1,009 Days
• Field of Search: 707713
• CPC Class Codes: G06Q 30/0613 Third-party assisted
• CPC Class Codes: G06Q 30/0617 Representative agent
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0637 Approvals
• CPC Class Codes: G06Q 30/0641 Shopping interfaces

Abstract

Disclosed are various embodiments for facilitating an assisted shopping experience. A speech input can be captured and transcribed. The transcribed speech input can be transmitted to a customer service agent in a data session that is contemporaneous with a voice session. The customer service agent can then facilitate an assisted shopping experience via the voice session and data session.

Claims

1. A non-transitory computer-readable medium having a computer-executable program stored thereon that, when executed by a computing device, configures the computing device to perform operations comprising:

receiving an audio input from a client device;

obtaining location information from a positioning system associated with the client device;

generating a transcription of the audio input;

performing a search query that is based at least upon the transcription of the audio input;

initiating a bidirectional voice session with the client device, the bidirectional voice session enabling a first user of the client device to speak with a second user;

rendering for display a user interface of content being shown on the client device;

rendering for display, contemporaneously with maintaining the bidirectional voice session with the client device, a search results user interface that displays a plurality of items identified from the search query on the client device, the plurality of items being generated based at least in part on the transcription of the audio input;

generating a refined search query that is based at least upon the plurality of items generated from the search query, the refined search query selected by the second user and obtained from the second user via a customer service agent user interface, the refined search query further being different from the search query; and

controlling the client device to display at least one search result in an instance after a selection of the at least one search result by the second user via the customer service agent user interface, the at least one search result being generated based at least upon the refined search query and the location information.

2. The non-transitory computer-readable medium of claim 1, the operations further comprising receiving information associated with an operation mode of the client device, the operation mode comprising a sleep mode or an active mode.

3. The non-transitory computer-readable medium of claim 2, the operations further comprising:

generating at least one product detail page from the computing device for display on the client device in response to a selection of a product by the second user via the customer service agent user interface;

obtaining a visual representation of content rendered on a display of the client device; and

updating the customer service agent user interface with a proximity sensor state and the visual representation of content rendered on the display of the client device.

4. The non-transitory computer-readable medium of claim 1, the operations further comprising:

transmitting a purchasing command to the client device, the purchasing command causes a shopping application executed on the client device to execute a purchase of item.

5. A system comprising:

a client device; and

an application executable in the client device, the application configured to cause the client device to;

capture an audio input for a search query;

initiate a data session with a computing device associated with a user;

generate a transcription of the audio input;

initiate a bidirectional voice session with the computing device;

render for display, contemporaneously with maintaining the bidirectional voice session with the computing device, a user interface in a display, the user interface comprising a first plurality of search results that includes a plurality of items identified from the search query on the client device, the plurality of items being generated based at least upon the transcription of the audio input;

receive location information from a positioning system associated with the client device;

transmit the location information and the first plurality of search results to the computing device; and

in response to receiving a second plurality of search results from the computing device, update the user interface to include the second plurality of search results, the second plurality of search results are generated based at least upon a refined search query performed via a customer service agent user interface, the refined search query being based at least upon the plurality of items generated from the search query and the location information.

6. The system of claim 5, wherein the application further causes the client device to render in the user interface the transcription of the audio input.

7. The system of claim 5, wherein the bidirectional voice session is initiated in response to rendering on the user interface the transcription of the audio input.

8. The system of claim 5, wherein the audio input comprises a product, and wherein the application further causes the client device to receive a page describing the product that is transmitted from the computing device in response to selection of an item by the user via the customer service agent user interface.

9. The system of claim 8, wherein the audio input comprises a first input, and wherein the application further causes the client device to:

receive a second input via the user interface corresponding to a purchase confirmation of the product rendered on the display; and

transmit the purchase confirmation to the computing device.

10. The system of claim 5, wherein the application further causes the client device to:

capture at least one of an image or a video with a capture device in communication with the client device; and

transmit the at least one of the image and the video to the computing device via the data session.

11. The system of claim 5, wherein the application further causes the client device to activate a speaker mode that comprises a speaker level setting and a microphone sensitivity setting.

12. A computer-implemented method comprising:

receiving, via at least one computing device over a network, an audio input from a client device;

generating, via the at least one computing device, a transcription based at least upon the audio input;

performing, via the at least one computing device, a search query based at least upon the transcription of the audio input;

initiating, via the at least one computing device, a bidirectional voice session with the client device;

rendering for display, via the at least one computing device and contemporaneously with maintaining the bidirectional voice session with the client device, a search results user interface that displays a plurality of items identified from the search query on the client device, the plurality of items being generated based at least in part on the transcription of the audio input;

identifying, via the at least one computing device, a refined search query that is based at least upon the plurality of items generated from the search query, the refined search query being different from the search query and being selected by a user via a customer service agent user interface in response to receiving the transcription of the audio input; and

transmitting, from the at least one computing device to the client device over the network, at least one search result in response to a selection of the at least one search result by the user via the customer service agent user interface, the at least one search result being generated based at least upon the refined search query, wherein transmitting the at least one search result to the client device causes the at least one search result to be rendered on a display of the client device.

13. The method of claim 12, wherein the user comprises a customer service agent, and further comprising initiating, via the at least one computing device, the bidirectional voice session between the customer service agent and the client device in response to rendering for display the transcription of the audio input on the client device.

14. The method of claim 13, further comprising initiating, via the at least one computing device, a data session with the client device, the bidirectional voice session operating contemporaneously with the data session.

15. The method of claim 12, further comprising receiving, via the at least one computing device, information associated with a state of a display device associated with the client device.

16. The method of claim 12, further comprising:

generating, via the at least one computing device, a product recommendation based at least upon a purchase history and a browse history associated with a user account of the client device; and

transmitting, via the at least one computing device, the product recommendation to the client device in response to selection of the product recommendation by the user via the customer service agent user interface.

17. The method of claim 12, wherein generating the transcription further comprises performing a speech recognition algorithm on the audio input, the speech recognition algorithm configured to convert the audio input to text.

18. The method of claim 12, further comprising:

transmitting, via the at least one computing device, at least one product detail page from the at least one computing device to the client device in response to a selection of a product by the user via the customer service agent user interface; and

receiving, via the at least one computing device, a visual representation of content rendered on a display of the client device.

19. The method of claim 12, wherein performing the search query further comprises executing a query of a product catalog associated with an electronic commerce site.

20. The method of claim 12, wherein generating the transcription of the audio input further comprises removing at least one term from the transcription based at least upon an identification of the at least one term having a plurality of search results that exceed a threshold.