Method, medium, and system for live video cooperative shopping

• US 9,984,408 B1
• Filed: 05/30/2012
• Issued: 05/29/2018
• Est. Priority Date: 05/30/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wong, Peter C., Hertschuh, Fabien G.
• Primary Examiner(s): Zimmerman, Matthew
• Application Number:
  US13/483,212
• Time in Patent Office: 2,190 Days
• Field of Search: 705 261, 705 271
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers

Abstract

Disclosed are various embodiments for facilitating a cooperative shopping session. A mobile computing device establishes a session with a remote computing device. Product data is displayed by the mobile computing device in association with the session. The product data is shared with the remote computing device in association with the session. The mobile computing device receives an indication of a user interaction with the product data at the remote computing device. The mobile computing device displays the user interaction with the product data in conjunction with the product data and in association with the session.

Claims

1. A non-transitory computer-readable medium embodying a program executable in a computing device, the program, when executed, causing the computing device to at least:

establish a session with another computing device;

receive product metadata in association with the session;

capture a live video stream of one or more physical products located in a retail location by a video capture device of the computing device;

identify a physical product in the live video stream based at least in part on a comparison between the live video stream and at least one identified product image;

display the product metadata and the live video stream of the one or more physical products in association with the session;

transmit the product metadata and the live video stream of the one or more physical products to the another computing device in association with the session;

receive a user input, from the another computing device and in association with the session, the user input indicating an annotation to the displayed live video stream;

display the annotation to the displayed live video stream in association with the session;

receive, from the another computing device, a request to establish a remote control by the another computing device of an application executed in the computing device facilitating the session;

in response to an acceptance of the request, remotely control the application executed in the computing device, the computing device being associated with the session by the another computing device;

inhibit, in response to a denial of the request, remote control of the application executed in the computing device, the computing device being associated with the session by the another computing device; and

display another annotation indicated by another user input, the annotation being applied to a virtual product shelf comprising a plurality of selected physical products.

2. The non-transitory computer readable medium of claim 1, wherein the program further causes the computing device to modify display of the live video stream of the one or more physical products with an overlay including the annotation.

3. A method, comprising:

establishing a session by a mobile computing device with a remote computing device, the mobile computing device comprising a video capture device;

capturing contemporaneous product imagery of a physical product located in a retail location with the video capture device;

displaying, by the mobile computing device, product data in association with the session, the product data comprising the contemporaneous product imagery;

identifying, by the mobile computing device, the physical product in the contemporaneous product imagery based at least in part on a comparison between the contemporaneous product imagery and at least one identified product image;

contemporaneously sharing, by the mobile computing device, the product data with the remote computing device in association with the session;

receiving, by the mobile computing device, an indication of a user interaction with the product data at the remote computing device in association with the session;

contemporaneously rendering, by the mobile computing device, the user interaction with the product data in conjunction with the product data and in association with the session;

receiving, from the remote computing device, a request to establish a remote control, by the remote computing device, of an application executed in the mobile computing device facilitating the session;

remotely controlling, in response to an acceptance of the request, by the mobile computing device, the application executed in the mobile computing device associated with the session by the remote computing device;

inhibiting, in response to a denial of the request, by the mobile computing device, the remote control of the application executed in the mobile computing device associated with the session by the remote computing device; and

displaying an annotation from the remote computing device as applied to a virtual product comprising a plurality of selected products.

4. The method of claim 3, further comprising interpreting the user interaction with the product data as a user input to the mobile computing device.

5. The method of claim 4, wherein the user input comprises at least one of a key event, a mouse event, or a gesture.

6. The method of claim 3, wherein the indication of the user interaction with the product data comprises a voice comment.

7. The method of claim 3, wherein displaying the user interaction with the product data in conjunction with the product data further comprises updating the display of the product data in accordance with received user interaction with the product data.

8. The method of claim 3, wherein displaying user interaction with the product data further comprises updating the display with an overlay including the indication of the user interaction with the product data.

9. The method of claim 3, wherein the product data further comprises product metadata.

10. The method of claim 3, wherein identifying the physical product in the contemporaneous product imagery further comprises:

transmitting a query including an image in the contemporaneous product imagery to a product image recognition service performing the comparison; and

receiving a result corresponding to the query from the product image recognition service, the result including a product identifier.

11. The method of claim 10, further comprising displaying the product identifier received from the product image recognition service.

12. The method of claim 10, wherein the result further includes product metadata associated with the product identifier.

13. The method of claim 10, further comprising:

receiving a selection of the physical product represented by the product data;

in response to the selection, adding the selected physical product to a virtual product shelf; and

displaying a visual representation of the virtual product shelf in association with the session.

14. A system, comprising:

a first computing device comprising a processor, a memory, and a video capture device; and

machine readable instructions stored in the memory that, when executed by the processor, cause the first computing device to at least;

establish a session with a second computing device;

receive product metadata in association with the session;

capture a live video stream of one or more physical products located in a retail location by the video capture device of the first computing device;

identify a physical product in the live video stream based at least in part on a comparison between the live video stream and at least one identified product image;

display the product metadata and the live video stream of the one or more physical products in association with the session;

transmit the product metadata and the live video stream of the one or more physical products to the second computing device in association with the session;

receive a user input, from the second computing device and in association with the session, the user input indicating an annotation to the displayed live video stream;

display the annotation to the displayed live video stream in association with the session;

receive, from the second computing device, a request to establish a remote control by the second computing device of an application executed in the first computing device facilitating the session;

in response to an acceptance of the request, remotely control the application executed in the first computing device, the first computing device being associated with the session by the second computing device;

inhibit, in response to a denial of the request, remote control of the application executed in the first computing device, the first computing device being associated with the session by the second computing device; and

display another annotation indicated by another user input, the annotation being applied to a virtual product shelf comprising a plurality of selected physical products.

15. The system of claim 14, wherein the annotation indicates a selection of a portion of the displayed physical product in the live video stream.

16. The system of claim 14, wherein the annotation indicates an addition of a graphical object to a portion of the displayed live video stream.

17. The system of claim 14, wherein the annotation is a first annotation and the machine readable instructions further cause the first computing device to communicate a second annotation to the second computing device, wherein the second annotation is applied to a virtual product shelf comprising a plurality of selected products.

18. The system of claim 17, wherein the machine readable instructions further cause the first computing device to:

receive, from the second computing device and in association with the session, a list of the plurality of selected products of the virtual product shelf; and

display a visual representation of the virtual product shelf in association with the session.

19. The system of claim 14, wherein an identification of the physical product in the live video stream based at least in part on the comparison between the live video stream and the at least one identified product image is generated by a product image recognition service.

20. The system of claim 14, wherein the machine readable instructions further cause the first computing device to create a voice channel between the first computing device and the second computing device in association with the session.

21. The system of claim 14, wherein the machine readable instructions further cause the first computing device to a multimedia channel between the first computing device and the second computing device in association with the session.

22. The system of claim 14, wherein the user input comprises a gesture on a touch screen of the second computing device.

23. The system of claim 14, wherein the user input comprises a key event.

24. The system of claim 14, wherein the user input comprises a mouse event.