Structured lighting-based content interactions in multiple environments

• US 9,373,025 B2
• Filed: 03/20/2012
• Issued: 06/21/2016
• Est. Priority Date: 03/20/2012
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Amacker, Matthew W., Tesler, Joel D.
• Primary Examiner(s): Debrow, James J
• Application Number:
  US13/425,275
• Publication Number:
  US 20130254646A1
• Time in Patent Office: 1,554 Days
• Field of Search: 715/234
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Information from a position and/or gesture detection system can be embedded in a Web page, or other such presentation of content, and used to select or otherwise interact with content on the page. In some embodiments, video is captured and displayed showing a current view of the user. Position data corresponding to the video is provided and used to determine directions and extents of motion without having to do significant amounts of image processing. The position data is used to determine locations on the page where the user is attempting to provide input, such as to select an item of content. The content can be modified and/or rendered to appear to be associated with the user in the rendered view. Information from multiple gesture systems can be combined on a single page, and information from one or more gesture systems can be shared among multiple pages and devices.

Claims

1. A computer-implemented method of rendering content, comprising:

initiating a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enabling a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

receiving position data and image data from a gesture system, the image data including a first view of a current orientation of the first user and the position data corresponding to features of the first user at the current orientation;

rendering a first Web page for the session on the first computing device using the first browser application and a second Web page for the session on the second computing device using the second browser application, the first Web page and the second Web page including first content and the first view of the current orientation of the first user, the first content including a first set of a plurality of user-selectable objects corresponding to a first portion of the plurality of items for consumption;

analyzing the position data to determine at least one of a motion or gesture by the first user being represented in the image data, the motion or gesture of the first user causing to be indicated one of the plurality user-selectable objects of the first set of the plurality of user-selectable objects, the one of the plurality user-selectable objects corresponding to a wearable item;

manipulating a representation of the wearable item to correspond to the shape of the first user;

rendering a second view that includes the representation of the wearable item to appear to be worn by the first user on the first Web page based at least in part on the current orientation of the first user and the first content; and

rendering a third view that includes the representation of the wearable item to appear to be worn by the first user on the second Web page and second content, the second content including a second set of a plurality of user-selectable objects, a portion of the second set of the plurality of user-selectable objects corresponding a second portion of the plurality of items for consumption.

2. The computer-implemented method of claim 1, wherein the gesture system is part of one of the first computing device rendering the first Web page or the second computing device rendering the second Web page and separate from a provider of content for one of the first Web page or the second Web page.

3. The computer-implemented method of claim 2, wherein the gesture system includes at least one video camera capable of providing video image data and at least one structured light subsystem operable to provide the position data.

4. The computer-implemented method of claim 1, further comprising:

requesting content associated with the each of the first set of the plurality user-selectable objects and the second set of the plurality of user-selectable objects to be displayed.

5. The computer-implemented method of claim 1, further comprising purchasing the one of the first set of the plurality of user-selectable objects or the second set of the plurality of user-selectable objects.

6. The computer-implemented method of claim 1, wherein the position data and image data are received from the gesture system through a response stream.

7. A computer-implemented method, comprising:

under control of one or more computer systems configured with executable instructions,

initiating a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enabling a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

receiving image data and position data from a gesture system, the position data corresponding to features of the first user at a current orientation of the first user relative to the gesture system, the image data including a first view of a current orientation of the first user;

rendering a first page for the session on the first computing device using the first browser application and a second Web page for the session on the second computing device using the second browser application, the first Web page and the second Web page including first content and the first view of the current orientation of the first user, the first content including a first set of a plurality of user-selectable objects corresponding to a first portion of the plurality of items for consumption;

analyzing the position data to determine at least one of a motion or gesture by the first user being represented in the image data, the motion or gesture of the first user causing to be indicated one of the user-selectable objects of the first set of the plurality of user-selectable objects, the one of the user-selectable objects corresponding to a wearable item;

manipulating a representation of the wearable item to correspond to the shape of the first user;

rendering a second view that includes the representation of the wearable item to appear to be worn by the first user on the first page based at least in part on the representation of the current orientation of the first user and the first content; and

rendering a third view that includes the representation of the wearable item to appear to be worn by the first user on the second Web page and second content, the second content including a second set of a plurality of user-selectable objects, a portion of the second set of the plurality of user-selectable objects corresponding a second portion of the plurality of items for consumption.

8. The computer-implemented method of claim 7, wherein each of the first page, the second page, and the third page is a Web page.

9. The computer-implemented method of claim 7, wherein the position data is received as a stream of gesture data.

10. The computer-implemented method of claim 7, wherein active script executing in a browser application is configured to accept and analyze gesture data to determine at least one of a motion or gesture of the first user.

11. The computer-implemented method of claim 9, wherein the stream of gesture data is JavaScript Object Notation (JSON) data.

12. The computer-implemented method of claim 7, wherein the gesture module is sourced from a first domain and at least a portion of content of one of the first page or the second page is sourced from a second domain.

13. The computer-implemented method of claim 12, wherein an application rendering at least one of the first page or the second page includes a frame configured to enable data to pass between the first domain and the second domain.

14. The computer-implemented method of claim 7, wherein a portion of the first set of user-selectable objects are images of clothing items that are available for the first user to purchase.

15. The computer-implemented method of claim 14, further comprising:

displaying a view of a clothing item in an indicated image with respect to a view of the first user in the gesture module on the first page.

16. The computer-implemented method of claim 15, wherein the view of the first user is provided using video data received with the position data from the gesture system.

17. The computer-implemented method of claim 16, wherein the view of the clothing item is modified to correspond to at least one of the current orientation of the first user and a shape of the first user as indicated by the received video data.

18. The computer-implemented method of claim 7, wherein the at least one of the motion or the gesture of the first user causes to be selected multiple indicated users-selectable objects.

19. The computer-implemented method of claim 7, wherein the position data is capable of being analyzed to determine two or more motions or gestures being performed concurrently.

20. A computing device, comprising:

a device processor;

a display screen; and

a memory device including instructions operable to be executed by the device processor to perform a set of actions, enabling the computing device to;

initiate a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enable a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

obtain image data and position data from a gesture system, the position data corresponding to a current orientation of a first user relative to the gesture system, the image data including a first view of a current orientation of the first user;

render a first display for the session on the first computing device using the first browser application and a second Web page for the session on the second computing device using the second browser application, the first Web page and the second Web page including first content and the first view of the current orientation of the first user, the first display including a first set of a plurality of user-selectable objects corresponding to a first portion of the plurality of items for consumption;

analyze the position data to determine at least one of a motion or gesture by the first user being represented in the image data, the motion or gesture of the first user causing to be indicated one of the user-selectable objects of the first set of the plurality of user-selectable objects, the one of the user-selectable objects corresponding to a wearable item;

manipulate a representation of the wearable item to correspond to the shape of the first user;

render a second display that includes the representation of the wearable item to appear to be worn by the first user on the first display screen based at least in part on the representation of the current orientation of the first user and the first content; and

render a third display that includes the representation of the wearable item to appear to be worn by the first user on the second Web page and second content, the second content including a second set of a plurality of user-selectable objects, a portion of the second set of the plurality of user-selectable objects corresponding a second portion of the plurality of items for consumption.

21. The computing device of claim 20, wherein each of the first display, the second display, and the third display is a Web page, the position data is received as a stream of JSON data, and JavaScript executing in a browser application is configured to accept and analyze the stream of JSON data to determine at least one of a motion or gesture of the user.

22. The computing device of claim 20, wherein the gesture module is sourced from a first domain and at least a portion of content of the first display is sourced from a second domain, and wherein an application rendering the first display includes a frame configured to enable data to pass between the first domain and the second domain.

23. The computing device of claim 20, further comprising:

displaying a view of a clothing item in an indicated image with respect to a view of the first user in the gesture module on the first display.

24. The computing device of claim 21, wherein the first display of the first user is provided using video data received with the position data from the gesture system, and wherein the first display of the clothing item is modified to correspond to at least one of the current orientation of the first user and a shape of the first user as indicated by the received video data.

25. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

initiate a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enable a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

receive image data and position data from a gesture system, the position data corresponding to features of a first user at a current orientation of the user relative to the gesture system, the image data including a first view of a current orientation of the first user;

provide a first for the session on the first computing device using the first browser application and a second Web page for the session on the second computing device using the second browser application, the first Web page and the second Web page including first content and the first view of the current orientation of the user, the first display including a first set of a plurality of user-selectable objects corresponding to a first portion of the plurality of items for consumption;

analyze the position data to determine at least one of a motion or gesture by the first user being represented in the image data, the motion or gesture of the first user causing to be indicated one of the user-selectable objects of the first set of the plurality of user-selectable objects, the one of the user-selectable objects corresponding to a wearable item;

manipulate a representation of the wearable item to correspond to the shape of the first user;

render a second display that includes the representation of the wearable item to appear to be worn by the first user on the first display screen based at least in part on the representation of the current orientation of the first user and the first content; and

render a third display that includes the representation of the wearable item to appear to be worn by the first user on the second Web page and second content, the second content including a second set of a plurality of user-selectable objects, a portion of the second set of the plurality of user-selectable objects corresponding a second portion of the plurality of items for consumption.

26. The non-transitory computer-readable storage medium of claim 25, wherein the gesture module is sourced from a first domain and at least a portion of content of the first display is sourced from a second domain, and wherein an application rendering the first display includes a frame configured to enable data to pass between the first domain and the second domain.

27. The non-transitory computer-readable storage medium of claim 25, further comprising:

displaying a view of a clothing item in an indicated image with respect to a view of the first user in the gesture module on the first display.

28. The non-transitory computer-readable storage medium of claim 27, wherein the view of the first user is provided using video data received with the position data from the gesture system, and wherein the view of the clothing item is modified to correspond to at least one of the current orientation of the first user and a shape of the first user as indicated by the received video data.