Structured lighting based content interactions

• US 9,213,420 B2
• Filed: 03/20/2012
• Issued: 12/15/2015
• Est. Priority Date: 03/20/2012
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Amacker, Matthew W., Tesler, Joel D.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): GILLIARD, DELOMIA L
• Application Number:
  US13/425,261
• Publication Number:
  US 20130249937A1
• Time in Patent Office: 1,365 Days
• Field of Search: 345/619
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0641 Shopping interfaces

Abstract

Information from a position and/or gesture detection system can be embedded in a Web page, or other such presentation of content, and used to select or otherwise interact with content on the page. In some embodiments, video is captured and displayed showing a current view of the user. Position data corresponding to the video is provided and used to determine directions and extents of motion without having to do significant amounts of image processing. The position data is used to determine locations on the page where the user is attempting to provide input, such as to select an item of content. The content can be modified and/or rendered to appear to be associated with the user in the rendered view. Information from multiple gesture systems can be combined on a single page, and information from one or more gesture systems can be shared among multiple pages and devices.

Claims

1. A computer-implemented method of rendering content on a computing device, comprising:

initiating a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enabling a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

receiving position data and image data from a gesture system, the image data including a view of a current orientation of the first user and the position data corresponding to features of the first user at the current orientation;

displaying, using the first browser application, in a first gesture module of a first Web page, a mirror image of the current orientation of the first user, the first Web page including a representation of a first set of one or more clothing items, the first set of the one or more clothing items being displayed outside the first gesture module;

analyzing the position data to determine at least one of a motion or gesture by the first user, the motion or gesture of the user indicating at least one clothing item from the first set of one or more clothing items;

analyzing the image data to determine a three-dimensional graphical model that approximates a shape of the first user; and

displaying, using the second browser application, in a second gesture module of a second Web page, the representation of the at least one clothing item to appear to be worn by the first user based at least in part on the current orientation of the first user, the second Web page including a second set of one or more clothing items, the second set of one or more clothing items being displayed outside the second gesture module, at least one clothing item of the second set of one or more clothing items being different from each clothing item of the first set of one or more clothing items.

2. The computer-implemented method of claim 1, wherein the gesture system includes at least one video camera capable of providing video image data and at least one structured light subsystem operable to provide the position data.

3. The computer-implemented method of claim 1, wherein the one or more clothing items are capable of being of different types, and wherein an appearance of at least one clothing item worn by the first user is modified based at least in part upon the type of the clothing item.

4. The computer-implemented method of claim 1, wherein modifying an appearance of the indicated image of the clothing item includes identifying at least one of an arm or a leg portion of the clothing item in the image and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

5. The computer-implemented method of claim 1, wherein modifying an appearance of the indicated image of the clothing item includes identifying a shape of a portion of the user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.

6. A computer-implemented method, comprising:

under control of one or more computer systems configured with executable instructions,

initiating a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enabling a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

obtaining position information and image information from a gesture system, the image information including a representation of a current orientation of at least a portion of the first user, the position information corresponding to features of the first user at the current orientation;

displaying, using the first browser application, in a first region of a first Web page, a mirror image of the current orientation of the at least a portion of the first user, the first Web page including a representation of a first set of one or more clothing items, the first set of the one or more clothing items being displayed outside a first gesture module;

analyzing the position information to determine at least one of a motion or gesture by the first user, the motion or gesture of the user indicating at least one clothing item from the first set of one or more clothing items;

determining a three-dimensional graphical model that approximates a shape of the first user; and

displaying, using the second browser application, in a second region of a second Web page, a view of the at least one clothing item to appear to be worn by the first user based at least in part on the current orientation of the first user, the second Web page including a second set of one or more clothing items, the second set of one or more clothing items being displayed outside a second gesture module, at least one clothing item of the second set of one or more clothing items being different from each clothing item of the first set of one or more clothing items; and

update the view with changes in the current orientation of at least a portion of the user to cause the item to appear to move with the user in the view.

7. The computer-implemented method of claim 6, wherein the gesture detection system includes at least one camera and at least one illumination source configured to project a pattern of structured light.

8. The computer-implemented method of claim 7, wherein the at least one illumination source projects at least one of visible light or infrared light.

9. The computer-implemented method of claim 6, wherein the gesture detection system is part of a computing device displaying the view of the first user.

10. The computer-implemented method of claim 6, further comprising:

analyzing the image information to identify at least one facial feature of the first user.

11. The computer-implemented method of claim 10, wherein at least one accessory is capable of being rendered on the view of the first user corresponding to at least one of a position or orientation of the at least one facial feature in the displayed view of the first user.

12. The computer-implemented method of claim 10, wherein a selection of one or more item capable of being displayed with the view of the first user is determined at least in part based upon an identity or gender of the first user as determined through an analysis of the image information.

13. The computer-implemented method of claim 6, further comprising:

modifying the image information by at least one of rotating, shearing, skewing, stretching, compressing or translating at least a portion of the view represented by the image information.

14. A computing device, comprising:

a device processor;

a display screen; and

a memory device including instructions operable to be executed by the device processor to perform a set of actions, enabling the computing device to;

initiate a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enable a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

obtain position information and image information from a gesture system, the image information including a representation of a current orientation of at least a portion of the first user, the position information corresponding to features of the first user at the current orientation;

display, using the first browser application, in a first region of a first Web page, a mirror image of the current orientation of the at least a portion of the first user, the first Web page including a representation of a first set of one or more clothing items, the first set of the one or more clothing items being displayed outside the first gesture module;

analyze the position information to determine at least one of a motion or gesture by the first user, the motion or gesture of the user indicating at least one clothing item from the first set of one or more clothing items;

determine a three-dimensional graphical model that approximates a shape of the first user and;

display, using the second browser application, in a second region of a second Web page, a view of the at least one clothing item to appear to be worn by the first user based at least in part on the current orientation of the first user, the second Web page including a second set of one or more clothing items, the second set of one or more clothing items being displayed outside the second gesture module, at least one clothing item of the second set of one or more clothing items being different from each clothing item of the first set of one or more clothing items.

15. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

update the view of the first user with changes in the current orientation of at least a portion of the first user to cause the item to appear to move with the first.

16. The computing device of claim 14, further comprising:

a video camera capable of providing the image information.

17. The computing device of claim 14, further comprising:

at least one illumination element operable to project a pattern of light; and

at least one sensor configured to detect reflections of the pattern of light,

wherein the device processor is operable to process the detected reflections to determine the position information for the current orientation of the first user.

18. The computing device of claim 17, wherein the at least one illumination element and the at least one sensor are part of a gesture subsystem.

19. The computing device of claim 14, wherein displaying, using the second browser application, includes identifying at least one of an arm or a leg portion of the clothing item represented in the image information and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

20. The computing device of claim 14, wherein displaying, using the second browser application, includes identifying a shape of a portion of the first user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.

21. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

initiate a session with an electronic marketplace, the electronic marketplace offering a plurality of items for consumption;

enable a first user and a second user to join the session, the first user joining via a first computing device that includes a first browser application and the second user joining via a second computing device that includes a second browser application;

receive position information and image information from a gesture system, the image information including a representation of a current orientation of at least a portion of the first user, the position information corresponding to features of the first user at the current orientation;

display, using the first browser application, in a first region of a first Web page, a mirror image of the current orientation of the at least a portion of the first user, the first Web page including a representation of a first set of one or more clothing items, the first set of the one or more clothing items being displayed outside the first gesture module;

analyze the position information to determine at least one of a motion or gesture by the first user, the motion or gesture of the user indicating at least one clothing item from the first set of one or more clothing items;

determine a three-dimensional graphical model that approximates a shape of the first user and;

display, using the second browser application, in a second region of a second Web page, a view of the at least one clothing item to appear to be worn by the first user based at least in part on the current orientation of the first user, the second Web page including a second set of one or more clothing items, the second set of one or more clothing items being displayed outside the second gesture module, at least one clothing item of the second set of one or more clothing items being different from each clothing item of the first set of one or more clothing items.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed further cause the computing device to:

update the view of the first user with changes in the current orientation of at least a portion of the user to cause the item to appear to move with the user in the displayed view.

23. The non-transitory computer-readable storage medium of claim 21, wherein displaying, using the second browser application, includes identifying at least one of an arm or a leg portion of the clothing item represented in the image information and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

24. The non-transitory computer-readable storage medium of claim 21, wherein displaying, using the second browser application, includes identifying a shape of a portion of the first user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.