STRUCTURED LIGHTING BASED CONTENT INTERACTIONS

• US 20130249937A1
• Filed: 03/20/2012
• Published: 09/26/2013
• Est. Priority Date: 03/20/2012
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Amacker, Matthew W., Tesler, Joel D.
• Granted Patent: US 9,213,420 B2
• Time in Patent Office: Days
• US Class Current: 345/619
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0641 Shopping interfaces

Abstract

Information from a position and/or gesture detection system can be embedded in a Web page, or other such presentation of content, and used to select or otherwise interact with content on the page. In some embodiments, video is captured and displayed showing a current view of the user. Position data corresponding to the video is provided and used to determine directions and extents of motion without having to do significant amounts of image processing. The position data is used to determine locations on the page where the user is attempting to provide input, such as to select an item of content. The content can be modified and/or rendered to appear to be associated with the user in the rendered view. Information from multiple gesture systems can be combined on a single page, and information from one or more gesture systems can be shared among multiple pages and devices.

Claims

1. A computer-implemented method of rendering content on a computing device, comprising:

receiving position data and image data from a gesture system, the image data including a view of a current orientation or a user and the position data corresponding to features of the user at the current orientation;

providing a view of the image data through a module on a Web page, the view enabling the user to see a mirror image of the current orientation of the user;

displaying images of one or more clothing items on the Web page;

analyzing the position data to determine at least one of a motion or gesture of the user, the motion or gesture of the user capable of indicating at least one of the one or more clothing items displayed outside the gesture module on the Web page; and

in response to the motion or gesture indicating one of the images of the clothing items;

determining a current orientation of the user using the position data;

modifying an appearance of the indicated image of the clothing item to correspond to a portion of the current orientation; and

rendering the modified image with respect to the view of the user on the Web page such that the clothing item is displayed as if worn by the user.

2. The computer-implemented method of claim 1, wherein the gesture system includes at least one video camera capable of providing video image data and at least one structured light subsystem operable to provide the position data.

3. The computer-implemented method of claim 1, wherein the one or more clothing items are capable of being of different types, and wherein the appearance of the indicated image is modified based at least in part upon the type of the clothing item.

4. The computer-implemented method of claim 1, wherein modifying an appearance of the indicated image of the clothing item includes identifying at least one of an arm or a leg portion of the clothing item in the image and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

5. The computer-implemented method of claim 1, wherein modifying an appearance of the indicated image of the clothing item includes identifying a shape of a portion of the user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.

7. The computer-implemented method of claim 1, further comprising:

obtaining position data representing positions of one or more features of the user in the current orientation.

8. The computer-implemented method of claim 7, wherein the position data and image data are obtained from a gesture detection system.

9. The computer-implemented method of claim 8, wherein the gesture detection system includes at least one camera and at least one illumination source configured to project a pattern of structured light.

10. The computer-implemented method of claim 9, wherein the at least one illumination source projects at least one of visible light or infrared light.

11. The computer-implemented method of claim 8, wherein the gesture detection system is part of a computing device displaying the view of the user.

6. A computer-implemented method, comprising:

under control of one or more computer systems configured with executable instructions,

obtaining image information showing a current orientation of at least a portion of a user;

determining an image of at least one item to be displayed with a view of the user;

modifying the image to correspond to the current orientation;

rendering the modified image with respect to a display of the view of the user in order to cause the view of the user to appear to include the at least one item; and

update the modifying of the image with changes in the current orientation of at least a portion of the user to cause the at least one item to appear to move with the user in the displayed view.

12. The computer-implemented method of claim 6, further comprising:

analyzing the image information to identify at least one facial feature of the user.

13. The computer-implemented method of claim 12, wherein at least one accessory is capable of being rendered on the view of the user corresponding to at least one of a position or orientation of the at least one facial feature in the displayed view of the user.

14. The computer-implemented method of claim 12, wherein a selection of one or more items capable of being displayed with the view of the user is determined at least in part based upon an identity or gender of the user as determined through an analysis of the image information.

15. The computer-implemented method of claim 6, wherein modifying the image includes at least one of rotating, shearing, skewing, stretching, compressing or translating at least a portion of the image.

16. The computer-implemented method of claim 6, wherein the view of the user is provided through a gesture module on a Web page, and wherein the image of the at least one item is obtained from outside the gesture module.

17. A computing device, comprising:

a device processor;

a display screen; and

a memory device including instructions operable to be executed by the device processor to perform a set of actions, enabling the computing device to;

obtain image information showing a current orientation a user;

determine an image of a clothing item to be displayed with a view of the user;

modify the determined image to correspond to the current orientation; and

render the modified image with respect to a display of the view of the user on the display screen in order to cause the view of the user to appear as if the user is wearing the clothing item.

18. The computing device of claim 17, wherein the instructions when executed further cause the computing device to:

update the modifying of the image with changes in the current orientation of at least a portion of the user to cause the item to appear to move with the user in the displayed view.

19. The computing device of claim 17, further comprising:

a video camera capable of providing the image information.

20. The computing device of claim 17, further comprising:

at least one illumination element operable to project a pattern of light; and

at least one sensor configured to detect reflections of the pattern of light,

wherein the device processor is operable to process the detected reflections to determine position data for the current orientation of the user.

21. The computing device of claim 20, wherein the at least one illumination element and the at least one sensor are part of a gesture subsystem.

22. The computing device of claim 17, wherein modifying an appearance of the determined image of the clothing item includes identifying at least one of an arm or a leg portion of the clothing item in the image and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

23. The computing device of claim 17, wherein modifying an appearance of the determined image of the clothing item includes identifying a shape of a portion of the user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.

24. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive image information showing a current orientation a user;

determine an image of a clothing item to be displayed with a view of the user;

modify the determined image to correspond to the current orientation; and

render the modified image with respect to a display of the view of the user in order to cause the view of the user to appear as if the user is wearing the clothing item.

25. The non-transitory computer-readable storage medium of claim 24, wherein the instructions when executed further cause the computing device to:

update the modifying of the image with changes in the current orientation of at least a portion of the user to cause the item to appear to move with the user in the displayed view.

26. The non-transitory computer-readable storage medium of claim 24, wherein the instructions when executed further cause the computing device to:

obtain position data representing positions of one or more features of the user in the current orientation, the position data and image data being obtained from a gesture detection system.

27. The non-transitory computer-readable storage medium of claim 24, wherein modifying an appearance of the determined image of the clothing item includes identifying at least one of an arm or a leg portion of the clothing item in the image and adjusting at least one of a position or orientation of at least a portion of the identified arm or leg portion to correspond to the portion of the current orientation.

28. The non-transitory computer-readable storage medium of claim 24, wherein modifying an appearance of the determined image of the clothing item includes identifying a shape of a portion of the user represented in the view and manipulating an appearance of the clothing item in the indicated image to correspond to the identified shape.