Augmented reality product preview

• US 10,192,364 B2
• Filed: 07/27/2017
• Issued: 01/29/2019
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Mott, David Creighton, Dhua, Arnab Sanat Kumar, Taylor, Colin Jon, Lou, Yu, Wang, Chun-Kai, Pantham, Sudeshna, Arora, Himanshu, Zhang, Xi
• Primary Examiner(s): Merouan, Abderrahim
• Application Number:
  US15/662,047
• Publication Number:
  US 20170323488A1
• Time in Patent Office: 551 Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

Systems and methods for displaying 3D containers in a computer generated environment are described. A computing device may provide a user with a catalog of objects which may be purchased. In order to view what an object may look like prior to purchasing the object, a computing device may show a 3D container that has the same dimensions as the object. As discussed herein, the 3D container may be located and oriented based on a two-dimensional marker. Moreover, some 3D containers may contain a representation of an object, which may be a 2D image of the object.

Claims

1. A computing device comprising:

at least one processor;

a camera;

a display screen; and

memory including instructions that, when executed by the at least one processor, cause the computing device to;

obtain image data of a physical environment containing a marker using the camera;

analyze the image data to determine a first position of the marker with respect to the computing device;

display, on the display screen in an interface, a representation of an item associated with the marker, the representation of the item appearing within a virtual container at a second position and determined based at least in part on dimensions associated with the item;

determine a first overlap value between the virtual container and an object located in the physical environment that is represented in the interface;

determine that the first overlap value fails to satisfy an overlap threshold; and

provide a listing of items, wherein a representation of an item of the listing of items, when positioned at the second position, is associated with a second overlap value that satisfies the overlap threshold, wherein the overlap threshold corresponds to the representation of the item of the listing of times having no overlap with the object represented in the interface.

2. The computing device of claim 1, wherein the instructions, when executed to display the representation of the item, further enable the computing device to:

determine at least one dimension of the marker;

determine an orientation of the computing device with respect to the marker; and

determine a plane of the marker with respect to the computing device.

3. The computing device of claim 1, wherein the virtual container includes a semi-transparent container within which a graphical representation of the item is presented, and wherein the semi-transparent container is rendered in the interface based at least in part on the dimensions of the marker and dimensions of the item.

4. The computing device of claim 1, wherein the instructions, when executed by the processor, further enable the computing device to:

receive a selection on the interface corresponding to an area of the physical environment; and

update the listing of items to include items having corresponding representations that when positioned in the area are associated with respective probabilities of overlap values that satisfy the overlap threshold.

5. The computing device of claim 4, wherein the selection includes one of a touch-based input on the interface or a selection of one of a plurality of identified areas of the physical environment represented on the interface.

6. The computing device of claim 4, wherein the instructions, when executed to update the listing of items, further enable the computing device to:

determine dimensions of the area; and

submit a search query that includes at least one of a search term or the dimensions.

7. The computing device of claim 1 wherein the interface includes a control interface to modify dimensions of the virtual container;

and wherein modifying the dimensions of the virtual container updates the listing of items.

8. The computing device of claim 1, wherein the instructions, when executed by the processor, further enable the computing device to:

receive input to move the representation of the item; and

move the representation of the item along a plane that the marker is located on or along a plane that is parallel to the plane that the marker is located on.

9. The computing device of claim 1, wherein the instructions, when executed by the processor, further enable the computing device to:

acquire metadata associated with the item; and

determine the dimensions of the item based on the metadata.

10. The computing device of claim 1, wherein the instructions, when executed by the processor, further enable the computing device to:

determine objects in the physical environment; and

display the representation of the item such that it does not overlap with the objects in the physical environment that are displayed in the interface.

11. A computer-implemented method, comprising:

obtaining image data of a physical environment containing a marker using a camera of a computing device;

analyzing the image data to determine a first position of the marker with respect to the computing device;

displaying, on a display screen of the computing device in an interface, a representation of an item associated with the marker, the representation of the item appearing within a virtual container at a second position and determined based at least in part on dimensions associated with the item;

determining a first overlap value between the virtual container and an object located in the physical environment that is represented in the interface;

determining that the first overlap value fails to satisfy an overlap threshold; and

providing a listing of items, wherein a representation of an item of the listing of items, when positioned at the second position, is associated with a second overlap value that satisfies the overlap threshold.

12. The computer-implemented method of claim 11, further comprising:

receiving a selection on the interface corresponding to an area of the physical environment; and

updating the listing of items to include items having corresponding representations that when positioned in the area are associated with respective overlap values that satisfy the overlap threshold, wherein the selection includes one of a touch-based input on the interface or a selection of one of a plurality of identified areas of the physical environment represented on the interface.

13. The computer-implemented method of claim 12, wherein updating the listing of items further includes:

determining dimensions of the area; and

submitting a search query that includes at least one of a search term or the dimensions.

14. The computer-implemented method of claim 11, further comprising:

receiving input to move the representation of the item; and

moving the representation of the item along a plane that the marker is located on or along a plane that is parallel to the plane that the marker is located on.

15. The computer-implemented method of claim 11, further comprising:

determining objects in the physical environment; and

displaying the representation of the item such that it does not overlap with the objects in the physical environment that are displayed in the interface.

16. A computing system comprising:

at least one processor;

memory including instructions that, when executed by the at least one processor, cause the computing system to;

obtain image data of a physical environment containing a marker using a camera of a computing device;

analyze the image data to determine a first position of the marker with respect to the computing device;

cause to be displayed, on a display screen of the computing device in an interface, a representation of an item associated with the marker, the representation of the item appearing within a virtual container at a second position and determined based at least in part on dimensions associated with the item;

determine a first overlap value between the virtual container and an object located in the physical environment that is represented in the interface;

determine that the first overlap value fails to satisfy an overlap threshold; and

provide a listing of items, wherein a representation of an item of the listing of items, when positioned at the second position, is associated with a second overlap value that satisfies the overlap threshold.

17. The computing system of claim 16, wherein the instructions, when executed by the processor, further enable the computing system to:

receive a selection on the interface corresponding to an area of the physical environment; and

update the listing of items to include items having corresponding representations that when positioned in the area are associated with respective overlap values that satisfy the overlap threshold, wherein the selection includes one of a touch-based input on the interface or a selection of one of a plurality of identified areas of the physical environment represented on the interface.

18. The computing system of claim 17, wherein the instructions, when executed by the processor to update the listing of items, further enable the computing system to:

determine dimensions of the area; and

submit a search query that includes at least one of a search term or the dimensions.

19. The computing system of claim 16, wherein the instructions, when executed by the processor, further enable the computing system to:

receive input to move the representation of the item; and

move the representation of the item along a plane that the marker is located on or along a plane that is parallel to the plane that the marker is located on.

20. The computing system of claim 16, wherein the instructions, when executed by the processor, further enable the computing system to:

determine objects in the physical environment; and

display the representation of the item such that it does not overlap with the objects in the physical environment that are displayed in the interface.