System and method for visualization of items in an environment using augmented reality

• US 9,449,342 B2
• Filed: 10/27/2011
• Issued: 09/20/2016
• Est. Priority Date: 10/27/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Sacco, Nathan
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US13/283,416
• Publication Number:
  US 20130106910A1
• Time in Patent Office: 1,790 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

In various example embodiments, a system and method for visualization of an item in an environment using augmented reality is provided. In example embodiments, environment image data containing an image of an environment is received from a client device. A selection of an item that is under consideration for purchase and placement into an indicated location of the environment is received. An item image of the selected item is scaled to a scale that is based on dimensions determined from the environment image data for the environment. The scaled item image is augmented into the image of the environment at the indicated location to generate an augmented reality image.

Claims

1. A method comprising:

receiving, from a client device, environment image data containing an image of an environment captured by the client device;

receiving a selection of an item that is under consideration for purchase and placement into an indicated location of the image of the environment;

determining a distance to the indicated location in the image of the environment, the distance being from a point of view of the client device that captured the image of the environment;

in response to receiving the selection of the item, scaling an item image of the selected item based on dimensions and the distance determined from the environment image data for the environment;

augmenting, using at least one processor of an application server, the scaled item image into the image of the environment at the indicated location to generate an augmented reality image;

providing the augmented reality image to a display device of a user, the scaled item image in the augmented reality image being selectable to cause display of additional information;

receiving a selection of the scaled item image in the augmented reality image; and

in response to the receiving the selection of the scaled item image, causing, by the application server, presentation of the additional information.

2. The method of claim 1, wherein the causing presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of

at least one alternative recommendation, the least one alternative recommendation being a smaller or larger item.

3. The method of claim 1, wherein the determining the distance to the indicated location comprises using environment image data derived from a focus capability of an image capture device associated with the client device to determine the distance.

4. The method of claim 1, further comprising determining the dimensions for the environment using a marker located in the environment.

5. The method of claim 1, further comprising:

determining an orientation of the indicated location; and

orienting the item image to the determined orientation of the indicated location.

6. The method of claim 1, wherein the image of the environment comprises a video of the environment, the scaling of the item image and the augmenting of the scaled item image being repeatedly performed for the video.

7. The method of claim 1, wherein the receiving of the selection of the item comprises receiving the item image for the item.

8. The method of claim 1, further comprising obtaining item data for the item from an item catalog, the item data including dimension data for the item.

9. The method of claim 1, further comprising extracting item data from a publication associated with the selection.

10. The method of claim 1, wherein the causing the presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of at least one alternative recommendation, the least one alternative recommendation being an item that is less expensive than the item.

11. The method of claim 1, wherein the causing presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of shopping information including a lowest, highest, and average price along with links to various marketplaces where the item may be purchased.

12. The method of claim 1, wherein the causing presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of a window presented over the augmented reality image that displays a dimension, weight, item identifier, and product description of the item.

13. The method of claim 1, wherein the causing the presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of at least one alternative recommendation, the least one alternative recommendation being a newer version of the item.

14. The method of claim 1, wherein the determining the distance to the indicated location comprises using environment image data based on an echo technique performed by a sound generator of the client device to determine the distance.

15. The method of claim 1, wherein the causing the presentation of the additional information in response to the receiving the selection of the scaled item image comprises causing presentation of at least one alternative recommendation, the least one alternative recommendation being an item that ranks higher than the item.

16. A system comprising:

at least one processor of a machine and executable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising;

receiving, from a client device, environment image data containing an image of an environment captured by the client device;

receiving a selection of an item that is under consideration for purchase and placement into an indicated location of the image of the environment;

determining a distance to the indicated location in the image of the environment, the distance being from a point of view of the client device that captured the image of the environment;

scaling an item image of the selected item based on dimensions and the distance determined from the environment image data for the environmental image data in response to receiving the selection of the item;

augmenting the scaled item image into the image of the environment at the indicated location to generate an augmented reality image;

providing the augmented reality image to a display device of a user, the scaled item image in the augmented reality image being selectable to cause display of additional information;

receiving a selection of the scaled item image in the augmented reality image; and

in response to the receiving the selection of the scaled item image causing presentation of the additional information.

17. The system of claim 16, wherein the operations further comprise determining the dimensions for the environment using a marker located in the environment.

18. The system of claim 16, wherein the operations further comprise

determining an orientation of the indicated location, and

orienting the item image to the determined orientation of the indicated location.

19. The system of claim 16, wherein the operations further comprise obtaining item data for the item from an item catalog, the item data including dimension data for the item.

20. A non-transitory machine-readable medium storing instructions which, when executed by the at least one processor of a machine, causes the machine to perform operations comprising:

receiving, from a client device, environment image data containing an image of an environment captured by the client device;

receiving a selection of an item that is under consideration for purchase and placement into an indicated location of the image of the environment;

determining a distance to the indicated location in the image of the environment, the distance being from a point of view of the client device that captured the image of the environment;

in response to receiving the selection of the item, scaling an item image of the selected item to a scale based on dimensions and the distance determined from the environment image data for the environment;

augmenting the scaled item image into the image of the environment at the indicated location to generate an augmented reality image;

providing the augmented reality image to a display device of a user, the scaled item image in the augmented reality image being selectable to cause display of additional information;

receiving a selection of the scaled item image in the augmented reality image; and

in response to the receiving the selection of the scaled item image causing presentation of the additional information.