System and method for visualization of items in an environment using augmented reality

• US 10,628,877 B2
• Filed: 10/16/2018
• Issued: 04/21/2020
• Est. Priority Date: 10/27/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Sacco, Nathan
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US16/162,153
• Publication Number:
  US 20190050939A1
• Time in Patent Office: 553 Days
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Systems and methods for visualization of an item in an environment using augmented reality are provided. Environment image data containing an image of an environment is received. A selection of an item for placement into an indicated location of the environment is received. An item image of the selected item is scaled based on dimensions determined from the environment image data for the environment. The scaled item image is augmented into the image of the environment at the indicated location to generate an augmented reality image. The augmented reality image is displayed on a device of a user, whereby the scaled item image in the augmented reality image is selectable to cause display of information. A selection of the scaled item image is received. In response to the selection of the scaled item image, the information is presented on the device of the user.

Claims

1. A method implemented by a client device, the method comprising:

capturing, by an image capture device of a client device, an image of an environment;

determining, by the client device, dimensions of the environment by identifying an object having known standard dimensions in the environment;

receiving, by the client device, a selection of an item for placement into the image of the environment;

receiving, by the client device, an indication of a location in the environment at which the item is to be placed, the location at which the item is to be placed being different from a location of the object in the environment;

scaling, by the client device, an image of the selected item based on the determined dimensions of the environment and dimensions of the selected item; and

generating, by the client device, an augmented image of the environment that includes the scaled image of the selected item.

2. The method of claim 1, further comprising:

determining, by the client device, a first distance between the client device and the location at which the item is to be placed; and

scaling, by the client device, the image of the selected item based on the first distance.

3. The method of claim 2, wherein determining the distance between the client device and the location at which the item is to be placed is performed using environment image data derived from a focus capability of the image capture device of the client device.

4. The method of claim 2, wherein determining the distance between the client device and the location at which the item is to be placed is performed using environment image data derived from a sound generator of the client device.

5. The method of claim 2, further comprising:

determining, by the client device, a second distance between the client device and the object having known dimensions; and

scaling, by the client device, the image of the selected item based on the first distance and the second distance.

6. The method of claim 5, further comprising:

determining, by the client device, an angle between the location at which the item is to be placed and the object having known dimensions;

determining, by the client device, a distance between the location at which the item is to be placed and the object having known dimensions using the first distance, the second distance, and the angle; and

scaling, by the client device, the image of the selected item based on the distance between the location at which the item is to be placed and the object having known dimensions.

7. The method of claim 1, wherein the image of the environment comprises a video of the environment, the method further comprising performing, by the client device, the scaling and the generating for individual frames of the video of the environment.

8. The method of claim 1, further comprising determining, by the client device, dimensions of the item by obtaining item data for the item from an item catalog.

9. A system comprising:

at least one processor; and

a storage device comprising instructions that are executable by the at least one processor to perform operations comprising;

capturing an image of an environment;

determining dimensions of the environment by identifying an object having known standard dimensions in the environment;

receiving a selection of an item for placement into the image of the environment;

receiving an indication of a location in the environment at which the item is to be placed, the location at which the item is to be placed being different from a location of the object in the environment;

scaling an image of the selected item based on the determined dimensions of the environment and dimensions of the selected item; and

generating an augmented image of the environment that includes the scaled image of the selected item.

10. The system of claim 9, the operations further comprising:

determining a distance between the location at which the item is to be placed and an image capture device used to capture the image of the environment; and

scaling the image of the selected item based on the distance.

11. The system of claim 10, wherein determining the distance between the location at which the item is to be placed and the image capture device is performed using environment image data derived from a focus capability of the image capture device.

12. The system of claim 10, wherein determining the distance between the location at which the item is to be placed and the image capture device is performed using environment image data derived from a sound generator of the image capture device.

13. The system of claim 9, wherein the image of the environment comprises a video of the environment, the operations further comprising performing the scaling and the generating for each frame of the video of the environment.

14. The system of claim 9, the operations further comprising determining dimensions of the item by obtaining item data for the item from a database.

15. A non-transitory machine-readable storage medium having instructions stored thereon that are executable by a computing device to perform operations comprising:

capturing an image of an environment;

determining dimensions of the environment by identifying an object having known standard dimensions in the environment;

receiving a selection of an item for placement into the image of the environment;

receiving an indication of a location in the environment at which the item is to be placed, the location at which the item is to be placed being different from a location of the object in the environment;

scaling an image of the selected item based on the determined dimensions of the environment and dimensions of the selected item; and

generating an augmented image of the environment that includes the scaled image of the selected item.

16. The non-transitory machine-readable storage medium of claim 15, the operations further comprising:

determining a distance between a computing device used to capture the image of the environment and the location at which the item is to be placed; and

scaling the image of the selected item based on the distance.

17. The non-transitory machine-readable storage medium of claim 16, wherein determining the distance between the client device and the location at which the item is to be placed is performed using one or more of environment image data derived from a focus capability of the image capture device of the client device or a sound generator of the client device.

18. The non-transitory machine-readable storage medium of claim 15, the operations further comprising orienting the scaled image of the item relative to the environment prior to generating the augmented image.

19. The non-transitory machine-readable storage medium of claim 15, the operations further comprising determining dimensions of the item by obtaining item data for the item from a database.

20. The non-transitory machine-readable storage medium of claim 15, wherein the image of the environment comprises a video of the environment, the operations further comprising performing the scaling and the generating for individual frames of the video of the environment.