Augmented reality system, method, and apparatus for displaying an item image in a contextual environment

• US 10,210,659 B2
• Filed: 09/28/2015
• Issued: 02/19/2019
• Est. Priority Date: 12/22/2009
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Tapley, John, Farraro, Eric J.
• Primary Examiner(s): Nguyen, Phong X
• Application Number:
  US14/868,105
• Publication Number:
  US 20160019723A1
• Time in Patent Office: 1,240 Days
• Field of Search: 345607, 345629, 345630, 345632, 345633
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Method, apparatus, and system for providing an item image to a client for display in a contextual environment are described. In some embodiments, the user may select an item for display in the contextual environment, and the user may position a camera coupled to a processing system to capture the contextual environment. A placeholder may be generated and associated with an item selected by a user. In an embodiment, the generated placeholder may be placed in a location within the contextual environment, and the user'"'"'s processing system may send a visual data stream of the camera-captured environment to a server. In an embodiment, the user'"'"'s processing device may receive a modified data stream including an image of the item, and the user'"'"'s processing device may display the item image in the same location as the placeholder.

Claims

1. A method comprising:

receiving, from a client device, a request to generate a placeholder for a particular item selected by a user;

in response to receiving the request, generating placeholder generation information for the placeholder for the particular item, the placeholder generation information including information that is specific to the particular item;

providing the placeholder generation information to the client device;

receiving visual data from a camera of the client device, the visual data including placeholder information associated with the placeholder;

determining placement information using a first portion of the placeholder, the first portion of the placeholder comprising two or more shaded objects of dissimilar color, the determining of the placement information comprising;

determining an orientation of the placeholder using the two or more shaded objects of the first portion of the placeholder; and

determining a distance of the placeholder from the camera using the two or more shaded objects of the first portion of the placeholder;

identifying the particular item corresponding to the placeholder based on a second portion of the placeholder;

retrieving, from a networked-database, an image of the item from among a plurality of images of the item based on the determined placement information, the retrieved image of the item being a best fit image according to physical dimensions of the item and the determined placement information; and

modifying, using a processor of a machine, the received visual data to include image data associated with the retrieved image with reference to the placeholder.

2. The method of claim 1, further comprising:

receiving an input including an item selection.

3. The method of claim 1, further comprising determining a scale factor based on the placement information.

4. The method of claim 3, wherein the retrieving of the image of the item associated with the placeholder further comprises selecting the image with reference to the determined scale factor.

5. The method of claim 3, wherein the retrieving of the image of an item comprises:

determining dimensions of the item; and

modifying the retrieved image of the item in reference to the determined dimensions and the determined scale factor of the item.

6. The method of claim 3, wherein the determining of the scale factor further comprises detecting at least one of a change in contrast and edge portions of the placeholder.

7. The method of claim 1, wherein the retrieving of the image of the item comprises:

accessing a database of images; and

selecting the image of the item from a plurality of images based on the placement information.

8. The method of claim 1, wherein the modifying of the received visual data further comprises replacing the placeholder information with item image information.

9. The method of claim 1, wherein the determining of placement information further comprises:

determining a scale factor based on the distance of the placeholder from the camera; and

determining an orientation factor based on an angle of the placeholder in relation to the camera.

10. The method of claim 9, wherein the modifying of the received visual data to include image data associated with the retrieved image with reference to the placeholder further comprises:

modifying the retrieved image with reference to the scale factor and the orientation factor; and

modifying the received visual data with data associated with the modified retrieved image.

11. A system comprising:

a processor-implemented input receiver module to receive, from a client device, a request to generate a placeholder for a particular item selected by a user;

a processor-implemented information provider module to generate, responsive to receiving the request, placeholder generation information for the placeholder for the particular item, the placeholder generation information including information that is specific to the particular item, the processor-implemented information provider module further to provide the placeholder generation information to the client device;

a processor implemented visual data receiver module to receive visual data from a camera of the client device, the visual data including placeholder information associated with the placeholder;

a processor-implemented placement determination module to determine placement information using a first portion of the placeholder, the first portion of the placeholder comprising two or more shaded objects of dissimilar color, the placement determination module to determine the placement information by performing operations comprising;

determining an orientation of the placeholder using the two or more shaded objects of the first portion of the placeholder; and

determining a distance of the placeholder from the camera using the two or more shaded objects of the first portion of the placeholder;

a processor-implemented retrieval module to identify the particular item corresponding to the placeholder based on a second portion of the placeholder, the retrieval module further to retrieve an image of the item from among a plurality of images of the item based on the determined placement information, the retrieved image of the item being a best fit image according to physical dimensions of the item and the determined placement information; and

a processor-implemented modification module to modify the received visual data to include image data associated with the retrieved image with reference to the placeholder.

12. The system of claim 11,

wherein the processor-implemented input receiver module is further to receive an input including an item selection.

13. The system of claim 11, wherein the processor-implemented placement determination module is further to determine a scale factor.

14. The system of claim 11, wherein the processor-implemented retrieval module is further to detect a unique marker in the second portion of the placeholder.

15. The system of claim 13, wherein the processor-implemented retrieval module is further to:

determine dimensions of the item; and

modify the retrieved image of the item utilizing the determined dimensions and the determined scale factor of the item.

16. The system of claim 11, further comprising a processor-implemented detection module to detect at least one of a change in contrast of the placeholder and edges of the placeholder.

17. The system of claim 11, wherein the processor-implemented modification module is further to retrieve information associated with the item for display with the retrieved image of the item.

18. The system of claim 11, wherein the processor-implemented retrieval module is to access a database of images, and select the image of the item from a plurality of images of the item based on the determined placement information.

19. The system of claim 11, wherein the processor-implemented placement determination module is further to determine a scale factor based on the distance of the placeholder from the camera, and to determine an orientation factor based on an angle of the placeholder in relation to the camera.

20. A non-transitory machine-readable medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving, from a client device, a request to generate a placeholder for a particular item selected by a user;

in response to receiving the request, generating placeholder generation information for the placeholder for the particular item, the placeholder generation information including information that is specific to the particular item;

providing the placeholder generation information to the client device;

receiving visual data from a camera of the client device, the visual data including placeholder information associated with the placeholder;

determining placement information using a first portion of the placeholder, the first portion of the placeholder comprising two or more shaded objects of dissimilar color, the determining of the placement information comprising;

determining an orientation of the placeholder using the two or more shaded objects of the first portion of the placeholder; and

determining a distance of the placeholder from the camera using the two or more shaded objects of the first portion of the placeholder;

identifying the particular item corresponding to the placeholder based on a second portion of the placeholder;

retrieving, from a networked database, an image of the item from among a plurality of images of the item based on the determined placement information, the retrieved image of the item being a best fit image according to physical dimensions of the item and the determined placement information; and

modifying the received visual data to include image data associated with the retrieved image with reference to the placeholder.