Augmented reality system, method, and apparatus for displaying an item image in a contextual environment

• US 9,164,577 B2
• Filed: 12/22/2009
• Issued: 10/20/2015
• Est. Priority Date: 12/22/2009
• Status: Expired due to Fees

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Tapley, John, Farraro, Eric J.
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): NGUYEN, PHONG X
• Application Number:
  US12/644,957
• Publication Number:
  US 20110148924A1
• Time in Patent Office: 2,128 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Method, apparatus, and system for providing an item image to a client for display in a contextual environment are described. In some embodiments, the user may select an item for display in the contextual environment, and the user may position a camera coupled to a processing system to capture the contextual environment. A marker may be generated and associated with an item selected by a user. In an embodiment, the generated marker may be placed in a location within the contextual environment, and the user'"'"'s processing system may send a visual data stream of the camera-captured environment to a server. In an embodiment, the user'"'"'s processing device may receive a modified data stream including an image of the item, and the user'"'"'s processing device may display the item image in the same location as the marker.

Claims

1. A computer-implemented method comprising:

receiving visual data from a client, the visual data including placeholder information associated with a placeholder, the placeholder comprising;

a first portion operable to provide detection of the placeholder information;

a second portion operable to provide placement information related to the placeholder, the second portion including at least two shaded objects that are dissimilar in color, the at least two shaded objects being operable to provide an orientation of the placeholder and a distance of the placeholder from a camera; and

a set of unique markings operable to identify an item corresponding to the placeholder;

detecting the placeholder information included with the visual data using the first portion of the placeholder;

determining the placement information using the second portion of the placeholder, the determining of the placement information comprising determining the orientation of the placeholder and the distance of the placeholder from the camera using the at least two shaded objects of the second portion of the placeholder;

identifying the item corresponding to the placeholder based on the set of unique markings of the placeholder;

retrieving information about the item using the set of unique markings of the placeholder;

selecting an image of the item from a plurality of images of the item based on the determined placement information, the selected image of the item being a best fit image according to physical dimensions of the item;

retrieving the selected image of the item; and

using a processor, modifying the received visual data to include image data associated with the retrieved image and to include the retrieved information about the item with reference to the location of the placeholder.

2. The method of claim 1, further comprising:

receiving an input including an item selection; and

providing generation information for generating the placeholder based on the received input.

3. The method of claim 1, wherein the determining of the placement information further comprises determining a scale factor.

4. The method of claim 3, wherein the selecting of the image of the item is based on the determined scale factor.

5. The method of claim 3, wherein the retrieving of the selected image of the item associated with the placeholder further comprises:

determining dimensions of the item; and

modifying the retrieved image of the item in reference to the determined dimensions and the determined scale factor of the item.

6. The method of claim 3, wherein the determining of the scale factor further comprises detecting at least one of a change in contrast or edge portions of the placeholder.

7. The method of claim 1, further comprising determining characteristics of the placeholder based on the placeholder information, wherein the selecting the image of the item from the plurality of images of the item is based on the determined characteristics of the placeholder.

8. The method of claim 1, wherein the retrieved information includes a suggested item related to the item.

9. The method of claim 1, wherein the determining of the placement information further comprises:

determining a scale factor based on the distance of the placeholder from a camera; and

determining an orientation factor based on an angle of the placeholder in relation to the camera.

10. The method of claim 9, wherein the modifying of the received visual data to include the retrieved image with reference to the location of the placeholder further comprises:

modifying the retrieved image with reference to the scale factor and the orientation factor; and

modifying the received visual data with modified retrieved image data.

11. A system comprising:

a processor-implemented visual data receiver module to receive visual data from a client, the visual data including placeholder information associated with a placeholder, the placeholder comprising;

a first region operable to provide detection of the placeholder;

a second region operable to identify a location and orientation of the placeholder, the second region including at least two shaded objects that are dissimilar in color, the at least two shaded objects being operable to provide an orientation of the placeholder and a distance of the placeholder from a camera; and

a set of unique markings operable to identify an item associated with the placeholder;

a processor-implemented detection module to detect the placeholder information using the first region of the placeholder;

a processor-implemented placement determination module to determine placement information using the second region of the placeholder, the determining of the placement information comprising determining the orientation of the placeholder and the distance of the placeholder from the camera using the at least two shaded objects of the second portion of the placeholder;

a processor-implemented retrieval module to identify the item associated with the placeholder based on the set of unique markings of the placeholder, the processor-implemented retrieval module further to retrieve information about the item using the set of unique markings, the processor-implemented retrieval module further to select an image of the item from a plurality of images of the item based on the placement information, the selected image of the item being a best fit image according to physical dimensions of the item, the processor-implemented retrieval module further to retrieve the selected image of the item; and

a processor-implemented modification module to modify the received visual data to include image data associated with the retrieved image and to include the retrieved information about the item at the location of the placeholder.

12. The system of claim 11, further comprising:

a processor-implemented input receiver module for receiving an input including an item selection; and

a processor-implemented information provider module for providing generation information for generating the placeholder based on the received input.

13. The system of claim 11, wherein the processor-implemented placement determination module is to determine a scale factor.

14. The system of claim 11, wherein the information about the item includes a price of the item.

15. The system of claim 13, wherein the processor-implemented image retrieval module is to:

determine dimensions of the item; and

modify the retrieved image of the item utilizing the determined dimensions and the determined scale factor of the item.

16. The system of claim 11, wherein the processor-implemented detection module is further to detect at least one of a change in contrast of the placeholder or edges of the placeholder.

17. The system of claim 11 wherein the processor-implemented modification module is further to retrieve information associated with the item for display with the item image.

18. The system of claim 11, wherein the processor-implemented placement determination module is further to determine parameters of the placeholder based on the placeholder information, and wherein the processor-implemented retrieval module is further to select the image of the item from the plurality of images of the item based on determined parameters of the placeholder.

19. The system of claim 11, wherein the processor-implemented placement determination module is further to determine a scale factor based on the distance of the placeholder from a camera, and to determine an orientation factor based on an angle of the placeholder in relation to the camera.

20. A non-transitory machine-readable medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving an input including a selection of an item;

providing generation information for generating a placeholder based on the received input, the placeholder comprising;

a first portion operable to identify the placeholder;

a second portion operable to identify a location and orientation of the placeholder, the second portion including at least two shaded objects that are dissimilar in color and operable to provide an orientation of the placeholder and a distance of the placeholder from a camera; and

a set of unique markings operable to identify the item;

receiving visual data from a client;

detecting placeholder information included with the visual data using the first portion of the placeholder;

determining the placement information using the second portion of the placeholder, the determining of the placement information comprising determining the orientation of the placeholder and the distance of the placeholder from the camera using the at least two shaded objects of the second portion of the placeholder;

identifying the item associated with the placeholder based on the set of unique markings of the placeholder;

retrieving information about the item using the set of unique markings of the placeholder;

selecting an image of the item from a plurality of images of the item based on the placement information, the selected image of the item being a best fit image according to physical dimensions of the item;

retrieving the selected image of the item associated with the placeholder determined based on the unique markings of the placeholder; and

modifying the received visual data to include image data associated with the retrieved image and to include the retrieved information about the item.