Augmented reality system and method for visualizing an item

• US 10,127,606 B2
• Filed: 07/29/2011
• Issued: 11/13/2018
• Est. Priority Date: 10/13/2010
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Tapley, John, Beach, David, Pugazhendhi, Bagath
• Primary Examiner(s): Georgalas, Anne M
• Application Number:
  US13/194,584
• Publication Number:
  US 20120192235A1
• Time in Patent Office: 2,664 Days
• Field of Search: 705 261, 705 271
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: G09G 2352/00 Parallel handling of streams of display data
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

In a system and method for searching, browsing, and visualizing items, a first image containing an item is received to be stored in an account associated with a user. The first image is processed using a processor to extract an image of the item from the first image. The image of the item is layered over a stream of video frames received from the client device to form a superimposed image. The superimposed image is caused to be displayed on the client device.

Claims

1. A computer-implemented method, comprising:

retrieving a first image from a commerce system that stores the first image in association with an account of a user of a client device, the retrieved first image depicting a background and depicting an item;

identifying edges of the item using image processing edge detection;

within the first image, distinguishing the background from the item based on the identified edges of the item;

making the background of the first image transparent;

accessing a video stream capturing a live camera feed of a reflection of the user on a reflective surface, the live camera feed being captured by an image capture device of the client device pointed at the reflective surface;

generating a superimposed image by performing an image overlay of the first image that depicts the background and the item over the video stream captured by the image capture device of the client device pointed at the reflection of the user on the reflective surface, the superimposed image depicting the item depicted in the first image such that at least a portion of the video stream that captures the reflection of the user is viewable through the background of the first image with the item being superimposed over the captured reflection of the user, the generating being performed by a processor; and

causing the superimposed image to be displayed on the client device that is capturing the video stream.

2. The computer-implemented method of claim 1, wherein the first image in the superimposed image is selectable to generate a request for additional information about the item.

3. The computer-implemented method of claim 2, further comprising:

generating the request for additional information about the item in response to receiving an indication of a selection of the superimposed image;

transmitting the generated request to the commerce system; and

receiving a product listing page corresponding to the item.

4. The computer-implemented method of claim 1, further comprising generating the video stream by capturing the live camera feed of the reflection of the user on the reflective surface.

5. The computer-implemented method of claim 4, wherein the capturing the live camera feed is based on the user holding the client device with the image capture device pointed at the reflective surface.

6. A non-transitory machine-readable storage medium storing a set of instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

retrieving a first image from a commerce system that stores the first image in association with an account a user of a client device, the retrieved first image depicting a background and depicting an item;

identifying edges of the item using image processing edge detection;

within the first image, distinguishing the background from the item based on the identified edges of the item;

making the background of the first image transparent;

accessing a video stream capturing a live camera feed of a reflection of the user on a reflective surface, the live camera feed being captured by an image capture device of the client device pointed at the reflective surface;

generating a superimposed image by performing an image overlay of the first image that depicts the background and the item over the video stream captured by the image capture device of the client device pointed at the reflection of the user on a reflective surface, the superimposed image depicting the item depicted in the first image such that at least a portion of the video stream that captures the reflection of the user is viewable through the background of the first image with the item being superimposed over the captured reflection of the user; and

causing the superimposed image to be displayed on the client device that is capturing the video stream.

7. The non-transitory machine-readable storage medium of claim 6, wherein the first image in the superimposed image is selectable to generate a request for additional information about the item.

8. The non-transitory machine-readable storage medium of claim 7, wherein the operations further comprise generating the request for additional information about the item in response to receiving an indication of a selection of the superimposed image.

9. The non-transitory machine-readable storage medium of claim 8, wherein the operations further comprise:

transmitting the generated request to the commerce system; and

receiving a product listing page corresponding to the item.

10. The non-transitory machine-readable storage medium of claim 6, wherein the operations further comprise generating the video stream by capturing the live camera feed of the reflection of the user on the reflective surface.

11. The non-transitory machine-readable storage medium of claim 10, wherein the capturing the live camera feed is based on the user holding the client device with the image capture device pointed at the reflective surface.

12. A system, comprising:

one or more processors; and

a memory storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising;

retrieving a first image from a commerce system that stores the first image in association with an account of a user of the client device, the retrieved first image depicting a background and depicting an item;

identifying edges of the item using image processing edge detection;

within the first image, distinguishing the background from the item based on the identified edges of the item;

making the background of the first image transparent;

accessing a video stream capturing a live camera feed of a reflection of the user on a reflective surface, the live camera feed being captured by an image capture device of the client device pointed at the reflective surface;

generating a superimposed image by performing an image overlay of the first image that depicts the background and the item over the video stream captured by the image capture device of the client device pointed at the reflection of the user on a reflective surface, the superimposed image depicting the item depicted in the first image such that at least a portion of the video stream that captures the reflection of the user is viewable through the background of the first image with the item being superimposed over the captured reflection of the user; and

causing the superimposed image to be displayed on the client device that is capturing the video stream.

13. The system of claim 12, wherein the operations further comprise transmitting the superimposed image to the commerce system, wherein the superimposed image is associated with the user.

14. The system of claim 12, wherein the first image in the superimposed image is selectable to generate a request for additional information about the item.

15. The system of claim 13, wherein the operations further comprise:

transmitting a request to the commerce system for additional information about the item in response to an indication of a selection of the superimposed image; and

receiving a product listing page corresponding to the item in response to the transmitting of the generated request.