Augmented reality view of product instructions

• US 9,953,350 B2
• Filed: 01/07/2016
• Issued: 04/24/2018
• Est. Priority Date: 09/21/2012
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Pugazhendhi, Bagath, Yankovich, Steve
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US14/990,291
• Publication Number:
  US 20160117863A1
• Time in Patent Office: 838 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0281 Customer communication at a business location, e.g. providing product or service information, consulting
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/12 Fingerprints or palmprints
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/027 Arrangements and methods specific for the display of internet documents
• CPC Class Codes: G09G 2370/04 Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
• CPC Class Codes: G09G 2380/14 Electronic books and readers
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

In a system for presenting augmented reality views of product instructions a method may include receiving a request from a client device, the request including image data. The method may further includes identifying an object in the image data and generating an augmented reality view of the identified object. The method may further include transmitting the augmented reality view to the client device.

Claims

1. A method comprising:

receiving, from a client device, a request for instructions related to an object, the request including image data;

identifying the object in the image data;

generating an augmented reality view of the identified object, the generating of the augmented reality view of the identified object including accessing, from a database, instructions correlated with the object;

determining location data for the instructions correlated with the object, the location data including coordinates in the image; and

transmitting the augmented reality view to the client device.

2. The method of claim 1, wherein the augmented reality view includes the instructions and the location data.

3. The method of claim 1, wherein generating an augmented reality view of the identified object based on the request type comprises retrieving promotional material associated with the object.

4. The method of claim 1, wherein generating an augmented reality view of the identified object further comprises retrieving object recommendations associated with the identified object.

5. The method of claim 4, wherein the augmented reality view includes the object recommendation.

6. The method of claim 5, further comprising:

receiving an indication the object recommendation was selected on the client device; and

transmitting a location to purchase the object recommendation to the client device.

7. A computer-readable storage device including instructions stored thereon, which when executed by at least one processor, cause the processor to perform operations comprising:

receiving, from a client device, a request for instructions related to an object, the request including image data;

identifying the object in the image data;

generating an augmented reality view of the identified object, the generating of the augmented reality view of the identified object including accessing, from a database, instructions correlated with the object;

determining location data for the instructions correlated with the object, the location data including coordinates in the image; and

transmitting the augmented reality view to the client device.

8. The storage device of claim 7, wherein the augmented reality view includes the instructions and the location data.

9. The storage device of claim 7, wherein the instructions for generating an augmented reality view of the identified object comprise instructions for retrieving promotional material associated with the object based on the request type including an ad request.

10. The storage device of claim 7, wherein the instructions for generating an augmented reality view of the identified object comprise instructions for retrieving product recommendations associated with the identified object.

11. The storage device of claim 10, wherein the augmented reality view includes the product recommendation.

12. The storage device of claim 11, wherein the operations further comprise:

receiving an indication the product recommendation was selected on the client device; and

transmitting a location to purchase the product recommendation to the client device.

13. A system comprising:

a server comprising;

a communication device configured to receive, from a client device, a request for instructions related to an object, the request including image data;

one or more processors in communication with the communication device; and

one or more non-transitory computer-readable media storing instructions thereon that, when executed by the one or more processors, cause the server to perform operations, the operations including;

identify the object in the image data;

generate an augmented reality view of the object by performing operations comprising accessing, from a database, instructions correlated with the object;

determine location data for the instructions correlated with the object, the location data including coordinates in the image; and

cause the communication device to transmit the augmented reality view to the client device.

14. The system of claim 13, wherein the augmented reality view includes the instructions and the location data.

15. The system of claim 13, wherein the operations further comprise generate a product recommendation associated with the identified object.

16. The system of claim 15, wherein the augmented reality view includes an element to allow a user to select the product recommendation.

17. The system of claim 16, wherein the augmented reality view is configured to, based on receiving a selection of the product recommendation, direct a user to an online marketplace to purchase the product recommendation.