Using augmented reality for electronic commerce transactions

• US 10,529,105 B2
• Filed: 02/05/2018
• Issued: 01/07/2020
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Zamer, Kamal
• Primary Examiner(s): Wang, Jin Cheng
• Application Number:
  US15/889,151
• Publication Number:
  US 20180240259A1
• Time in Patent Office: 701 Days
• Field of Search: 345633
• CPC Class Codes: G06Q 20/12 specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/321 using wearable devices
• CPC Class Codes: G06Q 20/322 Aspects of commerce using mobile devices [M-devices]
• CPC Class Codes: G06Q 20/40 Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
• CPC Class Codes: G06Q 30/0609 Buyer or seller confidence or verification
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0635 Processing of requisition or of purchase orders
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/53 using third party service providers

Abstract

Systems, methods, and computer program products for identifying objects of interest and providing relevant information about the objects of interest using augmented reality devices are disclosed. For example, a computer-implemented method may include identifying an object of interest among a plurality of objects present in an image view, determining real-time information for the object of interest based on the identifying, presenting the determined real-time information for the object of interest as part of the image view, and processing a transaction involving the object of interest based on a user selection associated with the image view.

Claims

1. A system comprising:

a hardware device for capturing an image view;

a non-transitory memory storing the image view; and

one or more hardware processors coupled to the non-transitory memory and configured to access the image view from the non-transitory memory and to execute instructions to cause the system to perform operations comprising;

determining, among a plurality of vehicles, a particular vehicle based on a location of the hardware device and a destination location associated with the user;

providing, on the image view, an indication of a loading zone location for the particular vehicle on route to the destination location;

determining that the user has moved to the loading zone location;

subsequent to determining that the user has moved to the loading zone location, identifying the particular vehicle among a plurality of objects present in the image view;

establishing a connection with the particular vehicle;

obtaining real-time status information related to the particular vehicle from the particular vehicle via the connection;

in response to obtaining the real-time status information, presenting, via a user interface associated with the hardware device, (i) the real-time status information related to the particular vehicle as part of the image view, and (ii) an element selectable to perform an electronic payment transaction with the particular vehicle;

receiving, via the user interface, a selection of the element;

in response to receiving the selection of the element, performing the electronic payment transaction with the particular vehicle via the established connection;

providing, via the user interface, a first indication of a first stop along a route of the particular vehicle for the destination location;

detecting, based on data received via the established connection, a change to the route of the particular vehicle; and

in response to detecting the change, displaying on the user interface a second indication of a second stop along the changed route of the particular vehicle for the destination location.

2. The system of claim 1, wherein the real-time status information comprises route information of the particular vehicle.

3. The system of claim 1, wherein the performing the electronic payment transaction with the particular vehicle comprises transmitting payment information to the particular vehicle via the connection.

4. The system of claim 1, wherein the electronic payment transaction is further for purchasing an item associated with the particular vehicle.

5. The system of claim 1, wherein the operations further comprise presenting an interactive user interface as part of the image view based on the obtained real-time status information.

6. The system of claim 1, wherein the operations further comprise:

obtaining additional information related to the particular vehicle from a computing device via a second connection; and

presenting the additional information as part of the image view.

7. The system of claim 1, wherein the operations further comprise:

retrieving calendar data from a calendar application installed on the hardware device based on a current time; and

determining the destination location associated with the user based in part on the retrieved calendar data.

8. A method comprising:

determining, by one or more hardware processors among a plurality of vehicles, a particular vehicle based on a location of a device of a user and a destination location associated with the user;

providing, by the one or more hardware processors on the device of the user, an indication of a loading zone location for the particular vehicle on route to the destination location;

determining, by the one or more hardware processors, that the user has moved to the loading zone location;

obtaining, by the one or more hardware processors, an image of a scene captured by the device;

identifying, by the one or more hardware processors, the particular vehicle among a plurality of objects present in the scene based on the image;

establishing, by the one or more hardware processors, a wireless connection with the particular vehicle;

obtaining, by the one or more hardware processors, real-time status data related to the particular vehicle directly from the particular vehicle via the wireless connection;

superimposing, by the one or more hardware processors, the obtained real-time status data on the image;

performing, by the one or more hardware processors, the electronic payment transaction with the particular vehicle via the established wireless connection;

providing, via the user interface of the device, a first indication of a first stop along a route of the particular vehicle for the destination location;

detecting, based on data received via the established connection, a change to the route of the particular vehicle; and

in response to detecting the change, displaying on the user interface of the device a second indication of a second stop along the changed route to the particular vehicle for the destination location.

9. The method of claim 8, wherein the wireless connection is one of a WiFi connection, a Bluetooth connection, a Near Field Communication connection, an infrared communication connection, or a radio-frequency identification connection.

10. The method of claim 8, wherein the real-time status data comprises route information of the particular vehicle.

11. The method of claim 8, wherein the performing the electronic payment transaction with the particular vehicle via the wireless connection comprises transmitting payment information to the particular vehicle via the wireless connection.

12. The method of claim 11, wherein the electronic payment transaction is further for purchasing an item associated with the particular vehicle.

13. The method of claim 8, further comprising presenting an interactive user interface based on the obtained real-time status data, wherein the interactive user interface comprises a link to an external website having additional information related to the particular vehicle.

14. The method of claim 8, further comprising:

retrieving calendar data from a calendar application of the device; and

determining the destination location associated with the user based on the calendar data.

15. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

determining, among a plurality of vehicles, a particular vehicle based on a location of a mobile device of a user and a destination location associated with the user;

providing, on the mobile device of the user, an indication of a loading zone location for the particular vehicle on route to the destination location;

determining that the user has moved to the loading zone location;

capturing an image via the mobile device;

identifying the particular vehicle among a plurality of objects present in the image;

establishing a connection between the mobile device and the particular vehicle;

determining information related to the vehicle via the connection;

presenting, via a user interface of the mobile device, the information as augmented-reality data on the image;

providing, via the user interface, a first indication of a first stop along a route of the particular vehicle for the destination location;

detecting, based on data received via the established connection, a change to the route of the particular vehicle; and

in response to detecting the change, displaying on the user interface of the mobile device a second indication of a second stop along the changed route of the particular vehicle for the destination location.

16. The non-transitory machine-readable medium of claim 15, wherein the information comprises a current availability of the particular vehicle.

17. The non-transitory machine-readable medium of claim 16, wherein the information further comprises fuel data associated with the particular vehicle.

18. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:

identifying an object associated with the particular vehicle, wherein the object is absent in the image;

obtaining additional information related to the object via the connection; and

presenting the additional information as augmented-reality data on the image.

19. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise performing an electronic payment transaction with the particular vehicle via the connection.

20. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:

retrieving calendar data from a calendar application of the mobile device; and

determining the destination location associated with the user based on the calendar data.