CONCURRENT VISUALIZATION TOOL USING AUGMENTED REALITY TO IMPROVE DECISION MAKING

• US 20190318408A1
• Filed: 04/11/2018
• Published: 10/17/2019
• Est. Priority Date: 04/11/2018
• Status: Active Grant

• Current Assignee: Wells Fargo Bank NA (Wells Fargo & Company)
• Original Assignee: Wells Fargo Bank NA (Wells Fargo & Company)
• Inventors: Chao, Waverly W., Devarajan, Haritha, Gross, Thomas W., Kushner, Kristine Ing, Munir, Muhammad Farukh, Murphy, Pamela M.
• Granted Patent: US 11,803,890 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 4/35 for the management of goods or merchandise
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Examples described herein relate to apparatuses and methods of providing real-time notifications on affordability and advisability of purchasing goods or services using augmented reality. The method includes receiving, from AR device associated with a user, visual data representing a view through a viewing area of the AR device. The visual data comprises geolocation data indicating a geolocation of the AR device. The method includes identifying an image of an object included in the visual data. The method includes determining, based on the visual data, a product identifier associated with the object. The method includes determining an attempt by the user to purchase the object. The method includes generating, in response to determining the attempt, recommendation data based on the product identifier and an account associated with the user. The method includes sending the recommendation data to the AR device to cause the AR device to display the recommendation data to the user.

Claims

1. A method of providing real-time notifications on affordability and advisability of purchasing goods or services using augmented reality (AR), the method comprising:

receiving, by a recommendation server and from an AR device associated with a user, visual data representing a view through a viewing area of the AR device, the visual data comprises geolocation data indicating a geolocation of the AR device;

identifying, by the recommendation server, an image of an object included in the visual data;

determining, by the recommendation server and based on the visual data, a product identifier associated with the object;

determining, by the recommendation server, an attempt by the user to purchase the object;

generating, by the recommendation server and in response to determining the attempt, recommendation data based on the product identifier and an account associated with the user; and

sending, by the recommendation server, the recommendation data to the AR device to cause the AR device to display the recommendation data to the user.

2. The method of claim 1, wherein identifying the object comprises:

determining the object occupies a predetermined amount of a predetermined region of the viewing area of the AR device.

3. The method of claim 1, wherein identifying the object comprises:

determining the object occupies a foreground of the viewing area of the AR device.

4. The method of claim 1, wherein determining the product identifier comprises:

extracting, by the recommendation server, the image of the object from the visual data;

comparing, by the recommendation server, the extracted image and a plurality of images to identify an image of the plurality of images matching the extracted image, wherein each image of the plurality of images comprises a known object; and

retrieving, by the recommendation server and from a database, a product identifier associated with the image of the plurality of images.

5. The method of claim 1, wherein determining the attempt by the user to purchase the object comprises:

extracting, by the recommendation server, the geolocation data from the visual data;

comparing, by the recommendation server, the extracted geolocation data and historical geolocation data associated with the account of the user to determine a number of matches; and

determining, by the recommendation server, the number of matches exceeds a predetermined threshold.

6. The method of claim 1, wherein determining the attempt by the user to purchase the object comprises:

comparing, by the recommendation server, the product identifier associated with the object and contextual data associated with the account of the user;

wherein the contextual data extracted from at least one of a social media account, a text message, and a transcript of a voice call.

7. The method of claim 1, wherein generating the recommendation data comprises:

discovering, by the recommendation server and based on the account of the user, a predetermined financial milestone associated with the user; and

comparing, by the recommendation server, the predetermined financial milestone, purchase data associated with a purchase of the object, and the account of the user to determine an affordability value, wherein the affordability value indicates whether the purchase of the object prevents the user from achieving the predetermined financial milestone; and

generating, by the recommendation server, the recommendation data based on the affordability value.

8. The method of claim 7, wherein generating the recommendation data comprises:

selecting, by the recommendation server based on the affordability value, a cross-product associated with the product identifier; and

generating, by the recommendation server, the recommendation data based on the plurality of cross-products, the product identifier, and the account associated with the user;

wherein the recommendation data causes the AR device to display the cross-product to the user.

9. The method of claim 1, wherein generating the recommendation data comprises:

discovering, by the recommendation server and based on the account of the user, a predetermined financial milestone associated with the user;

retrieving, by the recommendation server and from a database, visual content associated with the user;

generating, by the recommendation server and based on the visual content, a digital video simulating the user interacting with an object associated with the predetermined financial milestone; and

generating, by the recommendation server, the recommendation data based on the digital video, the product identifier, and the account associated with the user;

wherein the recommendation data causes the AR device to display the digital video to the user.

10. The method of claim 1, wherein the recommendation data causes the AR device to display the recommendation data in a first region of the viewing area of the AR device separate from a second region of the viewing area of the AR device occupied by the object.

11. The method of claim 1, wherein the recommendation data causes the AR device to display an indicator deterring the user from purchasing the object.

12. The method of claim 1, further comprising:

receiving, by the recommendation server and from the AR device, a request to share the recommendation data with a second AR device associated with a second user; and

sending, by the recommendation server, the recommendation data to the second AR device to cause the second AR device to display the recommendation data to the second user.

13. A system for providing real-time notifications on affordability and advisability of purchasing goods or services using augmented reality (AR), the system comprising a recommendation server configured to:

receive from an AR device associated with a user, visual data representing a view through a viewing area of the AR device, the visual data comprises geolocation data indicating a geolocation of the AR device;

identify an image of an object included in the visual data;

determine, based on the visual data, a product identifier associated with the object;

determine an attempt by the user to purchase the object;

generate, in response to determining the attempt, recommendation data based on the product identifier and an account associated with the user; and

send the recommendation data to the AR device to cause the AR device to display the recommendation data to the user.

14. The system of claim 13, wherein the recommendation server further configured to at least one of:

determine the object occupies a predetermined amount of a predetermined region of the viewing area of the AR device; and

determine the object occupies a foreground of the viewing area of the AR device.

15. The system of claim 13, wherein the recommendation server further configured to:

extract the image of the object from the visual data;

compare the extracted image and a plurality of images to identify an image of the plurality of images matching the extracted image, wherein each image of the plurality of images comprises a known object; and

retrieve, from a database, a product identifier associated with the image of the plurality of images.

16. The system of claim 13, wherein the recommendation server further configured to:

extract the geolocation data from the visual data;

compare the extracted geolocation data and historical geolocation data associated with the account of the user to determine a number of matches; and

determine the number of matches exceeds a predetermined threshold.

17. The system of claim 13, wherein the recommendation server further configured to:

compare, by the recommendation server, the product identifier associated with the object and contextual data associated with the account of the user;

wherein the contextual data extracted from at least one of a social media account, a text message, and a transcript of a voice call.

18. The system of claim 13, wherein the recommendation server further configured to:

discover, based on the account of the user, a predetermined financial milestone associated with the user;

compare the financial milestone, purchase data associated with a purchase of the object, and the account of the user to determine an affordability value, wherein the affordability value indicates whether the purchase of the object prevents the user from achieving the financial milestone;

select, based on the affordability value, a cross-product associated with the product identifier; and

generate the recommendation data based on the plurality of cross-products, the product identifier, and the account associated with the user, wherein the recommendation data causes the AR device to display the cross-product and wherein the recommendation data causes the AR device to display an indicator deterring the user from purchasing the object.

19. The system of claim 13, wherein the recommendation server further configured to:

discover, based on the account of the user, a financial milestone predetermined by the user;

retrieve, from a database, visual content associated with the user;

generate, based on the visual content, a digital video simulating the user interacting with an object associated with the financial milestone; and

generate the recommendation data based on the digital video, the product identifier, and the account associated with the user;

wherein the recommendation data causes the AR device to display the digital video to the user.

20. A method for providing real-time notifications on affordability and advisability of purchasing goods or services using augmented reality, the method comprising:

detecting, by AR device associated with a user, an observation of an object by a user through a viewing area of the AR device;

capturing, by the AR device and in response to the detection of the observation, visual data representing a view through the viewing area of the AR device;

sending, by the AR device, the visual data to a recommendation server, the visual data comprises geolocation data indicating the geolocation of the AR device;

receiving, by the AR device, recommendation data from the recommendation server, the recommendation data indicating whether a purchase of the object prevents the user from achieving a predetermined financial milestone associated with the user;

displaying, by the AR device and in the viewing area of the AR device, the recommendation data to the user;

detecting, by the AR device and in the viewing area of the AR device, an overlap of the recommendation data and the object; and

moving, by the AR device, the recommendation data to a region of the viewing area of the AR device to avoid the overlap.