Dynamic configuration of an augmented reality overlay

• US 10,311,646 B1
• Filed: 02/26/2018
• Issued: 06/04/2019
• Est. Priority Date: 02/26/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Wurmfeld, David Kelly, Osborn, Kevin
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US15/905,542
• Time in Patent Office: 463 Days
• Field of Search: 345419
• CPC Class Codes: G06Q 20/1085 involving automatic teller machines [ATMs]
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G07F 19/20 Automatic teller machines [ATMs]
• CPC Class Codes: G07F 19/209 Monitoring, auditing or diagnose of functioning of ATMs
• CPC Class Codes: G07F 19/211 Software architecture within ATMs or in relation to the ATM network

Abstract

A device may detect, in a field of view of a camera, one or more components of an automated teller machine (ATM) device using a computer vision technique based on generating a three dimensional model of the one or more components. The device may identify the ATM device as a particular device or as a particular type of device based on the one or more components of the ATM device, or first information related to the ATM device. The device may identify a set of tasks to be performed with respect to the ATM device. The device may provide, for display via a display associated with the device, second information associated with the set of tasks as an augmented reality overlay. The device may perform an action related to the set of tasks, the ATM device, or the augmented reality overlay.

Claims

1. A device, comprising:

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, configured to;

detect, in a field of view of a camera, one or more components of an automated teller machine (ATM) device using a scale-invariant feature transform (SIFT) technique and a monocular simultaneous localization and mapping (monoSLAM) technique,

the one or more components including at least one of;

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device;

the one or more components being detected based on;

comparing a 3D model of the one or more components;

identify the ATM device as a particular ATM device or as a particular type of ATM device based on at least one of;

comparing the 3D model of the one or more components, or first information related to the ATM device;

identify a set of tasks to be performed with respect to the ATM device after identifying the ATM device as the particular ATM device or the particular type of ATM device,

the set of tasks being associated with servicing the ATM device;

provide, for display via a display associated with the device, second information associated with the set of tasks as an augmented reality overlay after identifying the set of tasks;

perform an action related to the set of tasks, the ATM device, or the augmented reality overlay after providing the augmented reality overlay for display; and

monitor, via the camera, a performance of the set of tasks after providing the second information for display as the augmented reality overlay,

the performance of the set of tasks being monitored by at least one of;

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

2. The device of claim 1, wherein the one or more processors, when identifying the ATM device as the particular ATM device or as the particular type of ATM device, are configured to:

process the 3D model to identify the one or more components after detecting the one or more components; and

identify the ATM device as the particular type of ATM device based on the one or more components including an expected set of components,

wherein the expected set of components is identified in another 3D model.

3. The device of claim 2, wherein the one or more processors, when identifying the ATM device as the particular ATM device or as the particular type of ATM device, are configured to:

identify the ATM device as the particular ATM device based on the 3D model identifying at least one of;

another set of components in addition to the expected set of components, or

a modification to the expected set of components.

4. The device of claim 1, wherein the one or more processors, when identifying the set of tasks to be performed, are configured to:

identify the set of tasks based on at least one of;

location information that identifies a location of the ATM device,

schedule information that identifies a schedule of tasks to be performed with respect to the ATM device, or

historical information that identifies a set of historical tasks that has been performed with respect to the ATM device.

5. The device of claim 1, wherein the one or more processors are further configured to:

update the augmented reality overlay based on determining that a task, of the set of tasks, has been performed with respect to the ATM device.

19. The device of claim 1, wherein the one or more processors, are further configured to:

detect the one or more components by comparing the 3D model of the one or more components to 3D models stored by the device.

20. The device of claim 1, wherein the one or more processors are further configured to:

provide information related to the performance of the set of tasks,

the information being provided for performing analytics on the performance of the set of tasks; and

generate one or more reports based on performing the analytics on the performance of the set of tasks,

the or more reports identifying one or more trends or patterns.

6. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors, cause the one or more processors to;

detect, in a field of view of a camera associated with a device, one or more components of an automated teller machine (ATM) device,

wherein the one or more components are detected using a combination of a scale-invariant feature transform (SIFT) technique and a monocular simultaneous localization and mapping (monoSLAM) technique, and

wherein the one or more components include at least one of;

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device,

wherein the one or more components are detected based on;

comparing a 3D model of the one or more components;

identify the ATM device as a particular device or as a particular type of device after detecting the one or more components,

wherein a particular configuration of the one or more components of the ATM device indicates that the ATM device is the particular device or the particular type of device;

identify a set of tasks to be performed with respect to the ATM device after identifying the ATM device as the particular device or the particular type of device,

the set of tasks being associated with servicing the ATM device;

provide, for display via a display associated with the device, information associated with the set of tasks as an augmented reality overlay after identifying the set of tasks;

perform an action related to the set of tasks, the ATM device, or the augmented reality overlay after providing the augmented reality overlay for display; and

monitor, via the camera, a performance of the set of tasks after providing the information for display as the augmented reality overlay,

the performance of the set of tasks being monitored by at least one of;

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

7. The non-transitory computer-readable medium of claim 6, wherein the one or more instructions, that cause the one or more processors to identify the ATM device as the particular device or as the particular type of device, cause the one or more processors to:

identify the ATM device as the particular type of device after detecting the one or more components in the field of view of the camera; and

identify the ATM device as the particular device based on location information that identifies a location of the particular device after identifying the ATM device as the particular type of device.

8. The non-transitory computer-readable medium of claim 6, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

update the augmented reality overlay from being associated with a first subset of tasks to being associated with a second subset of tasks.

9. The non-transitory computer-readable medium of claim 6, wherein the one or more instructions, that cause the one or more processors to detect the one or more components of the ATM device, cause the one or more processors to:

detect the one or more components by detecting an identifier associated with the one or more components; and

wherein the one or more instructions, that cause the one or more processors to identify the ATM device as the particular device or as the particular type of device, cause the one or more processors to;

identify the ATM device as the particular device or as the particular type of device after detecting the one or more components by detecting the identifier associated with the one or more components.

10. The non-transitory computer-readable medium of claim 6, wherein the one or more instructions, that cause the one or more processors to perform the action, cause the one or more processors to:

capture, by the camera, a set of images of the set of tasks as the set of tasks is being performed after providing the information for display as the augmented reality overlay; and

process the set of images to determine that a task, of the set of tasks, has been completed after capturing the set of images.

11. The non-transitory computer-readable medium of claim 10, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

update, for the task, the information provided for display as the augmented reality overlay after determining that the task has been completed.

12. The non-transitory computer-readable medium of claim 6, wherein the one or more instructions, that cause the one or more processors to perform the action, cause the one or more processors to:

provide, for display to a user device associated with a technical support office of a third party, a report that indicates a progress of a performance of the set of tasks.

13. A method, comprising:

generating, by a device, a three dimensional (3D) model of one or more components of an automated teller machine (ATM) device,

the one or more components including at least one of;

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device;

detecting, by the device and in a field of view of a camera associated with the device, the one or more components of the ATM device using a scale-invariant feature transform (SIFT) technique, or a monocular simultaneous localization and mapping (monoSLAM) technique,

the one or more components of the ATM device being detected based on comparing a 3D model of the one or more components;

identifying, by the device, the ATM device as a particular device or as a particular type of device after detecting the one or more components based on the comparing the 3D model of the one or more components;

processing first information after identifying the ATM device as the particular device or as the particular type of device,

wherein the first information relates to at least the ATM device;

identifying, by the device, a set of tasks to be performed with respect to the ATM device based on the first information,

the set of tasks being associated with servicing the ATM device;

providing, by the device and for display via a display associated with the device, second information associated with the set of tasks as an augmented reality overlay after identifying the set of tasks;

monitoring, by the device, a performance of the set of tasks by capturing one or more images at different points in time after providing the second information for display as the augmented reality overlay for display;

updating, by the device, the augmented reality overlay after monitoring the performance of the set of tasks;

performing, by the device, an action related to the set of tasks, the ATM device, or the augmented reality overlay after providing the augmented reality for display; and

monitor, by the device and via the camera, a performance of the set of tasks after providing the second information for display as the augmented reality overlay,

the performance of the set of tasks being monitored by at least one of;

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

14. The method of claim 13, wherein generating the 3D model comprises:

generating the 3D model of the one or more components using a combination of computer vision techniques,

the combination of computer vision techniques including;

a scale-invariant feature transform (SIFT) technique, or

a monocular simultaneous localization and mapping (monoSLAM) technique; and

wherein detecting the one or more components comprises;

detecting the one or more components using the generated 3D model after generating the 3D model.

15. The method of claim 14, further comprising:

processing the generated 3D model using a feature extraction technique after generating the generated 3D model using the combination of computer vision techniques.

16. The method of claim 13, further comprising:

determining that a particular combination of components is included in the one or more components after detecting the one or more components; and

wherein identifying the ATM device as the particular device or as the particular type of device comprises;

identifying the ATM device as the particular type of device after determining that the particular combination of components is included in the one or more components.

17. The method of claim 16, further comprising:

determining that the particular combination of components has been modified from an expected set of components after determining that the particular combination of components is included in the one or more components; and

wherein identifying the ATM device as the particular device or the particular type of device comprises;

identifying the ATM device as the particular device after determining that the particular combination of components has been modified from the expected set of components.

18. The method of claim 13, wherein monitoring the performance of the set of tasks comprises:

processing the one or more images using an image processing technique after capturing the one or more images;

identifying, in the one or more images, one or more indicators that indicate that a task, of the set of tasks, has been completed,

wherein the one or more indicators are related to the set of tasks or the one or more components; and

wherein the method further comprises;

determining that the task has been performed after identifying the one or more indicators.