System and method for guiding card positioning using phone sensors

• US 10,498,401 B1
• Filed: 07/15/2019
• Issued: 12/03/2019
• Est. Priority Date: 07/15/2019
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Rule, Jeffrey, Ilincic, Rajko, Hart, Colin, Herrington, Daniel, Osborn, Kevin
• Primary Examiner(s): Senfi, Behrooz M
• Application Number:
  US16/511,683
• Time in Patent Office: 141 Days
• Field of Search: 348 46
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 20/32 using wireless devices
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/3278 RFID or NFC payments by means of M-devices
• CPC Class Codes: G06Q 20/341 Active cards, i.e. cards including their own processing means, e.g. including an IC or chip
• CPC Class Codes: G06Q 20/352 Contactless payments by cards
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04B 5/77 : for interrogation
• CPC Class Codes: H04M 1/0264 : for a camera module assembly
• CPC Class Codes: H04M 1/72454 : according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/04 : including near field communication means, e.g. RFID
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A position alignment system facilitates positioning of a contactless card in a ‘sweet spot’ in a target volume relative to a contactless card reading device. Alignment logic uses information captured from available imaging devices such as infrared proximity detectors, cameras, infrared sensors, dot projectors, and the like to guide the card to a target location. The captured image information is processed to identify a card position, trajectory and predicted location using one or both of a machine learning model and/or a Simultaneous Localization and Mapping logic. Trajectory adjustment and prompt identification may be intelligently controlled and customized using machine-learning techniques to customize guidance based on the preference and/or historical behavior of the user. As a result, the speed and accuracy of contactless card alignment is improved and received NFC signal strength is maximized, thereby reducing the occurrence of dropped transactions.

Claims

1. A method for guiding positioning of a card to a target position relative to a device includes the steps of:

detecting by a proximity sensor that the card is proximate to the device;

responsive to the card being proximate to the device, the device capturing a series of images of a three-dimensional volume proximate to the device;

processing the series of images to determine a position and a trajectory of the card within the three-dimensional volume proximate to the device;

predicting a projected position of the card relative to the device based on the position of the card and the trajectory of the card;

identifying one or more variances between the projected position and the target position including identifying at least one trajectory adjustment predicted to reduce the one or more variances and one or more prompts predicted to achieve the trajectory adjustments;

displaying the one or more prompts on a display of the device;

repeating the steps of capturing the series of images, determining the position and trajectory of the card, predicting the projected position of the card, identifying the one or more variances, the at least one trajectory adjustment and the one or more prompts and displaying the one or more prompts until the one or more variances are within a predetermined threshold; and

triggering an event at the device to retrieve data from the card in response to the one or more variances being within the predetermined threshold.

2. The method of claim 1 wherein the step of processing the series of images to determine the position and a trajectory of the card within the three-dimensional volume proximate to the device uses at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process.

3. The method of claim 2 including the steps of, during the event, repeating the steps of capturing the series of images, determining the position and the trajectory of the card, predicting the projected position of the card, identifying the one or more variances, the at least one trajectory adjustment and the one or more prompts and displaying the one or more prompts to ensure that the variances remain within a predetermined threshold to enable the device to read data from the card.

4. The method of claim 3 wherein the step of triggering the event comprises initiating a data exchange between the card and the device, wherein the data exchange is related to at least one of a financial transaction and an authorization transaction.

5. The method of claim 1 wherein the step of capturing the series of images is performed by one or more of a camera of the device, an infrared sensor of the device or a dot projector of the device, and wherein the series of images comprises one or both of two-dimensional image information and three-dimensional image information related to one or more of an infrared energy and a visible light energy measured at the device.

6. The method of claim 5 including the step of generating a volume map of the three-dimensional volume proximate to the device using the series of images obtained from one or more of the camera, the infrared sensor and the dot projector, the volume map comprising a pixel data for a plurality of pixel locations within the three-dimensional volume proximate to the device.

7. The method of claim 6 wherein the step of processing the series of images to determine the position and the trajectory of the card includes the step of forwarding the series of images to a feature extraction machine learning model trained to process the volume map to detect one or more features of the card and to identify the position and the trajectory of the card within the volume map in response to the one or more features.

8. The method of claim 7 wherein the step of predicting the projected position of the card relative to the device includes forwarding the position and the trajectory of the card to a second machine learning model trained to predict the projected position based on a historic attempt to position the card.

9. The method of claim 8 wherein the historic attempt used to train the second machine learning model is customized to a user of the device.

10. The method of claim 8 wherein the one or more prompts include at least one of a visible prompt, an audible prompt, or a combination of visible and audible prompts.

11. A device comprising:

a proximity sensor configured to detect whether a card is proximate to the device;

an image capture device coupled to the proximity sensor and configured to capture a series of images of a three-dimensional volume proximate to the device;

a processor coupled to the proximity sensor and the image capture device;

a display interface coupled to the processor;

a card reader interface coupled to the processor; and

a non-transitory medium storing alignment program code configured to guide a card to a target position relative to the device, the alignment program code operable when executed upon by the processor to;

monitor a proximity of the card to the device;

enable the image capture device to capture the series of images of the three-dimensional volume proximate to the device;

process the series of images to determine a position and a trajectory of the card within the three-dimensional volume proximate to the device and to predict a projected position of the card relative to the device based on the position of the card and the trajectory of the card;

identify one or more variances between the projected position and the target position including identifying at least one trajectory adjustment and one or more prompts to achieve the at least one trajectory adjustment, the at least one trajectory adjustment predicted to reduce the one or more variances;

display the one or more prompts on the display interface during at least one of prior to and during the card read operation; and

trigger a card read operation by the card reader interface when the one or more variances are within a predetermined threshold.

12. The device off claim 11 wherein the program code that is operable when executed upon to process the series of images to determine the position and a trajectory of the card within the three-dimensional volume proximate to the device uses at least one of a machine learning model or a simultaneous localization and mapping (SLAM) process.

13. The device of claim 11 wherein the card read operation is associated with one of a financial transaction and an authorization transaction.

14. The device of claim 11 wherein the image capture device comprises one or more of a camera, an infrared sensor or a dot projector, and the series of images capture one or more of an infrared energy and a visible light energy measured at the device.

15. The device of claim 14 wherein the series of images comprise one or both of two-dimensional image information and three-dimensional image information.

16. The device of claim 15 wherein the alignment program code is further configured to generate a volume map of the three-dimensional volume proximate to the device using the series of images, the infrared sensor and the dot projector, the volume map comprising pixel data for a plurality of pixel locations within the three-dimensional volume proximate to the device.

17. The device of claim 16 further including a feature extraction machine learning model is trained to locate the card within the three-dimensional volume proximate to the device and to predict a projected position using a historic attempt to position the card.

18. The device of claim 17 wherein the historic attempt is a user specific historic attempts.

19. The device of claim 18 wherein the one or more prompts include at least one of a visible prompt, an audible prompt, or a combination of visible and audible prompts.

20. A method for guiding a card to a target position relative to a device includes the steps of:

detecting a request by the device to perform a transaction;

measuring, using a proximity sensor of the device, a proximity of the card to the device;

controlling at least one of a camera and an infrared depth sensor of the device to capture a series of images of a three-dimensional volume proximate to the device when the card is determined to be proximate to the device;

processing the series of images to determine a position and trajectory of the card in the three-dimensional volume proximate to the device, the processing performed by at least one of a machine learning model trained using historic attempts to guide the card to the target position or a simultaneous localization and mapping (SLAM) process;

predicting a projected position of the card relative to the device based on the position and the trajectory of the card;

identifying one or more variances between the projected position and the target position including identifying at least one trajectory adjustment selected to reduce the one or more variances and identifying one or more prompts to achieve the trajectory adjustments;

displaying the one or more prompts on a display of the device;

repeating the steps of capturing image information, determining the position and trajectory of the card, predicting the projected position of the card, identifying the one or more variances, the at least one trajectory adjustment and the one or more prompts and displaying the one or more prompts until the one or more variances are within a predetermined threshold; and

triggering a read of the card by a card reader of the device when the variances are less than the predetermined threshold.