Providing financial data through real-time virtual animation

• US 9,378,664 B1
• Filed: 10/05/2009
• Issued: 06/28/2016
• Est. Priority Date: 10/05/2009
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Steckler, Ryan Michael
• Primary Examiner(s): Milef, Elda
• Assistant Examiner(s): Gregg, Mary
• Application Number:
  US12/573,464
• Time in Patent Office: 2,458 Days
• US Class Current: 1/1
• CPC Class Codes: G09F 1/00 Cardboard or like show-cards of foldable or flexible material
• CPC Class Codes: H04B 7/00 Radio transmission systems, i.e. using radiation field H04B10/00, H04B15/00 take precedence
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

One embodiment of the present invention provides a system that provides data associated with a financial instrument through real-time virtual animation anchored to the financial instrument in a video stream. During operation, the system receives at a computer system a video stream, wherein the video stream includes an image of a financial instrument, and wherein a financial instrument is a physical object that is associated with at least one of a financial account and financial data. Next, the system identifies the financial instrument in the video stream. The system then retrieves data associated with the financial instrument from a data source. Next, the system creates an animation which displays the data. Once the animation is created, the system anchors the animation to the financial instrument in the video stream. Finally, the system displays the video stream which includes the animation to a user.

Claims

1. A computer-implemented method for providing data associated with a financial instrument for a user through real-time virtual animation anchored to the financial instrument in a video stream, the method comprising:

receiving, at a computer system, from a camera for the computer system, a video stream comprising a video frame, wherein the video frame includes an image of the financial instrument, wherein the image includes information that identifies at least one of a financial account for the user and financial data for the user, wherein the financial instrument is a physical object that includes the information, and wherein the video stream is received from the camera as the user holds the financial instrument up to the camera;

identifying the financial instrument in the video stream by performing optical character recognition on the image to extract second data, the second data comprising at least some of the information that identifies the at least one of the financial account and the financial data;

using the second data to retrieve, from a data source, one or more monetary amounts for the at least one of the financial account and the financial data;

creating an animation which graphically represents the one or more monetary amounts;

modifying the video frame to include the animation such that, in the modified video frame, the animation is attached to the image of the financial document;

and displaying the modified video frame to the user at the computer system.

2. The computer-implemented method of claim 1, wherein the data source can include at least one of:

a personal information manager;

an accounting system;

a centralized database;

a network service; and

a web-based application.

3. The computer-implemented method of claim 1, wherein the data source can include the financial instrument.

4. The computer-implemented method of claim 1, further comprising:

identifying a motion of the financial instrument in the video stream;

determining an action associated with the motion; and

performing the action on one of the financial account and the financial data.

5. The computer-implemented method of claim 4, wherein the motion comprises shaking the financial instrument in the video stream, and wherein the action comprises switching to a next animation in a set of animations that comprises the animation.

6. The computer-implemented method of claim 1, further comprising:

at the computer system, recognizing whether the video stream comprises a face of the user;

wherein including the animation in the video stream comprises;

when the face of the user is recognized in the video stream, including the animation in the video stream; and

otherwise, when the face of the user is not recognized in the video stream, displaying the video stream without displaying financial information for the user in the video stream.

7. The computer-implemented method of claim 6, wherein the video stream is received from the user, and wherein the user is a human being.

8. The computer-implemented method of claim 6, wherein the financial information includes the animation.

9. The computer-implemented method of claim 1, further comprising:

determining a type for the financial instrument by comparing the image of the financial instrument to a database of documents;

wherein identifying the financial instrument in the video stream comprises, in response to determining the type of the financial instrument, determining an identifier for the financial instrument by performing the optical recognition; and

wherein retrieving one or more monetary amounts comprises using the identifier for the financial instrument.

10. The computer-implemented method of claim 1, wherein the user is a human being.

11. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for providing data associated with a financial instrument for a user, the method comprising:

at a computer system, while the user holds the financial instrument up to a camera coupled to the computer system, receiving a video stream from the camera, wherein the video stream comprises a set of video frames, each of the video frames including a corresponding image of the financial instrument captured by the camera as the user holds the financial instrument up to the camera, wherein the financial instrument is a physical object that includes information that identifies at least one of a financial account for the user and financial data for the user;

identifying the financial instrument in the video stream by performing optical character recognition on at least one of the images of the financial instrument in the video frames to extract second data, the second data comprising at least some of the information that identifies the at least one of the financial account and the financial data;

using the second data to retrieve, from a data source, one or more monetary amounts for the at least one of the financial account and the financial data;

creating a real-time virtual animation that graphically represents the one or more monetary amounts;

for each video frame in the set of video frames, modifying the video frame to include the animation such that, in the modified video frame, the animation is attached to the image of the financial document in the video frame; and

displaying the modified video frames to the user at the computer system.

12. The computer-readable storage medium of claim 11, wherein the data source can include at least one of:

a personal information manager;

an accounting system;

a centralized database;

a network service; and

a web-based application.

13. The computer-readable storage medium of claim 11, wherein the data source can include the financial instrument.

14. The computer-readable storage medium of claim 11, wherein the method further comprises:

identifying a second financial instrument in the video stream; and

retrieving data associated with a combination of the financial instrument and the second financial instrument from the data source.

15. The computer-readable storage medium of claim 11, wherein the method further comprises:

identifying a second financial instrument in the video stream; and

wherein displaying the video stream with the animation involves only displaying the animation when the second financial instrument is present in the video stream.

16. The computer-readable storage medium of claim 11, wherein the method further comprises:

identifying a motion of the financial instrument in the video stream;

determining an action associated with the motion; and

performing the action on one of the financial account and the financial data.

17. The computer-implemented method of claim 11, wherein displaying the second video stream comprises

displaying the real-time virtual animation anchored to the images of the financial instrument captured by the camera while the user holds the financial instrument up to the camera, and wherein the user is a human being.

18. An apparatus configured to provide data associated with a financial instrument for a user through real-time virtual animation anchored to the financial instrument in a video stream, comprising:

a processor;

a memory;

a camera;

a receiving mechanism configured to receive, at a computer system, from the camera, a video stream comprising a video frame, wherein the video frame includes an image of the financial instrument, wherein the image includes information that identifies at least one of a financial account for the user and financial data for the user, wherein the financial instrument is a physical object that includes the information, and wherein the video stream is received from the camera as the user holds the financial instrument up to the camera;

an identification mechanism configured to identify the financial instrument in the video stream by performing optical character recognition on the image to extract second data, the second data comprising at least some of the information that identifies the at least one of the financial account and the financial data;

a retrieval mechanism configured to use the second data to retrieve, from a data source, one or more monetary amounts for the at least one of the financial account and the financial data;

an animation mechanism configured to create an animation which graphically represents the one or more monetary amounts, wherein the animation mechanism is further configured to conditionally include the animation in the video stream;

a mechanism configured to modify the video frame to include the animation such that, in the modified video frame, the animation is attached to the image of the financial document; and

a display mechanism configured to display the modified video frame to the user at the computer system.