Device and method to assist user in conducting a transaction with a machine

• US 7,641,108 B2
• Filed: 04/01/2005
• Issued: 01/05/2010
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Albrecht, Paul, Kurzweil, Raymond C., Gashel, James
• Primary Examiner(s): Frech; Karl D.
• Application Number:
  US11/096,714
• Publication Number:
  US 20060011718A1
• Time in Patent Office: 1,740 Days
• Field of Search: 235/380, 235/379, 235/381, 235/451, 340/571.4, 705/43
• US Class Current: 235/380
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted

Abstract

A device for assisting a user to perform a transaction on a machine is described. The device receives data that specifies a transaction mode to use for processing an image and accesses a knowledge base to provide data to configure the device for the transaction mode, the data including data specific to the transaction mode. The device receives an image or images of a portion of a machine that the user will use to perform the transaction and processes the image or images to identify a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine. The device announces to the user the name or function of the control closest to an end of the user-controlled pointing item.

Claims

1. A computer program product residing on a computer readable medium for assisting a user of a device to use the device to perform a transaction on a machine, the computer program product comprises instructions for causing the device to:

receive data that specifies a transaction mode to use for processing an image;

access a knowledge base to provide data to configure the computer program product for the transaction mode, the data including data specific to the transaction mode;

receive an image or images of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine; and

announce to the user the name or function of the control closest to an end of the user-controlled pointing item.

2. The computer program product of claim 1 wherein the machine is an automatic teller machine (ATM), c-ticket machine, electronic voting machines, or credit/debit point of sale machine.

3. The computer program product of claim 1 wherein the machine has a layout of controls and the layout is included in data retrieved from the knowledge base.

4. The computer program product of claim 1 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls on the machine in the image of the machine by comparing the layout to a stored representation of the layout for the machine.

5. The computer program product of claim 1 wherein instructions to process the image, comprises instructions to:

process the image with character recognition to recognize text on controls in the image of the machine; and

convert text to speech to read the text on the controls in the image to the user.

6. The computer program product of claim 1 further comprising instructions to:

guide the user in operation of the machine.

7. The computer program product of claim 6 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

8. The computer program product of claim 1 further comprising instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

9. The computer program product of claim 1 further comprising instructions to:

announce to the user to move the user-controlled pointing item comprises instructions to;

determine where the location of the user-controlled pointing item in the image relative to a selected control in the image.

10. The computer program product of claim 1 wherein for transaction devices that have standardized or memorized layouts of controls, the computer program product further comprises instructions to:

retrieve a stored template to recognize the layout; and

process the image according to the stored template to navigate the user through use of the machine.

11. The computer program product of claim 1 further instructions to:

process a signal from an REID tag reader that receives a signal from an REID tag on the machine to provide a specific description of the layout of controls on the machine.

12. A method of operating a device, the method comprising:

receiving data by the device that specifies a transaction mode to use for operating the device;

accessing a knowledge base to provide data to configure the device for the transaction mode, the data including data specific to the transaction mode;

receiving an image or images of a portion of a machine that the user will use to perform the transaction;

processing the image or images to identity a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine; and

announcing to the user the name or function of the control closest to an end of the user-controlled pointing item.

13. The computer program product of claim 12 wherein the machine is an automatic teller machine (ATM), e-ticket machine, electronic voting machines, or credit/debit point of sale machine.

14. The method of claim 12 further comprising:

processing the image to recognize text on controls in the image; and

converting recognized text on the controls to speech to read aloud the text on the controls in the image to the user.

15. The method of claim 12 wherein the machine has a layout of controls and the layout is included in data retrieved from the knowledge base.

16. The method of claim 12 further comprising:

processing the image to recognize the controls in the image; and

identifying controls in the layout of the image of the machine by comparing the layout to a stored representation of the layout for the machine.

17. The method of claim 12 wherein announcing further comprises:

announcing to the user the function of a control over which the user-controlled pointing item is placed.

18. The method of claim 17 further comprising:

determining where the location of the user-controlled pointing item is in the image relative to a selected control in the image.

19. The method of claim 12 further comprising:

determining whether the user controlled pointing item is over a control; and

if not,

announcing to the user to move the user-controlled pointing item.

20. The method of claim 12 further comprising:

retrieving a stored template to recognize the layout; and

processing the image according to the stored template to navigate the user through use of the machine.

21. The method of claim 12 further comprising:

processing a signal from an RFID tag reader that receives a signal from an RFID tag on the machine to provide a specific description of the layout of controls on the machine.

22. An apparatus, comprising:

a computing device;

an image input device;

a computer readable medium storing a computer program product assisting a user of the apparatus to use the apparatus to perform a transaction with a machine, the computer program product comprises instructions for causing the computing device to;

receive data that specifies a transaction mode;

access a knowledge base to provide data to configure the apparatus for the transaction mode, the data including data specific to the transaction mode;

receive an image or images from the image input device the image or images being of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine; and

announce to the user the name or function of the control closest to an end of the user-controlled pointing item.

23. The apparatus of claim 22 wherein the computer program product is configured with data for a machine that is an automatic teller machine (ATM), c-ticket machine, electronic voting machines, or credit/debit point of sale machine.

24. The apparatus of claim 22 wherein the transaction-oriented device has a layout of controls and the layout is included in data retrieved from the knowledge base.

25. The apparatus of claim 22 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls in the image; and

identify the buttons in the layout of the image of the transaction-oriented device by comparing the layout to a stored representation of the layout for the transaction-orientated device.

26. The apparatus of claim 22 wherein instructions to process the image, comprises instructions to:

process the image to recognize text on controls in the image; and

convert recognized text on the controls to speech to read aloud the text on the controls in the image to the user.

27. The apparatus of claim 22 wherein the computer program product further comprises instructions to:

guide the user in operation of the machine.

28. The apparatus of claim 22 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

29. The apparatus of claim 22 wherein the computer program product further comprises instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

30. The apparatus of claim 22 wherein the computer program product further comprises instructions to:

determine where the location of the user-controlled pointing item in the image relative to a selected control in the image.

31. The apparatus of claim 22 wherein the computer program product comprises instructions to:

retrieve a stored template to recognize a layout for machines that have standardized or memorized layouts of controls; and

process the image according to the stored template to navigate the user through use of the machine.