Device and method to assist user in conducting a transaction with a machine

• US 8,186,581 B2
• Filed: 01/05/2010
• Issued: 05/29/2012
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Kurzweil, Raymond C., Albrecht, Paul, Gashel, James, Gibson, Lucy
• Primary Examiner(s): Frech, Karl D.
• Application Number:
  US12/652,104
• Publication Number:
  US 20100266205A1
• Time in Patent Office: 875 Days
• Field of Search: 235/380, 235/379, 235/381, 235/451, 340/571.4, 705/43
• US Class Current: 235/380
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted

Abstract

A device for assisting a user to perform a transaction on a machine is described. The device receives data that specifies a transaction mode to use for processing an image and accesses a knowledge base to provide data to configure the device for the transaction mode, the data including data specific to the transaction mode. The device receives an image or images of a portion of a machine that the user will use to perform the transaction and processes the image or images to identify a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine. The device announces to the user the name or function of the control closest to an end of the user-controlled pointing item.

Claims

1. A computer program product tangibly embodied on a computer readable storage device, the computer program product for assisting a user of an electronic device to use the electronic device to perform a transaction on a machine, the computer program product comprises instructions for causing the electronic device to:

receive an image or images of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine; and

announce to the user the name or function of one or more of the identified controls.

2. The computer program product of claim 1 wherein the computer program configures the electronic device to provide templates to recognize the machine, the machine selected from the group consisting of an automatic teller machine (ATM), e-ticket machine, electronic voting machines, and credit/debit point of sale machine.

3. The computer program product of claim 1 wherein the computer program configures the electronic device to recognize on the machine a layout of controls;

and

retrieve by the electronic device data including a representation of the layout of controls, the data retrieved by the electronic device from a knowledge base.

4. The computer program product of claim 1 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls on the machine in the image of the machine by comparing the layout of the controls on the machine to a stored representation of the layout for the machine.

5. The computer program product of claim 1 wherein instructions to process the image, comprises instructions to:

process the image with character recognition to recognize text on controls in the image of the machine;

convert recognized text to speech; and

cause the recognized text to be rendered as speech to provide input to the instructions to announce the text on the controls to the user.

6. The computer program product of claim 1 further comprising instructions to:

process the image to detect the presence of a user-controlled pointing item over controls on the machine; and

form instructions based on the presence of the pointing device to guide the user in operation of the machine.

7. The computer program product of claim 6 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

8. The computer program product of claim 6 further comprising instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

9. The computer program product of claim 6, further comprising instructions to:

determine where the location of the user-controlled pointing item is in the image relative to a selected control in the image; and

announce to the user instructions to guide the user to move the user-controlled pointing item relative to the selected control in the image.

10. The computer program product of claim 1 further comprises instructions to:

retrieve a stored template to recognize the layout of controls on the machine; and

process the image according to the stored template to navigate the user through use of the machine in accordance with the layout of controls provided by the template.

11. The computer program product of claim 1 further instructions to:

process a signal from an RFID tag reader that receives a signal from an RFID tag on the machine to provide a specific description of the layout of controls on the machine.

12. A method of operating a device having a processor, the method comprising:

receiving an image or images of a portion of a machine that the user will use to perform a transaction using the machine;

processing by the processor the image or images to identify a pattern of controls on the machine; and

announcing to the user the name or function of one or more of the controls.

13. The method of claim 12 further comprising;

configuring the device to recognize the machine, the machine selected from the group consisting of an automatic teller machine (ATM), e-ticket machine, electronic voting machines, and credit/debit point of sale machine.

14. The method of claim 12 further comprising:

processing the image to recognize text on controls in the image; and

converting recognized text on the controls to speech to read aloud the text on the controls in the image to the user.

15. The method of claim 12 wherein the machine has a layout of controls the method further comprising:

accessing a knowledge base to provide data to configure the device for a transaction mode with the machine, the data including data representing the layout of controls on the machine.

16. The method of claim 12 further comprising:

processing the image to recognize the controls in the image; and

identifying controls in the layout of the image of the machine by comparing the layout to a stored representation of the layout for the machine.

17. The method of claim 12 wherein announcing further comprises:

processing the image or images to detect the presence of a user-controlled pointing item over controls on the machine; and

announcing to the user the function of a control over which the user-controlled pointing item is placed.

18. The method of claim 17 further comprising:

determining whether the user controlled pointing item is over a control; and

if not,

announcing to the user to move the user-controlled pointing item.

19. The method of claim 17 further comprising:

determining where the location of the user-controlled pointing item is in the image relative to a selected control in the image.

20. The method of claim 12 further comprising:

retrieving a stored template to recognize the layout; and

processing the image according to the stored template to navigate the user through use of the machine.

21. The method of claim 12 further comprising:

processing a signal from an RFID tag reader that receives a signal from an RFID tag on the machine to provide a specific description of the layout of controls on the machine.

22. An apparatus, comprising:

a processor device;

an image input device;

a computer readable medium storing a computer program product for assisting a user of the apparatus to use the apparatus to perform a transaction with a machine, the computer program product comprises instructions for causing the processor device to;

receive from the image input device an image or images of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine; and

announce to the user the name or function of one or more of the identified controls.

23. The apparatus of claim 22 wherein the computer program product is configured with data for the machine, the machine selected from the group consisting of an automatic teller machine (ATM), e-ticket machine, electronic voting machines, and credit/debit point of sale machine.

24. The apparatus of claim 22 wherein the machine has a layout of controls and the layout is included in data retrieved from a knowledge base.

25. The apparatus of claim 22 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls on the machine in the image of the machine by comparing the layout of the controls on the machine in the image to a stored representation of the layout for the machine.

26. The apparatus of claim 22 wherein instructions to process the image, comprises instructions to:

process the image with character recognition to recognize text on controls in the image of the machine; and

convert recognized text to speech to provide input to the instructions to announce the text on the controls to the user.

27. The apparatus of claim 22 wherein the computer program product further comprises instructions to:

process the image to detect the presence of a user-controlled pointing item over controls on the machine; and

form instructions based on the presence of the pointing device to guide the user in operation of the machine.

28. The apparatus of claim 22 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

29. The apparatus of claim 28 wherein the computer program product further comprises instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

30. The apparatus of claim 28 wherein the computer program product further comprises instructions to:

determine where the location of the user-controlled pointing item is in the image relative to a selected control in the image.

31. The apparatus of claim 22 wherein the computer program product comprises instructions to:

retrieve a stored template to recognize a layout of controls on the machine; and

process the image according to the stored template to navigate the user through use of the machine.