Device and Method to Assist User in Conducting A Transaction With A Machine

• US 20100266205A1
• Filed: 01/05/2010
• Published: 10/21/2010
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Gibson, Lucy, Albrecht, Paul, Kurzweil, Raymond C., Gashel, James
• Granted Patent: US 8,186,581 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted

Abstract

A device for assisting a user to perform a transaction on a machine is described. The device receives data that specifies a transaction mode to use for processing an image and accesses a knowledge base to provide data to configure the device for the transaction mode, the data including data specific to the transaction mode. The device receives an image or images of a portion of a machine that the user will use to perform the transaction and processes the image or images to identify a pattern of controls on the machine and to detect the presence of a user-controlled pointing item over controls on the machine. The device announces to the user the name or function of the control closest to an end of the user-controlled pointing item.

Claims

1-31. -31. (canceled)

32. A computer program product residing on a computer readable medium for assisting a user of a device to use the device to perform a transaction on a machine, the computer program product comprises instructions for causing the device to:

receive an image or images of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine; and

announce to the user the name or function of one or more of the identified controls.

33. The computer program product of claim 32 wherein the machine is an automatic teller machine (ATM), e-ticket machine, electronic voting machines, or credit/debit point of sale machine.

34. The computer program product of claim 32 wherein the machine has a layout of controls and the layout is included in data retrieved by the device from a knowledge base.

35. The computer program product of claim 32 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls on the machine in the image of the machine by comparing the layout of the controls on the machine to a stored representation of the layout for the machine.

36. The computer program product of claim 32 wherein instructions to process the image, comprises instructions to:

process the image with character recognition to recognize text on controls in the image of the machine; and

convert text to speech to announce the text on the controls to the user.

37. The computer program product of claim 32 further comprising instructions to:

process the image to detect the presence of a user-controlled pointing item over controls on the machine; and

form instructions to guide the user in operation of the machine.

38. The computer program product of claim 37 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

39. The computer program product of claim 37 further comprising instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

40. The computer program product of claim 37 further comprising instructions to:

announce to the user to move the user-controlled pointing item comprises instructions to;

determine where the location of the user-controlled pointing item in the image relative to a selected control in the image.

41. The computer program product of claim 32 further comprises instructions to:

retrieve a stored template to recognize the layout; and

process the image according to the stored template to navigate the user through use of the machine for transaction devices that have standardized or memorized layouts of controls.

42. The computer program product of claim 32 further instructions to:

process a signal from an RFID tag reader that receives a signal from an RFID tag on the machine to provide a specific description of the layout of controls on the machine.

43. A method of operating a device, the method comprising:

receiving an image or images of a portion of a machine that the user will use to perform the transaction;

processing the image or images to identify a pattern of controls on the machine; and

announcing to the user the name or function of one or more of the controls.

44. The computer program product of claim 43 wherein the machine is an automatic teller machine (ATM), e-ticket machine, electronic voting machines, or credit/debit point of sale machine.

45. The method of claim 43 further comprising:

processing the image to recognize text on controls in the image; and

converting recognized text on the controls to speech to read aloud the text on the controls in the image to the user.

46. The method of claim 43 wherein the machine has a layout of controls the method further comprising:

accessing a knowledge base to provide data to configure the device for the transaction mode, the data including data representing the layout of controls on the machine.

47. The method of claim 43 further comprising:

processing the image to recognize the controls in the image; and

identifying controls in the layout of the image of the machine by comparing the layout to a stored representation of the layout for the machine.

48. The method of claim 43 wherein announcing further comprises:

processing the image or images to detect the presence of a user-controlled pointing item over controls on the machine; and

announcing to the user the function of a control over which the user-controlled pointing item is placed.

49. The method of claim 48 further comprising:

determining whether the user controlled pointing item is over a control; and

if not,

announcing to the user to move the user-controlled pointing item.

50. The method of claim 48 further comprising:

determining where the location of the user-controlled pointing item is in the image relative to a selected control in the image.

51. The method of claim 43 further comprising:

retrieving a stored template to recognize the layout; and

processing the image according to the stored template to navigate the user through use of the machine.

52. The method of claim 43 further comprising:

processing a signal from an RFID tag reader that receives a signal from an RFID tag on the machine to provide a specific description of the layout of controls on the machine.

53. An apparatus, comprising:

a computing device;

an image input device;

a computer readable medium storing a computer program product assisting a user of the apparatus to use the apparatus to perform a transaction with a machine, the computer program product comprises instructions for causing the computing device to;

receive an image or images of a portion of a machine that the user will use to perform the transaction;

process the image or images to identify a pattern of controls on the machine; and

announce to the user the name or function of one or more of the identified controls.

54. The apparatus of claim 53 wherein the computer program product is configured with data for a machine that is an automatic teller machine (ATM), e-ticket machine, electronic voting machines, or credit/debit point of sale machine.

55. The apparatus of claim 53 wherein the transaction-oriented device has a layout of controls and the layout is included in data retrieved from a knowledge base.

56. The apparatus of claim 53 wherein instructions to process the image, comprises instructions to:

process the image to recognize the controls on the machine in the image of the machine by comparing the layout of the controls on the machine to a stored representation of the layout for the machine.

57. The apparatus of claim 53 wherein instructions to process the image, comprises instructions to:

process the image with character recognition to recognize text on controls in the image of the machine; and

convert text to speech to announce the text on the controls to the user.

58. The apparatus of claim 53 wherein the computer program product further comprises instructions to:

process the image to detect the presence of a user-controlled pointing item over controls on the machine; and

form instructions to guide the user in operation of the machine.

59. The apparatus of claim 53 wherein instructions to guide the user in operation of the machine, further comprise instructions to:

announce to the user the function of a control over which the user-controlled pointing item is placed.

60. The apparatus of claim 59 wherein the computer program product further comprises instructions to:

determine whether the user controlled pointing item is over a control; and

if not,

announce to the user to move the user-controlled pointing item.

61. The apparatus of claim 59 wherein the computer program product further comprises instructions to:

determine where the location of the user-controlled pointing item in the image relative to a selected control in the image.

62. The apparatus of claim 53 wherein the computer program product comprises instructions to:

retrieve a stored template to recognize a layout for machines that have standardized or memorized layouts of controls; and

process the image according to the stored template to navigate the user through use of the machine.