Tactile graphics display

• US 5,736,978 A
• Filed: 05/26/1995
• Issued: 04/07/1998
• Est. Priority Date: 05/26/1995
• Status: Expired due to Fees

• Current Assignee: Air Force Department of United States of America
• Original Assignee: United States Of America As Represented By The Secretary Of The Air Force
• Inventors: Hasser, Christopher J., Roark, Marvin R.
• Primary Examiner(s): LEE, MICHAEL
• Application Number:
  US08/451,593
• Time in Patent Office: 1,047 Days
• Field of Search: 346/156, 346/157, 346/163, 346/173, 340/825.19, 434/112, 434/113, 434/114, 434/116, 463/30
• US Class Current: 345/173
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G09B 21/001 Teaching or communicating with blind persons G09B21/02 - G09B21/06 take precedence
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays

Abstract

A method for communicating graphic data such as plotted two-dimensional curves to a user such as a sight impaired person is disclosed. The disclosed arrangement uses haptic or tactile stimulation of a user'"'"'s extremity such as is relied--upon in the Braille code fingertip communication already known to many sight impaired persons. Computerized control over a Braille character-like display and use of the computer mouse as a data selection input device; are included in the invention. The invention also includes provision of data enhancement and data interpretation aids including axis names, multiple curve identifications, grid line identifications and the addition of audio information such as tick sounds and spoken utterances to supplement the tactile communication. A computer-based embodiment of the invention is disclosed, this in the form of hardware block diagrams, software flow diagrams and computer code listing, the latter being primarily in microfiche appendix form.

Claims

1. A computer lookthrough and virtual surface inclusive tactile method for communicating multi-dimensional graphic data to a sight impaired and hearing impaired human operator, said method comprising the steps of:

combining a computer data input function of a planar absolute position sensor assembly physically with a computer data output function of an operator finger-engageable, electrically energizable, unitary character, multiple pinned tactile display array to form a portable, bi-directional communication, data transducer device having operator-provided mobility over an absolute position-determining-digitizer pad surface;

storing a bit map representation of said multi-dimensional graphic data in a computer memory having a memory location relating to each pixel point of said absolute position-determining digitizer pad surface according to a selected virtual surface-defining relationship;

accessing said computer bit map data representation in said computer memory data in response to operator induced absolute position changes of said portable, bi-directional communication, data transducer device across said absolute position-determining digitizer pad surface and said virtual surface; and

energizing accessed memory data-determined pixel elements of said multiple pinned tactile character display array, in graphic data-detailed tactile communication with said human operator, and in synchronism with movement and absolute positioning of said portable, bi-directional communication, data transducer device to absolute position-determining digitizer pad surface and virtual surface locations of graphic data presence in said memory.

2. The method of claim 1 wherein:

said multiple pinned tactile display array energized pixel elements have a predetermined pattern relationship with portable, bi-directional communication, data transducer device accessing-determined features of said memory stored multi-dimensional graphic data; and

said predetermined pattern relationship includes a pattern of top to bottom aligned tactile pin display array element energization in response to a ninety degree intersection of said portable, bi-directional communication, data transducer device scanning movement with said graphic data.

3. The method of claim 1 wherein said energizing step includes changing operator perceived axial movement amplitude of pixel-related matrix array elements in said operator finger-engageable, electrically energizable, unitary character, multiple pinned tactile display array in response to accessed memory data and a pin control algorithm.

4. The method of claim 3 wherein said energizing step includes vibrating said pixel related matrix arrayed elements along at least one axis thereof.

5. The method of claim 1 further including the step of adding to said multi-dimensional graphic data additional data representing a display interpretation aid and comprising one of:

graphic data titles,

graphic axis names,

graphic axis tic marks; and

graphic axis tic numbers.

6. The method of claim 1 further including the step of adding to said accessed computer bit map data representation additional data representing tactile grid of horizontal and vertical lines.

7. The method of claim 4 further including the steps of:

adding to said computer memory data an additional array of computer memory data representing an additional data curve; and

coding said additional data with one of a different second pin element vibration frequency and a different second pin element axial amplitude movement.

8. The method of claim 1 further including the step of energizing an audible sound generator in response to selected of said energizing accessed memory data-determined pixel elements steps.

9. The method of claim 8 wherein said audible sound generator comprises one of a tone-pitch generator, a tick sound generator and a spoken sound generator.

10. The method of claim 4 wherein said vibrating step comprises element vibration at differing frequencies of vibration in response to predetermined different data accessings from computer memory.

11. An unconstrained in time method of data display for combined sight and hearing-impaired user haptic realization of graphic data representations, said method comprising the steps of:

representing said graphic data as surface perturbations received on a computer memory data-defined virtual surface area, said surface perturbations corresponding with altered data states in said computer memory and being represented in absolute position registration over said virtual surface;

enabling prolonged, piecemeal, repeated and absolute position-sensed, multiple pinned display-array-communicated haptic exploration of said virtual surface and said data surface perturbations by said user, said prolonged, piecemeal, repeated and absolute position-sensed, multiple pinned display-array-communicated haptic exploration including multiple data point haptic array continuing readout accessing of portions of said computer memory in response to physical manipulation of a memory access control across an absolute position-registered virtual surface by said user person;

communicating results of said continuing readout accessing of said computer memory, including virtual surface data perturbation presence and absence-related memory data, to said user person as finger-received multiple data point small area haptic data samples of said memory contents at attained absolute positions on said virtual surface.

12. The method of claim 11 further including the steps of conveying a location of each said haptic data sample on said virtual surface, and a location of said haptic data sample data on a digitizer pad surface, to said user person through accomplishing said conveying step at a digitizer pad surface physical position also determined by said memory access control.

13. The method of claim 12 wherein:

said enabling step physical manipulation of a memory access control comprises positioning an absolute position sensing mouse pointer on said digitizer pad surface by said user person; and

said conveying of a location of each said haptic data sample comprises positioning a combined absolute position sensing mouse pointer and a tactile feedback array matrix received in connection therewith at perturbation-related portions of said virtual surface area by said user person.

14. The method of claim 11 wherein said conveying step includes energizing data sample-determined individual pixel pins of an electrically energizable tactile feedback communicating array engaged by said user person.

15. The method of claim 11 wherein said continuing readout accessing of portions of said computer memory comprises reading a human finger size-related tactile array matrix determining portion of said memory.

16. The method of claim 15 wherein said continuing readout accessing of portions of said computer memory comprises reading a zoom-in and zoom-out control determined user selected image portion of said computer memory.

17. The method of claim 11 wherein said step of enabling prolonged, piecemeal and repeated haptic exploration of said virtual surface and said data surface perturbations by said user person, and said prolonged, piecemeal and repeated haptic exploration continuing readout accessing of portions of said computer memory in response to physical manipulation of a memory access control by said user person, further include the step of:

supplementing a less than complete data representation encounter of said data surface perturbations by said physical manipulation of a memory access control with a predetermined search-pattern-determined quantum of acquired additional contiguous data.

18. The method of claim 17 wherein:

said less than complete data representation encounter of said data surface perturbations comprises encountering a partial portion of a Braille character matrix by said memory access control; and

said predetermined search-pattern-determined quantum of acquired additional contiguous data comprises a remainder portion of said Braille character matrix.

19. The method of claim 18 wherein said supplementing step and said predetermined search pattern acquiring include execution of a computer software routine.

20. The method of claim 11 wherein said continuing readout accessing of portions of said computer memory comprises two dimensional physical manipulation of said memory access control by said user person, two dimensional perception of Braille coded letters on said virtual surface, and user access to a full page Braille display of characters.