Method and Apparatus for Facilitating Use of Touchscreen Devices

• US 20140282002A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Mckiel, Frank A. Jr.
• Granted Patent: US 9,507,561 B2
• Time in Patent Office: Days
• US Class Current: 715/727
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 2200/1633 Protecting arrangement for the entire housing of the computer
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Exemplary embodiments are described wherein an auxiliary sensor attachable to a touchscreen computing device provides an additional form of user input. When used in conjunction with an accessibility process in the touchscreen computing device, wherein the accessibility process generates audible descriptions of user interface features shown on a display of the device, actuation of the auxiliary sensor by a user affects the manner in which concurrent touchscreen input is processed and audible descriptions are presented.

Claims

1. A method comprising:

from at least one auxiliary sensor device that is removably attached to a computing device, receiving at the computing device a communication indicating an actuation state of the auxiliary sensor device;

responsive to a touch gesture received at a touchscreen included with the computing device, producing a first audio signal from the computing device if the actuation state of the auxiliary sensor device corresponds to a first state; and

responsive to the touch gesture, producing a second audio signal from the computing device, different from the first audio signal, if the actuation state of the auxiliary sensor device corresponds to a second state different from the first state.

2. The method of claim 1 further comprising:

obtaining descriptive information for a graphical user interface element displayed under the touchscreen,

producing at least one of the first and second audio signal to include audio information corresponding to the descriptive information.

3. The method of claim 2 wherein the descriptive information comprises descriptive text and wherein the method further comprises:

converting the descriptive text into at least one synthesized speech utterance; and

,

including the synthesized speech utterance in at least one of first and second audio signals.

4. The method of claim 1 further comprising:

determining at least one effect, from among a plurality of selectable effects, that is associated with the auxiliary sensor device and that affects the producing of audio signals by the computing device; and

responsive to the actuation state of the auxiliary sensor device corresponding to the second state, applying the effect to produce the second audio signal, wherein the second audio signal differs from the first audio signal by the application of the effect.

5. The method of claim 4 wherein the first audio signal comprises a first sound followed by a second sound and wherein the act of applying the effect causes the second audio signal to comprise the second sound followed by the first sound.

6. The method of claim 5 wherein at least one of the first sound and second sound is a speech utterance.

7. The method of claim 4 wherein the first audio signal comprises a first set of two or more sounds and wherein the act of applying the effect causes the second audio signal to comprise a subset of the sounds in the first set of sounds.

8. The method of claim 7 wherein the first set of two or more sounds comprises at least one sound that is a speech utterance.

9. The method of claim 4 wherein the first audio signal and second audio signal comprise the same sound and the applying the effect causes the second audio signal to substantially differ in amplitude from the first audio signal.

10. The method of claim 4 wherein the first audio signal and second audio signal both comprise a speech utterance and wherein the act of applying the effect causes the speech utterance as rendered in the second audio signal to exhibit a different speaking rate than the speech utterance as rendered in the first audio signal.

11. The method of claim 1 further comprising:

executing in the computing device, an audible accessibility process that receives event data objects describing touchscreen user input events and affects the producing of audio signals by the computing device;

instantiating at least one event data object pertaining to the touch gesture;

responsive to the actuation state of the auxiliary sensor device corresponding to the first state, providing the event data object to the audible accessibility process such that the first audio signal is produced; and

responsive to the actuation state of the auxiliary sensor device corresponding to the second state, performing a modification upon the event data object and providing the event data object, as modified, to the audible accessibility process such that the second audio signal is produced as a result of the modification.

12. A computing device comprising:

at least one processor executing instructions of an operating system and one or more applications;

at least one touchscreen interface coupled to the processor and operable to communicate gestural user input to the processor;

at least one display visible through the touchscreen and controlled by the processor;

at least one communications interface coupled to an auxiliary sensor device and operable to communicate auxiliary sensor state to the processor; and

at least one audio subsystem producing audio signals under control of the processor;

wherein the processor operates to;

obtain auxiliary sensor state information via the communications interface;

receive at least one instance of gestural user input via the touchscreen interface; and

,

responsive to receiving the at least one instance of gestural user input, controlling the audio subsystem to selectively produce a first audio signal if the auxiliary sensor state corresponds to a first state and to selectively produce a second audio signal if the auxiliary sensor state corresponds to a second state.

13. The computing device of claim 12 wherein the processor operates to:

obtain descriptive information for a graphical user interface element presented on the display;

control the audio subsystem to include audio information corresponding to the descriptive information in at least one of the first and second audio signals.

14. The computing device of claim 13 wherein the processor operates to:

convert the descriptive information into at least one synthesized speech utterance; and

,

control the audio subsystem to include the synthesized speech utterance in at least one of the first and second audio signals.

15. The computing device of claim 12 wherein the processor operates to:

determine at least one effect, from among a plurality of selectable effects, that is associated with the auxiliary sensor device and that affects the producing of audio signals by the computing device; and

responsive to the auxiliary sensor state corresponding to the second state, applying the effect to produce the second audio signal, wherein the second audio signal differs from the first audio signal by the application of the effect.

16. The computing device of claim 15 wherein the first audio signal comprises a first sound followed by a second sound and wherein the act of applying the effect causes the second audio signal to comprise the second sound followed by the first sound.

17. The computing device of claim 16 wherein at least one of the first sound and second sound is a speech utterance.

18. The computing device of claim 15 wherein the first audio signal comprises a first set of two or more sounds and wherein the act of applying the effect causes the second audio signal to comprise a subset of the sounds in the first set of sounds.

19. The computing device of claim 18 wherein the first set of two or more sounds comprises at least one sound that is a speech utterance.

20. The computing device of claim 15 wherein the first audio signal and second audio signal comprise the same sound and the act of applying the effect causes the second audio signal to substantially differ in amplitude from the first audio signal.

21. The computing device of claim 15 wherein the first audio signal and second audio signal both comprise a speech utterance and wherein the act of applying the effect causes the speech utterance as rendered in the second audio signal to exhibit a different speaking rate than the speech utterance as rendered in the first audio signal.

22. The computing device of claim 12 wherein the processor executes an audible accessibility process, operable to receive event data objects describing user input and to affect the producing of the first and second audio signals, and further operates to:

instantiate at least one event data object pertaining to the instance gestural user input;

responsive to the auxiliary sensor state corresponding to the first state, providing, to the audible accessibility process, the event data object to the audible accessibility process such that the first audio signal is produced; and

responsive to the auxiliary sensor state corresponding to the second state, performing a modification upon the event data object and providing the event data object, as modified, to the audible accessibility process such that the second audio signal is produced as a result of the modification.