Method and apparatus for facilitating use of touchscreen devices

• US 9,507,561 B2
• Filed: 03/15/2013
• Issued: 11/29/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Mckiel, Frank A. Jr.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): NGUYEN, NHAT HUY T
• Application Number:
  US13/838,603
• Publication Number:
  US 20140282002A1
• Time in Patent Office: 1,355 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 2200/1633 Protecting arrangement for the entire housing of the computer
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Exemplary embodiments are described wherein an auxiliary sensor attachable to a touchscreen computing device provides an additional form of user input. When used in conjunction with an accessibility process in the touchscreen computing device, wherein the accessibility process generates audible descriptions of user interface features shown on a display of the device, actuation of the auxiliary sensor by a user affects the manner in which concurrent touchscreen input is processed and audible descriptions are presented.

Claims

1. A method comprising:

from at least one auxiliary sensor device that is attached to a computing device as a removable outer housing surrounding a touchscreen of the computing device, receiving at the computing device a communication indicating an actuation state of the auxiliary sensor device;

obtaining descriptive information for a graphical user interface element displayed under the touchscreen;

responsive to a touch gesture received at the touchscreen included with the computing device, producing a first audio signal, based on the descriptive information, from the computing device if the actuation state of the auxiliary sensor device corresponds to a first state; and

responsive to the touch gesture, producing a second audio signal, based on the descriptive information, from the computing device, different from the first audio signal, if the actuation state of the auxiliary sensor device corresponds to a second state different from the first state,

wherein the first and second audio signals each include speech utterances, and the first and second audio signals are different from one another based at least on the first and second audio signals including different speech utterances,

wherein the first audio signal includes first speech utterances that describe a function of the graphical user element, and the second audio signal includes second speech utterances that describe the function of the graphical user element, the second speech utterances providing a more detailed description of the function of the graphical user interface than the first speech utterances.

2. The method of claim 1, wherein the descriptive information comprises descriptive text and wherein the method further comprises:

converting the descriptive text into at least one synthesized speech utterance; and

,

including the synthesized speech utterance in the first and second audio signals.

3. The method of claim 1, wherein the first audio signal comprises a first sound followed by a second sound and wherein the second audio signal includes the second sound followed by the first sound.

4. The method of claim 1, wherein the first audio signal comprises a first set of two or more sounds and wherein the second audio signal comprises a subset of the sounds in the first set of sounds.

5. The method of claim 1, further comprising:

determining at least one effect, from among a plurality of selectable effects; and

applying the effect to cause the second audio signal to substantially differ in amplitude from the first audio signal.

6. The method of claim 1, further comprising:

determining at least one effect, from among a plurality of selectable effects, wherein applying the effect causes the speech utterance as rendered in the second audio signal to exhibit a different speaking rate than the speech utterance as rendered in the first audio signal.

7. A method comprising:

from at least one auxiliary sensor device that is attached to a computing device as a removable outer housing surrounding a touchscreen of the computing device, receiving at the computing device a communication indicating an actuation state of the auxiliary sensor device;

obtaining descriptive information for a graphical user interface element displayed under the touchscreen;

responsive to a touch gesture received at the touchscreen included with the computing device, producing a first audio signal, based on the descriptive information, from the computing device if the actuation state of the auxiliary sensor device corresponds to a first state; and

responsive to the touch gesture, producing a second audio signal, based on the descriptive information, from the computing device, different from the first audio signal, if the actuation state of the auxiliary sensor device corresponds to a second state different from the first state,

wherein the first and second audio signals each include speech utterances, and the first and second audio signals are different from one another based at least on the first and second audio signals including different speech utterances,

wherein the first audio signal includes first speech utterances that describe a function of the graphical user element and the second audio signal includes second speech utterances that describe a type of the graphical user element.

8. The method of claim 7, further comprising:

determining at least one effect, from among a plurality of selectable effects; and

applying the effect to cause the second audio signal to substantially differ in amplitude from the first audio signal.

9. The method of claim 7, further comprising:

determining at least one effect, from among a plurality of selectable effects, wherein applying the effect causes the speech utterance as rendered in the second audio signal to exhibit a different speaking rate than the speech utterance as rendered in the first audio signal.

10. The method of claim 7, further comprising:

executing in the computing device, an audible accessibility process that receives event data objects describing touchscreen user input events and affects the producing of audio signals by the computing device;

instantiating at least one event data object pertaining to the touch gesture;

responsive to the actuation state of the auxiliary sensor device corresponding to the first state, providing the event data object to the audible accessibility process such that the first audio signal is produced; and

responsive to the actuation state of the auxiliary sensor device corresponding to the second state, performing a modification upon the event data object and providing the event data object, as modified, to the audible accessibility process such that the second audio signal is produced as a result of the modification.

11. A system comprising:

an auxiliary sensor device coupled to a computing device as a removable outer housing surrounding a touchscreen interface of the computing device; and

a computing device including;

at least one processor executing instructions of an operating system and one or more applications;

the touchscreen interface, the touchscreen interface being coupled to the processor and operable to communicate gestural user input to the processor;

at least one display visible through the touchscreen and controlled by the processor;

at least one communications interface coupled to the auxiliary sensor device and operable to communicate auxiliary sensor state to the processor; and

at least one audio subsystem producing audio signals under control of the processor;

wherein the processor operates to;

obtain auxiliary sensor state information via the communications interface;

obtain descriptive information for a graphical user interface element displayed by the display;

receive at least one instance of gestural user input via the touchscreen interface; and

responsive to receiving the at least one instance of gestural user input, controlling the audio subsystem to selectively produce a first audio signal, based on the descriptive information, if the auxiliary sensor state corresponds to a first state and to selectively produce a second audio signal, based on the descriptive information, if the auxiliary sensor state corresponds to a second state,

wherein the first and second audio signals each include speech utterances and the first and second audio signals are different from one another based at least on the first and second audio signals including different speech utterances,

wherein the first audio signal includes first speech utterances that describe a function of the graphical user element and the second audio signal includes second speech utterances that describe a type of the graphical user element.

12. The system of claim 11, wherein the processor operates to:

convert the descriptive information into at least one synthesized speech utterance; and

,

control the audio subsystem to include the synthesized speech utterance in the first and second audio signals.

13. The system of claim 11, wherein the first audio signal comprises a first sound followed by a second sound and wherein the second audio signal includes the second sound followed by the first sound.

14. The system of claim 11, wherein the first audio signal comprises a first set of two or more sounds and wherein the second audio signal comprises a subset of the sounds in the first set of sounds.

15. The system of claim 14 wherein the first set of two or more sounds comprises at least one sound that is a speech utterance.

16. A system comprising:

an auxiliary sensor device coupled to a computing device as a removable outer housing surrounding a touchscreen interface of the computing device; and

a computing device including;

at least one processor executing instructions of an operating system and one or more applications;

the touchscreen interface, the touchscreen interface being coupled to the processor and operable to communicate gestural user input to the processor;

at least one display visible through the touchscreen and controlled by the processor;

at least one communications interface coupled to the auxiliary sensor device and operable to communicate auxiliary sensor state to the processor; and

at least one audio subsystem producing audio signals under control of the processor;

wherein the processor operates to;

obtain auxiliary sensor state information via the communications interface;

obtain descriptive information for a graphical user interface element displayed by the display;

receive at least one instance of gestural user input via the touchscreen interface; and

responsive to receiving the at least one instance of gestural user input, controlling the audio subsystem to selectively produce a first audio signal, based on the descriptive information, if the auxiliary sensor state corresponds to a first state and to selectively produce a second audio signal, based on the descriptive information, if the auxiliary sensor state corresponds to a second state,

wherein the first and second audio signals each include speech utterances and the first and second audio signals are different from one another based at least on the first and second audio signals including different speech utterances,

wherein the first audio signal includes first speech utterances that describe a function of the graphical user element and the second audio signal includes second speech utterances that describe the function of the graphical user element, the second speech utterances providing a more detailed description of the function of the graphical user interface than the first speech utterances.

17. The system of claim 16, wherein the processor additionally operates to:

determine at least one effect, from among a plurality of selectable effects, and

applying the effect to cause the second audio signal to substantially differ in amplitude from the first audio signal.

18. The system of claim 16, wherein the processor additionally operates to:

determine at least one effect, from among a plurality of selectable effects, wherein applying the effect causes the speech utterance as rendered in the second audio signal to exhibit a different speaking rate than the speech utterance as rendered in the first audio signal.

19. The system of claim 16, wherein the processor executes an audible accessibility process, operable to receive event data objects describing user input and to affect the producing of the first and second audio signals, and further operates to:

instantiate at least one event data object pertaining to the instance of gestural user input;

responsive to the auxiliary sensor state corresponding to the first state, providing, to the audible accessibility process, the event data object to the audible accessibility process such that the first audio signal is produced; and

responsive to the auxiliary sensor state corresponding to the second state, performing a modification upon the event data object and providing the event data object, as modified, to the audible accessibility process such that the second audio signal is produced as a result of the modification.