CONTEXT-AWARE VOICE GUIDANCE

• US 20130322634A1
• Filed: 09/30/2012
• Published: 12/05/2013
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bennett, Jonathan A., Lemay, Stephen O., van Os, Marcel, Forstall, Scott, Moore, Bradford A., Vulcano, Emanuele, Pylappan, Seejo K.
• Granted Patent: US 10,156,455 B2
• Time in Patent Office: Days
• US Class Current: 381/17
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G01C 21/3617 using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3632 Guidance using simplified or iconic instructions, e.g. using arrows G01C21/365 takes precedence
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01C 21/3667 Display of a road map G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 : Speech classification or search
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 : Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command
• CPC Class Codes: G10L 21/00 : Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04R 2430/01 : Aspects of volume control, not necessarily automatic, in sound systems
• CPC Class Codes: H04R 2499/13 : Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 5/00 : Stereophonic arrangements stereophonic pick-ups H04R9/16, H04R11/12, H04R17/08, H04R19/10
• CPC Class Codes: H04R 5/04 : Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

A context-aware voice guidance method is provided that interacts with other voice services of a user device. The voice guidance does not provide audible guidance while the user is making a verbal request to any of the voice-activated services. Instead, the voice guidance transcribes its output on the screen while the verbal requests from the user are received. In some embodiments, the voice guidance only provides a short warning sound to get the user'"'"'s attention while the user is speaking on a phone call or another voice-activated service is providing audible response to the user'"'"'s inquires. The voice guidance in some embodiments distinguishes between music that can be ducked and spoken words, for example from an audiobook, that the user wants to pause instead of being skipped. The voice guidance ducks music but pauses spoken words of an audio book in order to provide voice guidance to the user.

Claims

1. A method of providing context aware voice prompts on an electronic device, the electronic device comprising a plurality of services utilizing audio, the method comprising:

suppressing the audio prompts when a service in the plurality of services is currently cognitively receiving voice input from a human using the electronic device;

allowing non-verbal prompts when a conversational audio session is active between a service in the plurality of services and the human but no service is currently cognitively receiving voice input from the human; and

allowing verbal prompts and non-verbal prompts when no service in the plurality of services has an active conversational audio session with the human and no service in the plurality of services is currently cognitively receiving voice input from the human.

2. The method of claim 1 further comprising:

informing a particular service in the plurality of service on whether non-verbal prompts, verbal prompts, or no verbal prompts are currently allowed;

receiving a voice prompt from the particular service;

playing the voice prompt the prompt comprises spoken words and verbal prompts are currently allowed; and

playing the voice prompt when the prompt does not comprise spoken words and non-verbal prompts are currently allowed.

3. The method of claim 2, wherein the particular service is navigation service, wherein the voice prompt received is a non-verbal prompt comprising directional information.

4. The method of claim 3, wherein the directional information is provided by playing a first set of tones to identify a left direction and a second set of tones to identify a right direction, wherein the first and second sets of tones comprise at least one different tone.

5. The method of claim 3, wherein the directional information is provided by playing a first set of tones to identify a left direction and a second set of tones to identify a right direction, wherein at least one particular tone is included in both the first and second sets of tones, wherein the particular tone is played with a different loudness in each of the first and second sets of tones.

6. The method of claim 3, wherein the electronic device audio is played on a set of stereo speakers, wherein the directional information is provided by panning the voice prompt to a right speaker in the set of stereo speakers when the voice prompt includes right directionality information.

7. The method of claim 3, wherein the electronic device audio is played on a set of stereo speakers, wherein the directional information is provided by panning the voice prompt to a left speaker in the set of stereo speakers when the voice prompt includes left directionality information.

8. The method of claim 1, wherein cognitively receiving voice input from a human comprises:

receiving verbal input from the human; and

utilizing voice recognition to interpret the verbal input.

9. The method of claim 1, wherein cognitively receiving voice input from a human comprises:

receiving a request from the human to record an audio file;

receiving voice input from the human; and

recording the received voice input in the audio file.

10. The method of claim 1, further comprising:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device; and

mixing the verbal prompt with the audio that is currently played on the device.

11. The method of claim 10, wherein the mixing comprises:

assessing how a human perceives a loudness of the audio currently being played on the device; and

ducking at least one of the verbal voice prompt or the audio currently being played based on the assessment.

12. The method of claim 10, wherein the electronic device comprises a set of relative loudness setting for the received voice prompt, wherein the mixing comprises ducking at least one of the verbal voice prompt or the audio currently being played based on the relative loudness setting of the received voice prompt.

13. The method of claim 1, further comprising:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device;

pausing the audio currently played on the device;

playing the verbal prompt; and

resuming playing the audio by the service after the voice prompt is played.

14. The method of claim 13, wherein the service is playing an audio book.

15. A non-transitory machine readable medium storing a program for providing context aware voice prompts on an electronic device, the electronic device comprising a plurality of services utilizing audio, the program executable by at least one processing unit, the program comprising sets of instructions for:

suppressing the audio prompts when a service in the plurality of services is currently cognitively receiving voice input from a human using the electronic device;

allowing non-verbal prompts when a conversational audio session is active between a service in the plurality of services and the human but no service is currently cognitively receiving voice input from the human; and

allowing verbal prompts and non-verbal prompts when no service in the plurality of services has an active conversational audio session with the human and no service in the plurality of services is currently cognitively receiving voice input from the human.

16. The non-transitory machine readable medium of claim 15, the program further comprising sets of instructions for:

informing a particular service in the plurality of service on whether non-verbal prompts, verbal prompts, or no verbal prompts are currently allowed;

receiving a voice prompt from the particular service;

playing the voice prompt the prompt comprises spoken words and verbal prompts are currently allowed; and

playing the voice prompt when the prompt does not comprise spoken words and non-verbal prompts are currently allowed.

17. The non-transitory machine readable medium of claim 16, wherein the particular service is navigation service, wherein the voice prompt received is a non-verbal prompt comprising directional information.

18. The non-transitory machine readable medium of claim 15, wherein the set of instructions for cognitively receiving voice input from a human comprises sets of instructions for:

receiving a request from the human to record an audio file;

receiving voice input from the human; and

recording the received voice input in the audio file.

19. The non-transitory machine readable medium of claim 15, the program further comprising sets of instructions for:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device; and

mixing the verbal prompt with the audio that is currently played on the device.

20. The non-transitory machine readable medium of claim 15, wherein the set of instructions for mixing comprises sets of instructions for:

assessing how a human perceives a loudness of the audio currently being played on the device; and

ducking at least one of the verbal voice prompt or the audio currently being played based on the assessment.

21. The method of claim 15, wherein the electronic device comprises a set of relative loudness setting for the received voice prompt, wherein the set of instructions for mixing comprises a set of instructions for ducking at least one of the verbal voice prompt or the audio currently being played based on the relative loudness setting of the received voice prompt.

22. The non-transitory machine readable medium of claim 15, the program further comprising sets of instructions for:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device;

pausing the audio currently played on the device;

playing the verbal prompt; and

resuming playing the audio by the service after the voice prompt is played.