Context-aware voice guidance

• US 10,156,455 B2
• Filed: 09/30/2012
• Issued: 12/18/2018
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bennett, Jonathan A., Lemay, Stephen O., van Os, Marcel, Forstall, Scott, Moore, Bradford A., Vulcano, Emanuele, Pylappan, Seejo K.
• Primary Examiner(s): Goins, Davetta W
• Assistant Examiner(s): SELLERS, DANIEL R
• Application Number:
  US13/632,121
• Publication Number:
  US 20130322634A1
• Time in Patent Office: 2,270 Days
• Field of Search: 381 17, 381 86, 381302, 700 94
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G01C 21/3617 using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3632 Guidance using simplified or iconic instructions, e.g. using arrows G01C21/365 takes precedence
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01C 21/3667 Display of a road map G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G01C 21/3676 Overview of the route on the road map
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 : Speech classification or search
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 : Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command
• CPC Class Codes: G10L 21/00 : Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04R 2430/01 : Aspects of volume control, not necessarily automatic, in sound systems
• CPC Class Codes: H04R 2499/13 : Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 5/00 : Stereophonic arrangements stereophonic pick-ups H04R9/16, H04R11/12, H04R17/08, H04R19/10
• CPC Class Codes: H04R 5/04 : Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: Y02D 30/70 : in wireless communication networks

Abstract

A context-aware voice guidance method is provided that interacts with other voice services of a user device. The voice guidance does not provide audible guidance while the user is making a verbal request to any of the voice-activated services. Instead, the voice guidance transcribes its output on the screen while the verbal requests from the user are received. In some embodiments, the voice guidance only provides a short warning sound to get the user'"'"'s attention while the user is speaking on a phone call or another voice-activated service is providing audible response to the user'"'"'s inquires. The voice guidance in some embodiments distinguishes between music that can be ducked and spoken words, for example from an audiobook, that the user wants to pause instead of being skipped. The voice guidance ducks music but pauses spoken words of an audio book in order to provide voice guidance to the user.

Claims

1. A method of providing context aware audio prompts associated with a navigation presentation on an electronic device, the method comprising:

identifying a navigation instruction to provide for a navigation maneuver;

determining an allowed type of audio prompt for the electronic device to provide for the identified navigation instruction by determining whether any audio service of the device is currently being utilized to receive first or second type of voice input,

said determining an allowed type of audio prompt comprising;

based on detecting that a first audio service is currently receiving the first type of voice input, determining that no audio prompt is allowed;

based on detecting that a second audio service is currently receiving the second type of voice input different from the first type of audio input, determining that a non-verbal audio prompt is allowed; and

based on detecting that no audio service is currently receiving the first or second type of voice input, determining that a verbal audio prompt is allowed; and

providing the allowed type of audio prompt for the identified navigation instruction.

2. The method of claim 1, wherein receiving the first type of voice input comprises:

receiving voice input; and

utilizing voice recognition to interpret the voice input.

3. The method of claim 2, wherein the second type of voice input is audio input associated with a phone call or video conference.

4. The method of claim 1, wherein receiving the first type of voice input comprises:

receiving a request to record an audio file;

receiving voice input; and

recording the received voice input in the audio file.

5. The method of claim 1, wherein providing the allowed type of audio prompt further comprises:

providing a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device; and

mixing the verbal prompt with the audio that is currently played on the device.

6. The method of claim 5, wherein the mixing comprises:

assessing loudness of the audio currently being played on the device; and

ducking at least one of the verbal prompts or the audio currently being played based on the assessment.

7. The method of claim 5, wherein the electronic device comprises a set of relative loudness settings for the received verbal prompt, wherein the mixing comprises ducking at least one of the verbal prompts or the audio currently being played based on the relative loudness setting of the received verbal prompt.

8. The method of claim 5, wherein the mixing comprises ducking at least one of the verbal prompts or the audio currently being played.

9. The method of claim 1, wherein providing the allowed type of audio prompt further comprises:

providing a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device;

pausing the audio currently played on the device;

playing the verbal prompt; and

resuming playing the audio by the service after the verbal prompt is played.

10. The method of claim 9, wherein the service is playing an audio book.

11. The method of claim 1, wherein determining that no audio prompt is allowed further comprises determining that all audio prompts from all audio services are suppressed.

12. A method of providing context aware audio prompts associated with a navigation presentation of a navigation service on an electronic device, the method comprising:

receiving an inquiry about a type of audio prompt that can currently be played for an identified navigation instruction;

determining whether any audio service of the device is currently being utilized to receive first or second type of voice input; and

based on the determination, dynamically specifying the type of audio prompt for the electronic device to provide for the identified navigation instruction,

said dynamically specifying comprising;

informing the navigation service that no audio prompt is allowed when a first audio service is currently receiving the first type of voice input;

informing the navigation service that a non-verbal audio prompt is allowed when a second audio service is currently receiving the second type of voice input different from the first type of audio input; and

informing the navigation service that a verbal audio prompts is allowed when no audio service is currently receiving the first or second type of voice input.

13. The method of claim 12, wherein a non-verbal prompt comprises directional information.

14. The method of claim 13, wherein the directional information is provided by playing a first set of tones to identify a left direction and a second set of tones to identify a right direction, wherein the first and second sets of tones comprise at least one different tone.

15. The method of claim 13, wherein the directional information is provided by playing a first set of tones to identify a left direction and a second set of tones to identify a right direction, wherein at least one particular tone is included in both the first and second sets of tones, wherein the particular tone is played with a different loudness in each of the first and second sets of tones.

16. The method of claim 13, wherein the electronic device audio is played on a set of stereo speakers, wherein the directional information is provided by panning the audio prompt to a right speaker in the set of stereo speakers when the audio prompt includes right directionality information.

17. The method of claim 13, wherein the electronic device audio is played on a set of stereo speakers, wherein the directional information is provided by panning the audio prompt to a left speaker in the set of stereo speakers when the audio prompt includes left directionality information.

18. A non-transitory machine readable medium storing a program for providing context aware audio prompts associated with a navigation presentation on an electronic device, the program executable by at least one processing unit, the program comprising sets of instructions for:

identifying a navigation instruction to provide for a navigation maneuver;

determining an allowed type of audio prompt for the electronic device to provide for the identified navigation instruction by determining whether any audio service of the device is currently being utilized to receive first or second type of voice input,

said set of instructions for determining an allowed type of audio prompt comprising sets of instructions for;

based on detecting that a first audio service is currently receiving the first type of voice input, determining that no audio prompt is allowed;

based on detecting that a second audio service is currently receiving the second type of voice input different from the first type of audio input, determining that a non-verbal audio prompt is allowed; and

based on detecting that no audio service is currently receiving the first or second type of voice input, determining that a verbal audio prompt is allowed; and

providing the allowed type of audio prompt for the identified navigation instruction.

19. The non-transitory machine readable medium of claim 18, the program further comprising sets of instructions for:

informing a navigation service of the electronic device whether non-verbal prompts, verbal prompts, or no verbal prompts are currently allowed; and

receiving an audio prompt from the navigation service;

said receiving an audio prompt comprising,

when the navigation service has been informed that no audio prompt is allowed, receiving no audio prompt;

when the navigation service has been informed that a non-verbal audio prompt is allowed, receiving a non-verbal audio prompt;

when the navigation service has been informed that a verbal audio prompt is allowed, receiving a verbal audio prompt,

wherein the set of instructions for providing the allowed type of audio prompt comprises a set of instructions for playing the received audio prompt.

20. The non-transitory machine readable medium of claim 19, wherein the received audio prompt is a non-verbal prompt comprising directional information.

21. The non-transitory machine readable medium of claim 18, wherein receiving a first type of voice input comprises:

receiving a request to record an audio file;

receiving voice input; and

recording the received voice input in the audio file.

22. The non-transitory machine readable medium of claim 18, wherein the set of instructions for providing the allowed type of audio prompt further comprises sets of instructions for:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device; and

mixing the verbal prompt with the audio that is currently played on the device.

23. The non-transitory machine readable medium of claim 22, wherein the set of instructions for mixing comprises sets of instructions for:

assessing loudness of the audio currently being played on the device; and

ducking at least one of the verbal prompts or the audio currently being played based on the assessment.

24. The non-transitory machine readable medium of claim 22, wherein the electronic device comprises a set of relative loudness settings for the received verbal prompt, wherein the set of instructions for mixing comprises a set of instructions for ducking at least one of the verbal prompts or the audio currently being played based on the relative loudness setting of the received verbal prompt.

25. The non-transitory machine readable medium of claim 18, wherein the set of instructions for providing the allowed type of audio prompt further comprises sets of instructions for:

receiving a verbal prompt when verbal prompts are allowed and a service is currently playing audio on the device;

pausing the audio currently played on the device;

playing the verbal prompt; and

resuming playing the audio by the service after the verbal prompt is played.

26. The non-transitory machine readable medium of claim 18, wherein the set of instructions for determining that no audio prompt is allowed further comprises determining that all audio prompts from all audio services are suppressed.