Interface for a Virtual Digital Assistant

• US 20140040748A1
• Filed: 10/04/2013
• Published: 02/06/2014
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lemay, Stephen O., Sabatelli, Alessandro Francesco, Anzures, Freddy Allen, Chaudhri, Imran, Forstall, Scott, Novick, Gregory
• Granted Patent: US 10,241,752 B2
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 16/9562 Bookmark management
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 10/109 Time management, e.g. calendars, reminders, meetings or time accounting
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 50/10 Services
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

The digital assistant displays a digital assistant object in an object region of a display screen. The digital assistant then obtains at least one information item based on a speech input from a user. Upon determining that the at least one information item can be displayed in its entirety in the display region of the display screen, the digital assistant displays the at least one information item in the display region, where the display region and the object region are not visually distinguishable from one another. Upon determining that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, the digital assistant displays a portion of the at least one information item in the display region, where the display region and the object region are visually distinguishable from one another.

Claims

1. A computer-implemented method of operating a digital assistant on a computing device having at least one processor, memory, and a video display screen, the method comprising:

displaying, at any time, a digital assistant object in an object region of the video display screen;

receiving a speech input from a user;

obtaining at least one information item based on the speech input;

determining whether the at least one information item can be displayed in its entirety in a display region of the video display screen;

upon determining that the at least one information item can be displayed in its entirety in the display region of the display screen, displaying the at least one information item in the display region, where the display region and the object region are not visually distinguishable from one another; and

upon determining that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, displaying a portion of the at least one information item in the display region, where the display region and the object region are visually distinguishable from one another.

2. The method of claim 1, wherein the digital assistant object is displayed in the object region of the video display screen before receiving the speech input.

3. The method of claim 1, wherein the digital assistant object is displayed in the object region of the video display screen after receiving the speech input.

4. The method of claim 1, wherein the digital assistant object is displayed in the object region of the video display screen after determining whether the at least one information item can be displayed in its entirety in a display region of the video display screen.

5. The method of claim 1, wherein the digital assistant object is an icon for invoking a digital assistant service.

6. The method of claim 5, wherein the icon is a microphone icon.

7. The method of claim 1, wherein the digital assistant object shows the status of a current digital assistant process.

8. The method of claim 7, wherein a pending digital assistant process is shown by a swirling light source around the perimeter of the digital assistant object.

9. The method of claim 1, further comprising:

when the entirety of the at least one information item cannot be displayed in the display region;

receiving an input from the user to scroll through the at least one information item so as to display an additional portion of the at least one information item in the display region;

scrolling the portion of the at least one information item towards the object region so that the portion of the at least one information item appears to slide out of view under the object region.

10. The method of claim 1, further comprising:

when the entirety of the at least one information item cannot be displayed in the display region;

receiving a input from the user to scroll through the at least one information item so as to display an additional portion of the at least one information item in the display region;

scrolling the portion of the at least one information item away from the object region so that the additional portion of the at least one information item appears to slide into view from under the object region.

11. The method of claim 1, wherein the speech input is a question or a command from a user.

12. The method of claim 1, wherein obtaining at least one information item comprises obtaining results, a dialog between the user and the digital assistant, a list, or a map.

13. The method of claim 1, wherein when the entirety of the at least one information item is displayed in the display region, the display region and the object region share the same continuous background.

14. The method of claim 1, wherein when the entirety of the at least one information item cannot be displayed in the display region, the display region and the object region are separated by a dividing line.

15. The method of claim 1, wherein when the entirety of the at least one information item cannot be displayed in the display region, the object region looks like a pocket into and out of which the at least one information item can slide.

16. The method of claim 1, wherein when the entirety of the at least one information item cannot be displayed in the display region, an edge of the object region closest to the display region is highlighted while an edge of the at least one information item closest to the object region are tinted.

17. The method of claim 1, further comprising:

displaying, at any time, device information in an information region of the video display screen;

determining whether the at least one information item can be displayed in its entirety in the display region of the video display screen;

upon determining that the at least one information item can be displayed in its entirety in the display region of the display screen, displaying the at least one information item in the display region, where the display region and an information region are not visually distinguishable from one another; and

upon determining that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, displaying a portion of the at least one information item in the display region, where the display region and the information region are visually distinguishable from one another.

18. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to:

display, at any time, a digital assistant object in an object region of a video display screen;

receive a speech input from a user;

obtain at least one information item based on the speech input;

determine whether the at least one information item can be displayed in its entirety in a display region of the video display screen;

upon determining that the at least one information item can be displayed in its entirety in the display region of the display screen, display the at least one information item in the display region, where the display region and the object region are not visually distinguishable from one another; and

upon determining that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, display a portion of the at least one information item in the display region, where the display region and the object region are visually distinguishable from one another.

19. A computing device having at least one processor, memory, and a video display screen, the memory comprising instructions that when executed by a processor, cause the processor to:

display, at any time, a digital assistant object in an object region of a video display screen;

receive a speech input from a user;

obtain at least one information item based on the speech input;

determine whether the at least one information item can be displayed in its entirety in a display region of the video display screen;

upon determining that the at least one information item can be displayed in its entirety in the display region of the display screen, display the at least one information item in the display region, where the display region and the object region are not visually distinguishable from one another; and

upon determining that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, display a portion of the at least one information item in the display region, where the display region and the object region are visually distinguishable from one another.

20. The computing device of claim 19, wherein the computing device is mobile computing device.

21. A graphical user interface on an electronic device with a video display screen, a memory, and one or more processors to execute one or more programs stored in the memory, the graphical user interface comprising:

a digital assistant object in an object region of a video display screen,

wherein;

a speech input is received from a user;

at least one information item based on the speech input is obtained;

it is determined whether the at least one information item can be displayed in its entirety in a display region of the video display screen;

in accordance with a determination that the at least one information item can be displayed in its entirety in the display region of the display screen, the at least one information item is displayed in the display region, where the display region and the object region are not visually distinguishable from one another, and

in accordance with a determination that the at least one information item cannot be displayed in its entirety in the display region of the video display screen, a portion of the at least one information item is displayed in the display region, where the display region and the object region are visually distinguishable from one another.