Interaction with notifications across devices with a digital assistant

• US 10,528,228 B2
• Filed: 06/21/2017
• Issued: 01/07/2020
• Est. Priority Date: 06/21/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Seixeiro, Steve, Virdi, Gurpreet, Qin, Lijuan, Wood, David Jeremy Guy, Goulding, Michael Burt, Scotta, Lucas
• Primary Examiner(s): Pham, Linh K
• Application Number:
  US15/629,450
• Publication Number:
  US 20180373398A1
• Time in Patent Office: 930 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04M 1/00 Substation equipment, e.g. for use by subscribers subscriber services or facilities provided at exchanges H04M3/00; prepayment telephone coin boxes H04M17/00; current supply arrangements H04M19/08
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04W 4/12 Messaging; Mailboxes; Announcements
• CPC Class Codes: H04W 68/00 User notification, e.g. alerting and paging, for incoming communication, change of service or the like

Abstract

A digital assistant supported on computing devices such as smartphones, tablet computers, personal computers (PCs), wearable devices, game consoles, smart home devices and the like is configured with a notification system that interfaces with applications and/or other systems and resources so that the digital assistant may collect notifications across a range of a user'"'"'s devices. The digital assistant may utilize a natural language interface (which may cooperate with touch and gesture interfaces) and applies awareness of context and user'"'"'s behaviors to intelligently present the collected notifications on a single device for the user as a native digital assistant user experience. The digital assistant can sort notifications, group notifications together, and interact with the user to answer questions, provide information and status, and enable the user to action on a notification from the collection.

Claims

1. A computing device, comprising:

one or more processors;

a natural language user interface (UI) configured for natural language interactions with a user of the computing device; and

one or more hardware-based memory devices not consisting of propagated signals storing one or more applications and computer-readable instructions which, when executed by the one or more processors, cause the computing device to

receive notifications collected across a plurality of different computing devices that are each associated with and utilized by the user, and wherein the notifications are directed to the user who utilizes the plurality of different computing devices,

provide a digital assistant on the computing device, the digital assistant configured to interact with the user through the natural language UI,

select and group notifications from the collected notifications to expose to the user,

present the notifications in the group in a single view on the natural language UI of the computing device,

enable the user to interact with the presented notifications from the collection using the digital assistant, and

operate the digital assistant in response to the user interactions.

2. The computing device of claim 1 in which the executed instructions further cause the computing device to receive an interaction from the user of the digital assistant using a physical interaction with the computing device or a gesture that is detected by the computing device.

3. The computing device of claim 1 in which the executed instructions further cause the computing device to expose the selected notification using one or more of graphics, audio, text, animation, or combinations thereof.

4. The computing device of claim 1 in which the executed instructions further cause the computing device to select notifications by filtering the collected notifications by one or more of priority, computing device, application, sender, subject matter, type, or context.

5. The computing device of claim 1 in which the executed instructions further cause the computing device to group notifications by one or more of priority, computing device, application, sender, subject matter, type, or context.

6. The computing device of claim 5 in which the context comprises one or more of time/date, location of the user or device, language, schedule, applications installed on the computing device, user preferences, user behaviors, user activities, stored contacts, call history, messaging history, browsing history, device type, device capabilities, or communication network type.

7. The computing device of claim 5 in which the plurality of computing devices is associated with the user by one or more of account, identity, or sign-in.

8. The computing device of claim 1 in which the operating includes performing an action on one or more of the notifications, the action comprising one or more of sharing contact information, sharing a stored contact, scheduling a meeting, looking at a user'"'"'s calendar, scheduling a reminder, making a call, operating a device, playing games, making purchases, taking notes, scheduling an alarm or wake-up reminder, sending a message, checking social media for updates, scraping a website, interacting with a search service, sharing or showing files, sending a link to a website, or sending a link to a resource.

9. The computing device of claim 1 in which the collected notifications pertain to one or more of message, event, or alert.

10. A computer-implemented method for utilizing a digital assistant operating on a computing device, comprising:

configuring the digital assistant for context-awareness and interactions with a user of the computing device using a natural language interface;

monitoring for notifications that are presented on the computing device;

sending one or more of the presented notifications to a remote service for incorporation in a collection along with notifications presented on one or more other computing devices that are associated with the user;

presenting the collection of notifications on the computing device in a single view; and

enabling the user to interact with the presented collection using the digital assistant.

11. The computer-implemented method of claim 10 further comprising configuring notifications in the presented collection to be actionable.

12. The computer-implemented method of claim 10 in which the notifications are generated by an application or a system that operates on the computing device.

13. The computer-implemented method of claim 10 in which the presentation of the notification collection is invoked by keyword.

14. The computer-implemented method of claim 10 in which the presentation of the notification collection is invoked by user speech received at the natural language interface.

15. The computer-implemented method of claim 10 in which the sent notifications are represented by respective signals comprising content and metadata.

16. The computer-implemented method of claim 10 in which the single view provides a comprehensive presentation of notifications across a plurality of computing devices.

17. One or more hardware-based computer-readable memory devices not consisting of propagated signals storing instructions which, when executed by one or more processors disposed in a computer server, cause the computer server to:

receive signals from a plurality of computing devices, each of the signals being respectively associated with one or more notifications that are respectively presented on each of the plurality of computing devices, wherein the plurality of computing devices are associated with and utilized by a unique user, and wherein the one or more notifications are directed to the unique user who utilizes the plurality of computing devices;

store the notifications associated with the received signals in a collection; and

send a group of the notifications from the collection to a remote computing device of the plurality of computing devices for presentation, in which the notifications are configured in a group format for presentation in a single view on the remote computing device.

18. The one or more hardware-based computer-readable memory devices of claim 17 in which the signals include notification content and metadata associated with the notification.

19. The one or more hardware-based computer-readable memory devices of claim 17 in which the executed instructions further cause the computer server to subject the collection of notifications to one or more of filtering, sorting, or actioning.