Contextual note taking

• US 10,121,474 B2
• Filed: 03/07/2016
• Issued: 11/06/2018
• Est. Priority Date: 02/17/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Yu, Jamie, Rajamani, Roopesh, Krishna, Om, Vaidyanathan, Priya
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Le, Thuykhanh
• Application Number:
  US15/062,797
• Publication Number:
  US 20170236517A1
• Time in Patent Office: 974 Days
• Field of Search: None
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/08 providing for individual presentation of information to a plurality of student stations
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

Contextual note taking is described. A note taking assistant can receive an indication of a specific presentation session. This indication can be used by the note taking assistant to access information or content related to the session. The note taking assistant can receive specific presentation session content, which includes identifiable context images. Identifiable context images are meant to define an individual page, an individual slide, or other atomic unit in the presentation. The note taking assistant operates by receiving a navigation message, changing the current assistant context image to a current presenter context image based on the navigation message; receiving a speech-to-text message comprising a unit of text; displaying the current presenter context image, and displaying the unit of text associated with the current presenter context image; and storing the unit of text associated with the current presenter context image.

Claims

1. A method facilitating contextual note taking, the method comprising:

receiving, at an end user device, an indication of a specific presentation session;

receiving, at the end user device, specific presentation session content, the specific presentation session content comprising identifiable context images;

receiving, from a presenter device, a navigation message via a communication interface of the end user device, wherein the navigation message comprises a location identifier, the location identifier being used to identify a current presenter context image;

responsive to receiving the navigation message, changing, at the end user device, a current assistant context image to the current presenter context image, the current assistant context image corresponding to a context image to which any units of text being received are assigned and the current presenter context image corresponding to a context image a presenter is currently on;

receiving, at the end user device, a speech-to-text message comprising a unit of text associated with the current presenter context image from the presenter device;

displaying, at the end user device, the unit of text associated with the current presenter context image; and

storing the unit of text associated with the current presenter context image regardless of a context image that an end user is currently viewing at the end user device.

2. The method of claim 1, further comprising:

receiving an indication of a target language;

as the unit of text associated with the current presenter context image is received, sending a request to a translation service, the request including the target language and the unit of text associated with the current presenter context image;

receiving a translated unit of text associated with the current presenter context image;

displaying the translated unit of text associated with the current presenter context image; and

storing the translated unit of text associated with the current presenter context image.

3. The method of claim 1, further comprising:

populating a note taking form with the specific presentation session content, including a plurality of the identifiable context images.

4. The method of claim 3, further comprising:

identifying the current presenter context image in the note taking form, wherein the unit of text is inserted and displayed in the note taking form associated with the current presenter context image.

5. The method of claim 4, further comprising:

receiving and displaying user input to the note taking form in a region associated with a different context image than that having the unit of text being inserted and displayed.

6. The method of claim 3, wherein populating the note taking form with the specific presentation session content comprises:

communicating with a note taking application to input and store the specific presentation session content.

7. The method of claim 1, wherein the receiving specific presentation session content comprises:

receiving a unified resource locator (URL) message; and

obtaining the specific presentation specific content at a URL location, identified by the URL message.

8. A system comprising:

a processing system,

a display,

a communication interface,

one or more computer readable storage media, and

a contextual note taking program stored on the one or more computer readable storage media that, when executed by the processing system, directs the processing system to;

in response to receiving, from a presenter device, a navigation message via the communication interface, change a current assistant context image of a specific presentation session content to a current presenter context image of the specific presentation session content, the current assistant context image corresponding to a context image to which any units of text being received are assigned and the current presenter context image corresponding to a context image a presenter is currently on, and wherein the navigation message comprises a location identifier, the location identifier being used to identify the current presenter context image;

in response to receiving, from the presenter device, a speech-to-text message comprising a unit of text, at least display the unit of text associated with the current presenter context image and store the unit of text associated with the current presenter context image regardless of a context image that an end user is currently viewing in the display of the system.

9. The system of claim 8, further comprising:

a note stored on the one or more computer readable storage media,

wherein the contextual note taking program further directs the processing system to;

populate a note taking form in the note with the specific presentation session content, including a plurality of the identifiable context images.

10. The system of claim 9, wherein the contextual note taking program further directs the processing system to:

identify the current presenter context image in the note taking form and insert the unit of text in the note taking form associated with the current presenter context image.

11. The system of claim 10, wherein the contextual note taking program further directs the processing system to:

in response to receiving user input to the note taking form in a region associated with a different context image than that having the unit of text being inserted, display the user input.

12. The system of claim 8, wherein the contextual note taking program further directs the processing system to:

communicate with a note taking application to input and store the specific presentation session content.

13. The system of claim 12, wherein the contextual note taking program further directs the processing system to:

communicate with the note taking application to input the unit of text associated with the current presenter context image.

14. The system of claim 8, wherein the contextual note taking program further directs the processing system to:

in response to receiving a unified resource locator (URL) message for an indicated specific presentation session, obtain the specific presentation specific content at a URL location, identified by the URL message.

15. The system of claim 8, wherein the contextual note taking program further directs the processing system to:

provide translation options for target languages;

receive an indication of a target language;

as the unit of text associated with the current presenter context image is received, send a request to a translation service, the request including the target language and the unit of text associated with the current presenter context image;

receive a translated unit of text associated with the current presenter context image;

display the translated unit of text associated with the current presenter context image; and

store the translated unit of text associated with the current presenter context image.

16. One or more computer readable storage devices having a contextual note taking program stored thereon that when executed by a processing system, directs the processing system to:

in response to receiving, from a presenter device, a navigation message via a communication interface, change a current assistant context image of a specific presentation session content to a current presenter context image of the specific presentation session content at an end user device, the current assistant context image corresponding to a context image to which any units of text being received are assigned and the current presenter context image corresponding to a context image a presenter is currently on, and wherein the navigation message comprises a location identifier, the location identifier being used to identify the current presenter context image;

in response to receiving, from the presenter device, a speech-to-text message comprising a unit of text, at least display the unit of text associated with the current presenter context image at the end user device and store the unit of text associated with the current presenter context image regardless of a context image that an end user is currently viewing at the end user device.

17. The media of claim 16, wherein the contextual note taking program further directs the processing system to populate a note taking form in a note with the specific presentation session content, including a plurality of the identifiable context images.

18. The media of claim 17, wherein the contextual note taking program further directs the processing system to:

identify the current presenter context image in the note taking form and insert the unit of text in the note taking form associated with the current presenter context image.

19. The media of claim 18, wherein the contextual note taking program further directs the processing system to:

in response to receiving user input to the note taking form in a region associated with a different context image than that having the unit of text being inserted, display the user input.

20. The media of claim 16, wherein the contextual note taking program further directs the processing system to:

communicate with a note taking application to input and store the specific presentation session content.