System and method for creating a presentation using natural language

• US 9,026,915 B1
• Filed: 10/31/2005
• Issued: 05/05/2015
• Est. Priority Date: 10/31/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Ehlen, Patrick, Gibbon, David Crawford, Gilbert, Mazin, Johnston, Michael, Liu, Zhu, Shahraray, Behzad
• Primary Examiner(s): Tran, Quoc A
• Application Number:
  US11/263,051
• Time in Patent Office: 3,473 Days
• Field of Search: 707/100, 707/500, 707/513, 709/204, 709/217, 709/218, 715/200, 715/201, 715/224, 715/234, 715/236, 715/255, 715/256, 715/500, 715/760, 715/202, 715/862, 715/730, 715/732, 715/273, 715705-721
• US Class Current: 715/728
• CPC Class Codes: G06F 16/2445 Data retrieval commands; View definitions
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: H04N 5/74 Projection arrangements for image reproduction, e.g. using eidophor

Abstract

The invention provides for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. The method embodiment of the invention is a method for the retrieval of information on the basis of its content for incorporation into an electronic presentation. The method comprises receiving from a user a content-based request for at least one segment from a first plurality of segments within a media presentation preprocessed to enable natural language content searchability; in response to the request, presenting a subset of the first plurality of segments to the user; receiving a selection indication from the user associated with at least one segment of the subset of the first plurality of segments and adding the selected at least one segment to a deck for use in a presentation.

Claims

1. A method comprising:

at a first time, prior to a presentation on a presentation device;

receiving, on the presentation device, a grammar which assigns semantic representations to natural language input; and

updating the grammar based on a searchable plurality of presentation source files stored on a storage device connected to the presentation device via a network, to yield an updated grammar, wherein the searchable plurality of presentation source files comprise one of audio clips and video clips, wherein the searchable plurality of presentation source files is processed by extracting content from each searchable presentation source file to generate

(1) a web-based form of each searchable presentation source file;

(2) a listing of key phrases; and

(3) a corpus of all sentences that appear in all the searchable presentation source files; and

at a second time after the first time, during the presentation, and on the presentation device;

receiving, from a user, a spoken natural language content request for a presentation source file from the searchable plurality of presentation source files;

identifying possible presentation source files in the searchable plurality of presentation source files, wherein the possible presentation source files are identified by using the updated grammar to process the spoken natural language content request;

receiving, from the user, input identifying the presentation source file from the possible presentation source files, wherein the input is in a modality distinct from speech; and

adding the presentation source file to a deck for use in the presentation.

2. The method of claim 1, wherein the presentation source file is confirmed by the user via one of graffiti, handwritten, typed, motion, and button click input.

3. The method of claim 1, wherein the presentation source files are viewgraphs.

4. The method of claim 1, wherein the list of keyphrases is used to populate a menu available to the user.

5. The method of claim 4, wherein the user can use the menu of key phrases to select searchable presentation source files.

6. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

at a first time, prior to a presentation on a presentation device;

receiving, on the presentation device, a grammar which assigns semantic representations to natural language input; and

updating the grammar based on searchable plurality of presentation source files stored on a storage device connected to the presentation device via a network, to yield an updated grammar, wherein the searchable plurality of presentation source files comprise one of audio clips and video clips, wherein the searchable plurality of presentation source files is processed by extracting content from each searchable presentation source file to generate

(1) a web-based form of each searchable presentation source file;

(2) a listing of key phrases; and

(3) a corpus of all sentences that appear in all the searchable presentation source files; and

at a second time after the first time, during the presentation, and on the presentation device;

receiving, from a user, a spoken natural language content request for a presentation source file from the searchable plurality of presentation source files;

identifying possible presentation source files in the searchable plurality of presentation source files, wherein the possible presentation source files are identified by using the updated grammar to process the spoken natural language content request;

receiving, from the user, input identifying the presentation source file from the possible presentation source files, wherein the input is in a modality distinct from speech; and

adding the presentation source file to a deck for use in the presentation.

7. The system of claim 6, wherein presentation source file is confirmed by the user via one of graffiti, typed, motion, and button click input.

8. The system of claim 6, wherein the presentation source files are viewgraphs.

9. The system of claim 6, wherein the list of keyphrases is used to populate a menu available to the user.

10. The system of claim 9, wherein the user can use the menu of key phrases to select searchable presentation source files.

11. A computer-readable storage medium having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

at a first time, prior to a presentation on a presentation device;

receiving, on the presentation device, a grammar which assigns semantic representations to natural language input; and

updating the grammar based on searchable plurality of presentation source files stored on a storage device connected to the presentation device via a network, to yield an updated grammar, wherein the searchable plurality of presentation source files comprise one of audio clips and video clips, wherein the searchable plurality of presentation source files is processed by extracting content from each searchable presentation source file to generate

(1) a web-based form of each searchable presentation source file;

(2) a listing of key phrases; and

(3) a corpus of all sentences that appear in all the searchable presentation source files; and

at a second time after the first time, during the presentation, and on the presentation device;

receiving, from a user, a spoken natural language content request for a presentation source file from the searchable plurality of presentation source files;

identifying possible presentation source files in the searchable plurality of presentation source files, wherein the possible presentation source files are identified by using the updated grammar to process the spoken natural language content request;

receiving, from the user, input identifying the presentation source file from the possible presentation source files, wherein the input is in a modality distinct from speech; and

adding the presentation source file to a deck for use in the presentation.