System and method for using speech for data searching during presentations

• US 9,116,989 B1
• Filed: 12/29/2005
• Issued: 08/25/2015
• Est. Priority Date: 08/19/2005
• Status: Expired due to Fees

• Current Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Gibbon, David C., Gilbert, Mazin, Johnston, Michael, Liu, Zhu, Shahraray, Behzad, Ehlen, Patrick
• Primary Examiner(s): Nguyen, Loan T
• Application Number:
  US11/319,779
• Time in Patent Office: 3,526 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

There is provided for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. A method for the retrieval of information on the basis of its content for real-time incorporation into an electronic presentation is discussed. One method includes controlling a media presentation using a multimodal interface. The method involves receiving from a presenter a content-based request associated with a plurality of segments within a media presentation preprocessed for context-based searching; displaying the media presentation and displaying to the presenter results in response to the content-based request; receiving a selection from the presenter of at least one result; and displaying the selected result to an audience.

Claims

1. A method comprising:

receiving, via a first display device, a spoken content-based free-form natural language query to search content of a plurality of segments within a media presentation that has been processed for content-based searching, the media presentation comprising a series of slides;

displaying, via the first display device, the media presentation, text from a speech recognition process applied to the spoken content-based free-form natural language query, and a scrollable search result set in response to the spoken content-based free-form natural language query, the scrollable search result set comprising a portion of the content of the plurality of segments which is associated with the spoken content-based free-form natural language query, while simultaneously transmitting the media presentation to a second display device for display at the second display device without the text and without the scrollable search result set;

receiving, via the first display device, a selection from the scrollable search result set, to yield a selected segment of the plurality of segments, wherein the selection is based on a motion input; and

transmitting the selected segment to the second display device for display at the second display device as part of the media presentation.

2. The method of claim 1, wherein the search result set is obtained from a network.

3. The method of claim 1, wherein each of the plurality of segments further comprises a string of text.

4. The method of claim 1, wherein the content comprises text associated with a segment of the plurality of segments.

5. The method of claim 4, wherein the text is one of text within the segment, file names, metadata, presenter notes, and words describing non-text content.

6. The method of claim 1, wherein the media presentation is processed by extracting text and using linguistics associated with the text for content-based searching.

7. The method of claim 1, wherein the spoken content-based audible free-form natural language query is a spoken command.

8. The method of claim 7, wherein the spoken command is made distinguishable from other vocalizations of a presenter by way of a speech control initiation.

9. The method of claim 8, wherein an activation action indicates a beginning of a speech search query.

10. The method of claim 9, wherein the activation action is input an activation motion input.

11. The method of claim 1, wherein the media presentation is processed via an n-gram analysis of the plurality of segments.

12. The method of claim 1, wherein the plurality of segments are associated with a deck.

13. The method of claim 1, wherein the content-based searching comprises disambiguation of a homonym associated with the plurality of segments.

14. A system comprising:

a processor; and

a computer-readable medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving a spoken content-based free-form natural language query to search content of a plurality of segments within a media presentation having been processed for content-based searching, the media presentation comprising a series of slides;

displaying, via a first display device, the media presentation, text from a speech recognition process applied to the spoken content-based free-form natural language query, and a scrollable search result set in response to the spoken content-based free-form natural language query, the scrollable search result set comprising a portion of the content of the plurality of segments which is associated with the spoken content-based free-form natural language query, while simultaneously transmitting the media presentation to a second display device for display at the second display device without the text and without the scrollable search result set;

receiving, via the first display device, a selection from the scrollable search result set, to yield a selected segment of the plurality of segments, wherein the selection is based on a motion input; and

transmitting the selected segment to the second display device for display at the second display device as part of the media presentation.

15. A computer-readable device having instructions stored which, when executed by a processor, cause the processor to perform operations comprising:

receiving, at a first display device, a spoken content-based free-form natural language query to search content of a plurality of segments within a media presentation that has been processed for spoken content-based free-form natural language query, the media presentation comprising a series of slides;

displaying, via the first display device, the media presentation, text from a speech recognition process applied to the spoken content-based free-form natural language query, and a scrollable search result set in response to the spoken content-based free-form natural language query, the scrollable search result set comprising a portion of the content of the plurality of segments which is associated with the spoken content-based free-form natural language query, while simultaneously transmitting the media presentation to a second display device for display at the second display device without the text and without the scrollable search result set;

receiving, via the first display device, a selection from the scrollable search result set, to yield a selected segment of the plurality of segments, wherein the selection is based on a motion input; and

transmitting the selected segment to the second display device for display at the second display device as part of the media presentation.

16. The computer-readable device of claim 15, wherein the search result set is obtained from the Internet.

17. The computer-readable device of claim 15, wherein each of the plurality of segments comprises a string of text.

18. The computer-readable device of claim 15, wherein the media presentation is processed via an n-gram analysis of the plurality of segments.

19. The computer-readable device of claim 15, wherein the content-based searching comprises disambiguation of a homonym associated with the plurality of segments.