System and method for using speech for data searching during presentations

• US 9,489,432 B2
• Filed: 08/24/2015
• Issued: 11/08/2016
• Est. Priority Date: 08/19/2005
• Status: Active Grant

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Ehlen, Patrick, Gibbon, David Crawford, Gilbert, Mazin, Johnston, Michael, Liu, Zhu, Shahraray, Behzad
• Primary Examiner(s): Nguyen, Loan T
• Application Number:
  US14/833,645
• Publication Number:
  US 20150379094A1
• Time in Patent Office: 442 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

There is provided for a system, method, and computer readable medium storing instructions related to controlling a presentation in a multimodal system. A method for the retrieval of information on the basis of its content for real-time incorporation into an electronic presentation is discussed. One method includes controlling a media presentation using a multimodal interface. The method involves receiving from a presenter a content-based request associated with a plurality of segments within a media presentation preprocessed for context-based searching; displaying the media presentation and displaying to the presenter results in response to the content-based request; receiving a selection from the presenter of at least one result; and displaying the selected result to an audience.

Claims

1. A method comprising:

transmitting, from a first device having a processor and a first display, a current segment of a media presentation to a second device, the second device having a second display;

receiving, at the first device, a spoken query to search a plurality of segments within the media presentation;

displaying, on the first display of the first device, the current segment of the media presentation and a search result set in response to the spoken query, the search result set comprising a portion of the plurality of segments, the search result set not being transmitted to the second device;

receiving, via motion input from a user, a selection from the search result set, to yield a selected segment; and

transmitting the selected segment to the second device for display, the selected segment replacing the current segment being displayed on the second display of the second device.

2. The method of claim 1, wherein the search result set is obtained from a network.

3. The method of claim 1, wherein each of the plurality of segments further comprises a string of text.

4. The method of claim 1, wherein the spoken query is associated with content comprising text associated with a segment of the plurality of segments.

5. The method of claim 4, wherein the text is one of text within the segment, file names, metadata, presenter notes, and words describing non-text content.

6. The method of claim 1, wherein the media presentation is processed by extracting text and using linguistics associated with the text for content-based searching.

7. The method of claim 1, wherein the spoken query is a spoken command.

8. The method of claim 7, wherein the spoken command is made distinguishable from other vocalizations of a presenter by way of a speech control initiation.

9. The method of claim 8, wherein an activation action indicates a beginning of a speech search query.

10. The method of claim 9, wherein the activation action is an activation motion input.

11. The method of claim 1, wherein the media presentation is processed via an n-gram analysis of the plurality of segments.

12. The method of claim 1, wherein the plurality of segments is associated with a deck.

13. A system comprising:

a processor;

a display; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

transmitting a current segment of a media presentation from a first device having a first display to a second device, the second device having a second display;

receiving a spoken query to search a plurality of segments within the media presentation;

displaying on the first display the current segment of the media presentation and a search result set in response to the spoken query, the search result set comprising a portion of the plurality of segments, the search result set not being transmitted to the second device;

receiving, via motion input from a user, a selection from the search result set, to yield a selected segment; and

transmitting the selected segment to the second device for display, the selected segment replacing the current segment being displayed on the second display of the second device.

14. The system of claim 13, wherein the search result set is obtained from a network.

15. The system of claim 13, wherein each of the plurality of segments further comprises a string of text.

16. The system of claim 13, wherein the spoken query is associated with content comprising text associated with a segment of the plurality of segments.

17. The system of claim 16, wherein the text is one of text within the segment, file names, metadata, presenter notes, and words describing non-text content.

18. The system of claim 13, wherein the media presentation is processed by extracting text and using linguistics associated with the text for content-based searching.

19. The system of claim 13, wherein the spoken query is a spoken command.

20. A non-transitory computer-readable storage device having instructions stored which, when executed by a first device having a first display, cause the first device to perform operations comprising:

transmitting a current segment of a media presentation to a second device, the second device having a second display;

receiving a spoken query to search a plurality of segments within the media presentation;

displaying, on the first display of the first device, the current segment of the media presentation and a search result set in response to the spoken query, the search result set comprising a portion of the plurality of segments, the search result set not being transmitted to the second device;

receiving, via motion input from a user, a selection from the search result set, to yield a selected segment; and

transmitting the selected segment to the second device for display, the selected segment replacing the current segment being displayed on the second display of the second device.