System and method for multimodal interaction aids

• US 10,237,082 B2
• Filed: 08/31/2012
• Issued: 03/19/2019
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Incorporated
• Inventors: Seligmann, Doree Duncan, Klemm, Reinhard, Yoakum, John H.
• Primary Examiner(s): Chacko, Joe
• Application Number:
  US13/600,817
• Publication Number:
  US 20140067936A1
• Time in Patent Office: 2,391 Days
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for providing multimodal interaction aids in multimedia events. The system presents a multimedia event to a participant which has limited access to the multimedia event, wherein the limited access prevents the participant from experiencing every mode of the multimedia event. The system also generates a cue associated with the multimedia event, wherein the cue is structured according to the limited access of the participant. The system then transmits the cue to a device associated with the participant.

Claims

1. A method comprising:

identifying a plurality of devices participating in a multimedia event having a plurality of different modes of interaction, wherein a first identified device of the plurality of devices has limited access to the plurality of different modes of interaction, and wherein the limited access of the device comprises having access to a first mode of interaction and not having access to a second mode of interaction;

determining access of each device to each mode of the plurality of different modes of interaction;

presenting the multimedia event to the plurality of devices participating in the multimedia event;

generating, based on the determined access of each device of the plurality of devices, a plurality of cues associated with an event within the multimedia event and comprising a first cue in the first mode of interaction and associated with the second mode of interaction and a second cue in the second mode of interaction and associated with the first mode of interaction or the second mode of interaction;

determining a current context of the presented multimedia event and a current context of a participant in the multimedia event associated with each of the plurality of devices, wherein the current context of the multimedia event comprises at least one of a presented order of content in the multimedia event, a delivery of the multimedia event in at least one of the plurality of different modes of interaction, or a pause in delivery of the multimedia event in at least one of the plurality of modes of interaction;

transmitting the generated first cue to the first device based on the determined current context of the presented multimedia event and the determined current context of the participant associated with the first device; and

transmitting the generated second cue to the second device based on the determined current context of the presented multimedia event and the determined current context of the participant associated with the second device.

2. The method of claim 1, wherein the generated first and second cue are embedded in the multimedia event.

3. The method of claim 1, wherein the multimedia event comprises one of a slide presentation, a conference call, a web conference, a video conference, a webcast, a virtual reality display, a graphical display, a gesture, a special effect, a voice, and an online meeting.

4. The method of claim 1, wherein the multimedia event comprises a slide presentation, and wherein the generated first cue associated with the second mode of interaction is triggered by one of a slide opening, a slide closing, a pause, a message, a parameter, a schedule, a transition, and an external event.

5. The method of claim 1, wherein the multimedia event comprises a slide presentation, and wherein the generated first cue associated with the second mode of interaction comprises a slide number that is converted to speech and presented as audio when a respective slide is opened.

6. The method of claim 1, wherein the generated first cue associated with the second mode of interaction is triggered by one of a visual event, an audio event, an external event, an input, an interaction, a tag, a profile, a message, a schedule, a transition, and a parameter.

7. The method of claim 1, wherein the multimedia event comprises a slide presentation, and wherein the generated first cue associated with the second mode of interaction comprises one of a verbal transition announcement, a sound effect, and an audio alert.

8. The method of claim 7, further comprising transmitting the verbal transition announcement to the first device during a pause in a presenter'"'"'s verbal delivery.

9. The method of claim 8, wherein the generated second cue comprises at least one of a video cue, a video enhancement, a textual enhancement, a pop-up, a sound effect, an image, an animation, an alert, an action by an avatar, a message, a number, a symbol, an announcement, or a character.

10. The method of claim 1, further comprising delivering the generated first cue associated with the second mode of interaction to the first device associated with the participant by using one of text, image, video, and audio.

16. The method of claim 1, wherein the generated first cue further comprises an audio effect.

17. The method of claim 1, wherein the multimedia event comprises a presentation, and wherein the generated first cue and second cue are generated and presented based on an analysis of a transition in the presentation.

11. A system comprising:

a processor; and

a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform operations comprising;

identifying a plurality of devices participating in a multimedia event having a plurality of different modes of interaction, wherein a first identified device of the plurality of devices has limited access to the plurality of different modes of interaction, and wherein the limited access of the device comprises having access to a first mode of interaction and not having access to a second mode of interaction;

determining access of each device to each mode of the plurality of different modes of interaction;

presenting the multimedia event to the plurality of devices participating in the multimedia event;

generating, based on the determined access of each device of the plurality of devices, a plurality of cues associated with an event within the multimedia event and comprising a first cue in the first mode of interaction and associated with the second mode of interaction and a second cue in the second mode of interaction and associated with the first mode of interaction or the second mode of interaction;

determining a current context of the presented media event and a current context of a participant in the multimedia event associated with each of the plurality of devices, wherein the current context of the multimedia event comprises at least one of a presented order of content in the multimedia event, a delivery of the multimedia event in at least one of the plurality of different modes of interaction, or a pause in delivery of the multimedia event in at least one of the plurality of modes of interaction;

modifying the first cue based on the determined current context of the presented media event and the determined current context of the participant associated with the first device;

transmitting the modified first cue to the first device; and

transmitting the generated second cue to the second device and the transmitted generated second cue is based on the determined current context of the participant associated with the second device.

12. The system of claim 11, wherein the generated first cue is one of a caption, a visual enhancement, a textual enhancement, a pop-up, an image, an animation, an action by an avatar, an alert, a message, a number, a symbol, an announcement, and a character.

13. A computer-readable storage device storing instructions which, when executed by a processor, cause the processor to perform operations comprising:

identifying a plurality of devices participating in a multimedia event having a plurality of different modes of interaction, wherein a first identified device of the plurality of devices has limited access to the plurality of different modes of interaction, and wherein the limited access of the device comprises having access to a first mode of interaction and not having access to a second mode of interaction;

determining access of each device to each mode of the plurality of different modes of interaction;

presenting the multimedia event to the plurality of devices participating in the multimedia event;

generating, based on the determined access of each device of the plurality of devices, a plurality of cues associated with an event within the multimedia event and comprising a first cue in the first mode of interaction and associated with the second mode of interaction and a second cue in the second mode of interaction and associated with the first mode of interaction or the second mode of interaction;

determining a current context of the presented multimedia event and a current context of a participant in the multimedia event associated with each of the plurality of devices, wherein the current context of the multimedia event comprises at least one of a presented order of content in the multimedia event, a delivery of the multimedia event in at least one of the plurality of different modes of interaction, or a pause in delivery of the multimedia event in at least one of the plurality of modes of interaction;

transmitting the generated first cue to the first device based on the determined current context of the presented multimedia event and the determined current context of the participant associated with the first device;

modifying the generated second cue based on the determined current context of the presented multimedia event and the determined current context of the participant associated with the second device; and

transmitting the modified second cue to the second device.

14. The computer-readable storage device of claim 13, wherein the generated first cue is triggered by one of a visual event, an audio event, an external event, an input, an interaction, a tag, a profile, a message, a schedule, a transition, and a parameter.

15. The computer-readable storage device of claim 13, wherein the generated first cue comprises one of a pop-up, a sound effect, an action by an avatar, an alert, a message, a number, a symbol, an announcement, and a character.