Synchronization mechanism for multimedia captioning and audio description

• US 20040128702A1
• Filed: 12/26/2002
• Published: 07/01/2004
• Est. Priority Date: 12/26/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kaneko, Masahiko
• Granted Patent: US 7,631,326 B2
• Time in Patent Office: Days
• US Class Current: 725/137
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 7/08 Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band , e.g. by time division H04N7/007 takes precedence
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles
• CPC Class Codes: H04N 7/165 Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

A method for outputting supplemental media streams with a main media stream is presented. The method monitors for triggering events and, upon detecting a triggering event, causes one or more supplemental media streams associated with the triggering event to output. The associated supplemental media streams may be output either synchronously or asynchronously with the main media stream. Synchronous supplemental media streams are output concurrently with the main media stream. Asynchronously output supplemental media streams suspend the main media stream while they are output and terminate any currently outputting supplemental media streams. A supplemental media stream may cause a conditional triggering event to trigger a second supplemental media stream to output synchronously with the first. A second main media stream with associated supplemental media streams may be output asynchronously as a compiled supplemental media stream in connection with a first main media stream.

Claims

1. A method for outputting one or more supplemental media streams designed to assist sense impaired people in conjunction with a main media stream, comprising:

repeatedly monitoring for and detecting triggering events while outputting the main media stream; and

upon detecting a triggering event;

determining which of the one or more supplemental media streams are associated with the triggering event;

determining for each associated supplemental media stream whether each associated supplemental media stream is to be output synchronously or asynchronously with the main media stream; and

iteratively outputting each associated supplemental media stream either synchronously or asynchronously with the main media stream according to the results of the previous determinations.

2. The method of claim 1, wherein the associated supplemental media stream are iteratively output according to a predetermined order among the associated supplemental media streams.

3. The method of claim 2, wherein the predetermined order among associated supplemental media streams is established according to priority values associated with each supplemental media stream.

4. The method of claim 1, further comprising suspending the main media stream while an associated supplemental media stream is output asynchronously.

5. The method of claim 4, further comprising terminating a second supplemental media stream currently being output when the associated supplemental media stream is output asynchronously.

6. The method of claim 4, further comprising resuming output of the main media stream after asynchronously outputting the associated supplemental media stream.

7. The method of claim 6, wherein the triggering events occur according to the timing of the main media stream.

8. The method of claim 7, wherein the timing of the main media stream is suspended when the main media stream is suspended.

9. The method of claim 1, wherein a first associated supplemental media stream iteratively causes a second associated supplemental media stream to output when the first associated supplemental media stream terminates its output.

10. The method of claim 1, wherein an associated supplemental media stream issues a conditional triggering event that causes a conditional supplemental media stream to output synchronously with the associated supplemental media stream.

11. The method of claim 10, wherein the associated supplemental media stream selectively issues the conditional triggering event according to a condition external to the associated supplemental media stream.

12. The method of claim 1, wherein determining which of the one or more supplemental media streams are associated with the triggering event comprises selectively excluding, according to an external condition, a supplemental media stream that would otherwise be associated with the triggering event.

13. The method of claim 12, wherein the external condition identifies the type of supplemental media stream to be selectively excluded.

14. The method of claim 1, wherein a supplemental media stream comprises either an audio description stream, a visual description stream, or a compiled media stream.

15. The method of claim 14, wherein a compiled media stream comprises a compiled main media stream and one or more compiled supplemental media streams associated with the compiled media stream.

16. The method of claim 15, wherein outputting a compiled media stream comprises:

repeatedly monitoring for and detecting compiled triggering events while outputting the compiled main media stream; and

upon detecting a compiled triggering event;

determining which of the one or more compiled supplemental media streams are associated with the compiled triggering event;

determining for each associated compiled supplemental media stream whether each associated compiled supplemental media stream is to be output synchronously or asynchronously with the compiled main media stream; and

iteratively outputting each associated compiled supplemental media stream either synchronously or asynchronously with the compiled main media stream in accordance with the results of the previous determinations.

17. The method of claim 16, wherein a compiled supplemental media stream comprises either an audio description stream, a visual description stream, or a compiled media stream.

18. The method of claim 16, wherein the compiled triggering events occur according to the timing of the compiled main media stream.

19. A computer readable medium having computer executable instructions for carrying out the method comprising the steps of:

repeatedly monitoring for and detecting triggering events while outputting the main media stream; and

upon detecting a triggering event;

determining which of the one or more supplemental media streams are associated with the triggering event;

determining for each associated supplemental media stream whether each associated supplemental media stream is to be output synchronously or asynchronously with the main media stream; and

iteratively outputting each associated supplemental media stream either synchronously or asynchronously with the main media stream according to the results of the previous determinations.

20. The computer readable medium of claim 19, wherein a supplemental media stream comprises either an audio description stream, a visual description stream, or a compiled media stream, a compiled media stream comprising a compiled main media stream and one or more compiled supplemental media streams associated with the compiled media stream, and outputting a compiled media stream comprises:

repeatedly monitoring for and detecting compiled triggering events while outputting the compiled main media stream; and

upon detecting a compiled triggering event;

determining which of the one or more compiled supplemental media streams are associated with the compiled triggering event;

determining for each associated compiled supplemental media stream whether each associated compiled supplemental media stream is to be output synchronously or asynchronously with the compiled main media stream; and

iteratively outputting each associated compiled supplemental media stream either synchronously or asynchronously with the compiled main media stream in accordance with the results of the previous determinations.

21. A system for outputting at least one supplemental media stream designed to assist sense impaired people in conjunction with a main media stream, the system comprising:

a media stream output device for outputting a main media stream and at least one supplemental media stream; and

a media stream output controller communicatively coupled to the media stream output device for;

receiving a main media stream and at least one supplemental media stream;

outputting the main media stream via the media stream output device;

monitoring for and detecting triggering events associated with the output of the main media stream; and

upon detecting a triggering event;

determining which of the at least one supplemental media streams are associated with the detected triggering event;

determining for each associated supplemental media stream whether each associated supplemental media stream is to be output synchronously or asynchronously with the main media stream; and

iteratively outputting each associated supplemental media stream, via the media stream output device, either synchronously or asynchronously with the main media stream in accordance with the results of the previous determinations.

22. The system of claim 21, wherein the associated supplemental media streams are iteratively output according to a predetermined order among the associated supplemental media streams.

23. The system of claim 22, wherein the predetermined order among the associated supplemental media streams is established according to priority values assigned to each supplemental media stream.

24. The system of claim 21, wherein the media stream output controller is further for suspending the main media stream while an associated supplemental media stream is output asynchronously.

25. The system of claim 24, wherein the media stream output controller is further for terminating a second supplemental media stream currently being output when the associated supplemental media stream is output asynchronously.

26. The system of claim 24, wherein the media stream output controller is further for resuming output of the main media stream after asynchronously outputting the associated supplemental media stream.

27. The system of claim 21, wherein a first associated supplemental media stream iteratively causes a second associated supplemental media stream to output when the first associated supplemental media stream terminates its output.

28. The system of claim 21, wherein an associated supplemental media stream issues a conditional triggering event that causes a conditional supplemental media stream to output synchronously with the associated supplemental media stream.

29. The system of claim 28, wherein the associated supplemental media stream selectively issues the conditional triggering event according to a condition external to the associated supplemental media stream.

30. The system of claim 21, wherein a supplemental media stream comprises an audio description stream, a visual description stream, or a compiled media stream.

31. The system of claim 30, wherein a compiled media stream comprises a compiled main media stream and at least one compiled supplemental media stream associated with the compiled media stream.

32. A media display system for outputting one or more supplemental media streams designed to assist sense impaired people in conjunction with outputting a main media stream, the system comprising:

a monitoring means for monitoring and detecting triggering events associated while outputting the main media stream;

a supplemental media stream selection means for selecting which of the one or more supplemental media streams are associated with a detected triggering event;

a determination means for determining for each associated supplemental media stream whether each associated supplemental media stream is to be output synchronously or asynchronously with the main media stream; and

a control means for iteratively outputting each associated supplemental media stream either synchronously or asynchronously with the main media stream according to the results of the determinations by the determination means, terminating any currently outputting supplemental media streams when an associated supplemental media stream is output asynchronously, and suspending the main media stream while the associated supplemental media stream is output asynchronously.