Synchronization control of multimedia objects in an MHEG engine

• US 5,748,187 A
• Filed: 11/22/1995
• Issued: 05/05/1998
• Est. Priority Date: 12/13/1994
• Status: Expired due to Term

• Current Assignee: Electronics and Telecommunications Research Institute, Korea Telecommunication Authority
• Original Assignee: Electronics and Telecommunications Research Institute, Korea Telecommunication Authority
• Inventors: Park, Sung-Yul, Jin, Byeong-Won, Cho, Sung-Been, Kim, Jin-Suk, Lee, Hae-Won
• Primary Examiner(s): Burwell, Joseph R.
• Application Number:
  US08/562,190
• Time in Patent Office: 895 Days
• Field of Search: 395/806, 395/807, 395/762, 345/302
• US Class Current: 715/201
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs

Abstract

A method and apparatus for synchronization control of multimedia objects in an MHEG engine are specified. The present system overcomes the shortcomings in the conventional apparatus, which is the fact that interactive processing is not allowed during presentation and that the conventional apparatus cannot control synchronized presentation for failing to provide control of the conditional, temporal, and spatial synchronization. The present apparatus has a user interface for receiving a user input during multimedia presentation whereby interactive processing can be supported, a time token processor for generating a time token event according to the times set by the time token timer, a link processor for transferring a list of action commands associated with the satisfied conditional clauses and a list of the MHEG object information to the action processor with reference to the status table and the link table upon generation of various events, an action processor for generating the action event and for creating a plurality of presentation information units to be presented for each of a plurality of channels to execute the actions received from the link processor, and a plurality of channels for transferring the corresponding frame information to the presentation server according to the times set in the channel timer.

Claims

1. An MHEG engine for presenting multimedia information in a synchronized manner, comprising:

an ASN.1 decoder for reading data stream including encoded MHEG object information from a storage device to convert it into the structure defined in ASN.1;

a mapping processor for converting the decoded data into the internal data structure to be used within said MHEG engine to store it in an MHEG class library and for creating a look-up table so that multimedia data used in an action processor can be accessed in real time;

an MHEG class library including the MHEG object information converted into the internal data structure;

an element table including an address of another MHEG object element which is included in the MHEG object and related to represented presentation;

a status table for representing the process status of the MHEG object to be presented;

a link table including conditional clauses of the MHEG objects which can be presented and addresses of the MHEG objects associated therewith;

a look-up table for linking relative addresses used in the action processor with physical addresses of locations where physical information objects are stored;

a synchronization controller for checking conditional clauses in the link table and for controlling synchronization of the MHEG object information by a list of the MHEG object information with the conditional clause associated therewith being satisfied and related action commands to be executed;

a presentation server for receiving presentation information from said synchronization controller to present multimedia information; and

an action process module for storing program modules to be executed by the action commands,

wherein said synchronization controller includes

a user interface for receiving a user input during multimedia presentation whereby an interactive processing can be supported and for generating an interaction event;

a time token processor for changing a time stone status within the status table at a channel start time previously stored, and for generating a time token event;

a link processor for checking whether the conditional clauses in the link table are satisfied with reference to values in the status table, upon generation of an action event, which is generated after execution of an action behavior, the time token event, or the interaction event, and for transferring a list of action commands associated with the satisfied conditional clauses and a list of the MHEG object information to the action processor;

an action processor for generating the action event after execution of the action behavior in said link processor, writing the channel start time in said time token processor to execute actions received from said link processor, and creating a plurality of presentation information units to be presented for each of a plurality of channels, each channel having multimedia information; and

a plurality of channels for transferring the corresponding frame information to said presentation server at the frame start time.

2. The MHEG engine of claim 1, wherein said time token processor is controlled by a time token timer with channel driving times being preset, each channel corresponding to different media information.

3. The MHEG engine of claim 1, wherein each of said plurality of channels has therein channel timers for transferring the corresponding frame information to said presentation server according to the preset times.

4. The MHEG engine of claim 1, wherein said presentation information units include frame related information which is a multimedia data presented by said presentation server.

5. The MHEG engine of claim 4, wherein said presentation information units include Frame Address (FA), Frame Start Time (FST), Frame End Time (FET), Frame Space (FS), Frame Volume (FV), and Frame Presentation Speed (FPS).

6. A method for synchronization control of multimedia objects in a MHEG engine for presenting multimedia information and which comprises a status table for representing the process status of MHEG objects for presentation of multimedia information, a link table for storing conditional clauses which enable the MHEG objects to be presented and addresses of the MHEG objects associated therewith, a presentation server for receiving presentation information from a synchronization controller to present multimedia information, said method comprising the steps of:

checking the conditional clauses in said link table to retrieve an action behavior to be subsequently executed and transferring the retrieved action behavior to an action processor, upon occurrence of an event, when the synchronization controller starts its operation for presentation of multimedia in the MHEG engine,

wherein said step of checking the conditional clauses includes the steps of;

starting the operation of the synchronization controller when the MHEG engine is ready to provide the initial presentation (S1);

checking whether at least one of an action event, an interaction event, and a time token event is generated as the synchronization controller starts its operation (S2);

changing the time token status in the status table and executing said checking step (S2) again after waiting until the time token event is generated by a time token processor, if no event is generated (S3);

checking whether there is any satisfied conditional clause of the conditional clauses in the link table, if at least one event is generated (S4); and

executing said checking step (S2) again if there is no satisfied conditional clause, and initializing a parameter "i" to 1 if there is a satisfied conditional clause (S5);

synchronizing multimedia for each of associated channels by executing the action behavior transferred by said checking step; and

presenting multimedia information by transferring them to said presentation server in sequence based on synchronization information on frames within the channels inputted by said synchronizing step.

7. The method of claim 6, wherein said synchronizing step includes the steps of:

checking whether the ith action in the action list is null (S6);

generating the action event, changing the action status in the status table, and then executing said checking step (S2) again when the result of the check at step S6 is affirmative (S7);

checking whether the ith action in the action list is the action which sets the frame information in channels, if the ith action is not null (S8);

setting the synchronization information of the frame related to the corresponding action into the corresponding channel, and changing the action status in the status table, if the ith action is the channel information setting action (S9);

checking whether the ith action in the action list is the action which sets the channel start time in the time token (TT), if the ith action is not the channel information setting action (S10);

setting the corresponding channel start time information into the time token, and changing the action status in the status table, if the ith action is the time token information setting action (S11);

checking whether the ith action in the action list is a run action, if the ith action is not the time token information setting action (S12);

driving the corresponding channel, if the ith action is the run action (S13);

changing the action status in the status table (S14);

checking whether the ith action is a stop action, if the ith action is not the run action (S15);

stopping the presentation of the corresponding frame in the corresponding channel and changing the action status in the status table, if the ith action is the stop action (S16);

incrementing the parameter i by 1 and repeating from said step S6, after execution of steps S9, S11, S14, and S16 (S17);

checking whether the ith action in the action list is a prepare action, if the ith action is not the stop action (S18);

executing the error processing routine and terminating the operation of the MHEG engine, if the ith action is not the prepare action (S19);

inputting the MHEG object from the storage device into the MHEG engine, storing information obtained by analyzing the objects into the element table, link table, and look-up table, changing the action status in the status table, generating the action event, and executing said checking step (S2), if the ith action is the prepare action (S20); and

changing the interaction status in the status table, generating the interaction event, and executing said checking step (S2) whereby a user command can be processed, when the user inputs the user command for the purpose of changing the presentation status (S21).

8. The method of claim 6, wherein said presenting step includes the steps of:

storing the number of presentation information units (PIU) in the channel into the parameter j, and displaying the frame start time (FST) of the presentation information units (PIU) in each channel onto the channel timer (CT), according to the corresponding channel driving command at said driving step (S13) (S1302);

checking whether the parameter j is 0 (S1303);

clearing the channel, if the parameter j is 0 (S1304);

terminating the driving of the corresponding channel (S1305);

checking by the use of the channel timer (CT) whether the parameter j is the frame start time (FST), if the parameter j is not 0 (S1306);

transferring the corresponding frame information to the presentation server so as to initiate the presentation after waiting until the parameter j becomes the frame start time (FST), if the parameter j is not the frame start time (FST) (S1307); and

decrementing the parameter j by 1 and repeating from said step S1303 (S1308).