Multimedia control system

• US 5,680,639 A
• Filed: 05/10/1993
• Issued: 10/21/1997
• Est. Priority Date: 05/10/1993
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Object Technology Licensing Corporation
• Inventors: Tindell, James Michael, Milne, Steve H.
• Primary Examiner(s): Feild, Joseph H.
• Application Number:
  US08/059,483
• Time in Patent Office: 1,625 Days
• Field of Search: 395/154, 395/550, 395/155, 395/159, 395/161, 395/600, 395/806, 395/807, 395/326, 395/348-349, 395/339, 395/601, 395/173, 395/551, 395/556, 395/614, 395/615
• US Class Current: 715/202
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A method and system for controlling various multimedia objects throughout the course of a multimedia presentation on a computer with a storage and a display utilizes a control object defined in the storage and associated with each multimedia object. The control objects each contain an internal current time value which is used to control presentation of multimedia data by the associated multimedia object. The internal current time value is calculated as a function of an input time value and control objects can be connected together so that the internal current time value of one clock object serves as the input time value of another clock object so that multimedia objects can be synchronized. Each control object can also be directly manipulated via a mouse or other pointing device to directly control the associated multimedia object to position the object to a particular position in time, or adjust the playback rate.

Claims

1. A computer system for presenting a first multimedia time-based media sequence synchronized with a second multimedia time-based media sequence in response to a presentation request, comprising:

(a) a storage;

(b) a presentation mechanism;

(c) first and second control objects, each of the first and second control objects being resident in the storage and including methods for receiving an input value and methods responsive to the input value for calculating a current time value as a direct function of the input value determined from an offset value, a rate value and an input time value resident in the storage;

(d) first and second multimedia objects resident in the storage, the first multimedia object including methods responsive to the current time value of the first control object for presenting the first multimedia time-based media sequence at a first playback position on the presentation mechanism, the second multimedia object including methods responsive to the current time value of the second control object for presenting the second multimedia time-based media sequence at a second playback position on the presentation mechanism;

(e) means responsive to a connection request for providing the current time value of the first control object as an input value to the second control object so that the second playback position is a direct function of the first playback position; and

(f) a time source for providing a sequence of input values to the first control object to cause the first multimedia time-based media sequence to be presented in lockstep with the second multimedia time-based media sequence.

2. The computer system as recited in claim 1, including means for forcing the current time maintained in each of the first and the second control objects to proceed backward based on adjustments of the first and the second control objects.

3. The computer system as recited in claim 1, wherein the first multimedia object is located in a first address space and the computer system comprises a third multimedia object including means responsive to changes of the current time value of the first control object for presenting multimedia data on the presentation device, the third multimedia object being located in a second address space different from the first address space.

4. The computer system as recited in claim 1, including means for linking the first control object to the first multimedia object via a geometric figure joining the first control and the first multimedia objects on the display.

5. The computer system as recited in claim 4, wherein the geometric figure is a line segment.

6. The computer system as recited in claim 1, wherein each of the first and the second control objects include:

means for generating a wakeup request containing a wakeup time;

means for receiving wakeup requests generated by other control objects;

means for placing received wakeup times in an ordered sequence on a wakeup list;

means responsive to the current time of the each control object and to wakeup times on the wakeup list for generating a detection signal when the current time value of the each control object corresponds to a wakeup time on the wakeup list, and

means responsive to the detection signal for firing a wakeup message to a control object that transmitted a wakeup request containing the wakeup time.

7. The computer system as recited in claim 6, wherein the first control object includes:

means for generating a first timebase;

means for inserting the first timebase into a wakeup request and sending the wakeup request to the second control object;

and wherein the second control object includes;

means for generating a second timebase; and

means for converting the first timebase in a received wakeup request to the second timebase so that the first control object maintains its own internal timebase and sends wakeup requests to the second control object in the first timebase.

8. The computer system as recited in claim 7 wherein the means for converting comprises:

means for obtaining the function used by the first control object to maintain its current time value from the first input time value;

means responsive to the obtained function for creating an inverse function; and

means responsive to the inverse function for converting the wakeup time in the received wakeup request to the second timebase.

9. The computer system as recited in claim 6 wherein the first control object sends a delay wakeup request to the second control object, and wherein the first control object includes means responsive to the sending of the delay wakeup request for blocking further execution of the first multimedia object;

and means responsive to a wakeup request received from the second control object for continuing further execution of the first multimedia object.

10. The computer system as recited in claim 6 wherein the first control object sends an alarm wakeup request to the second control object, and wherein the first control object includes means responsive to the sending of the alarm wakeup request for continuing further execution of the first multimedia object.

11. The computer system as recited in claim 1, wherein the first control object further includes means for adjusting its rate value and its offset value to change the presentation of the first multimedia data without affecting the presentation of the second multimedia data.

12. A method for synchronizing a first multimedia presentation of a first multimedia time-based media sequence to a second multimedia presentation of a second multimedia time-based media sequence on a computer having a storage and a presentation mechanism, the method comprising the steps of:

(a) creating a first control object in the storage, the first control object having methods for receiving a first input time value, for calculating a current time value as a direct function of the first input time value;

(b) creating a second control object in the storage, the second control object having methods for receiving a second input time value and for calculating a current time value as a direct function of the second input time value;

(c) creating a first multimedia object resident in the storage, the first multimedia object including methods for presenting the first multimedia time-based media sequence at a first playback position on the presentation mechanism determined by the current time value of the first control object

(d) creating a second multimedia object resident in the storage, the second multimedia object including methods for presenting the second multimedia time-based media sequence at a second playback position on the presentation mechanism determined by the current time value of the second control object;

(e) providing the current time value of the first control object as the second input value to the second control object so that the second playback position is a direct function of the first playback position; and

(f) providing a sequence of input values to the first control object to cause the first multimedia time-based media sequence to be presented in lockstep with the second multimedia time-based media sequence.

13. The method as recited in claim 12, wherein the first multimedia object is located in a first address space and the method further comprises the step of:

(f) creating a third multimedia object including means responsive to changes of the current time value of the first control object for presenting multimedia data on the presentation device, the third multimedia object being located in a second address space different from the first address space.

14. The method as recited in claim 13, including the step of:

(g) linking the first control object to the first multimedia object via a geometric figure joining the first control and the first multimedia objects on the display to associate the first control object with the first multimedia object.

15. The method as recited in claim 14, wherein the geometric figure is a line segment.

16. The method as recited in claim 12, wherein step (a) includes the steps of:

(a1) creating a first control object having methods for generating a wakeup request containing a wakeup time; and

wherein the method further comprises the steps of;

(h) receiving wakeup requests generated by the second control object;

(i) placing received wakeup times in an ordered sequence on a wakeup list;

(j) detecting when the current time value of the first control object corresponds to a wakeup time on the wakeup list, and

(k) firing a wakeup message to the second control object when a detection is accomplished in step (j).

17. The method as recited in claim 16 wherein the method further includes the steps of:

(l) generating a first timebase;

(m) inserting the first timebase into a wakeup request and sending the wakeup request to the second control object;

(n) generating a second timebase; and

(o) converting the first timebase in a received wakeup request to the second timebase so that the first control object maintains its own internal timebase and sends wakeup requests to the second control object in the first timebase.

18. The method as recited in claim 17 wherein step (o) includes the steps of:

(o1) obtaining the function used by the first control object to maintain its current time value from the first input time value;

(o2) creating an inverse function from the obtained function; and

(o3) using the inverse function to convert the wakeup time in the received wakeup request to the second timebase.

19. The method as recited in claim 16 further comprising the steps of:

(p) sending a delay wakeup request from the first control object to the second control object;

(q) blocking further execution of the first multimedia object in response to the delay wakeup request;

(r) receiving a wakeup request from the second control object; and

(s) continuing further execution of the first multimedia object in response to the wakeup request received from the second control object.

20. The method as recited in claim 16 further comprising the steps of:

(t) sending an alarm wakeup request from the first control object to the second control object; and

(u) continuing further execution of the first multimedia object in response to the alarm wakeup request.

21. A control framework for controlling time relationships in a multimedia application comprised of a first multimedia object for presenting first multimedia data and a second multimedia object for presenting second multimedia data, the control framework comprising:

a storage;

data defining an application-subclassable clock class stored in the storage, the clock class having an input time value attribute, a current time attribute, an offset value attribute and a rate value attribute and at least one virtual member function for calculating a value for the current time attribute from the input time value attribute, the offset value attribute and the rate value attribute;

means controlled by the multimedia application for instantiating a first clock object from the clock class;

means controlled by the multimedia application for instantiating a second clock object from the clock class;

means controlled by the multimedia application for associating the first multimedia object to the first clock object so that the first multimedia object presents the first multimedia data in response to changes in the current time attribute of the first clock object;

means controlled by the multimedia application for associating the second multimedia object to the second clock object so that the second multimedia object presents the second multimedia data in response to changes in the current time attribute of the second clock object;

means controlled by the multimedia application for setting the value of the input time value attribute in the second clock object to the value of the current time attribute in the first clock object so that presentation of the second multimedia data is synchronized to presentation of the first multimedia data.

22. The control framework as recited in claim 21 wherein the framework further comprises:

data defining a driven-time clock subclass stored in the storage, the driven-time subclass derived from the clock class and having;

a member function for setting the value of the current time attribute to the value of the input time attribute

a wakeup request list structure;

a member function for generating a wakeup request comprising a wakeup time and transmitting the wakeup request to a clock object;

a member function for receiving wakeup requests from clock objects and for inserting the wakeup time and information identifying the clock object that generated the wakeup request into the wakeup request list structure;

a member function for comparing the value of the current time attribute to each wakeup time in the wakeup request list structure and generating a wakeup signal when the value of the current time attribute equals a wakeup time stored in the list; and

a member function which responds to the wakeup signal by firing a wakeup message to a clock object identified by information stored in the list structure.

23. The control framework as recited in claim 22 further comprising data for defining a time source class stored in the storage, the time source class having a time value attribute, means for generating a time value for the time value attribute and a member function for transmitting the value of the time value attribute to a clock object.

24. The control framework as recited in claim 23 further comprising means controlled by the multimedia application for instantiating a time source object from the time source class wherein the time value attribute of the time source object, and the offset value, the rate value and the virtual member calculating function of the first and second clock objects are maintained in a shared region of the storage and protected by a semaphore to insure that the time value attribute of the time source object, and the offset value, the rate value and the virtual member calculating function of the first and second clock objects are not changed when they are accessed.

25. The control framework as recited in claim 23 wherein time value generating means comprises a system timer.

26. The control framework as recited in claim 21 wherein the time value generating means comprises means responsive to the first multimedia data for extracting a time value signal.

27. The control framework as recited in claim 21 further including data defining a jog/shuttle knob subclass stored in the storage, the jog/shuttle knob subclass being derived from the clock class and having a member function for displaying on a presentation device a representative indicia of a jog/shuttle knob, a member function responsive to user manipulations of the representative indicia for adjusting the rate value attribute and the offset value attribute.

28. A computer program product operable on a computer system having a storage and a presentation mechanism for presenting a first multimedia time-based media sequence synchronized with a second multimedia time-based media sequence in response to a presentation request, comprising a computer usable medium having computer readable program code thereon, the computer readable program code including:

(a) program code for creating first and second control objects in the storage, each of the first and second control objects including methods for receiving an input value and methods responsive to the input value for calculating a current time value as a direct function of the input time value determined from an offset value, a rate value and an input time value resident in the storage;

(b) program code for creating first and second multimedia objects in the storage, the first multimedia object including methods responsive to the current time value of the first control object for presenting the first multimedia time-based media sequence at a first playback position on the presentation mechanism, the second multimedia object including methods responsive to the current time value of the second control object for presenting the second multimedia time-based media sequence at a second playback position on the presentation mechanism;

(c) program code responsive to the connection request for providing the current time value of the first control object as an input value to the second control object so that the second playback position is a direct function of the first playback position; and

(d) program code for creating a time source in the storage, the time source providing a sequence of input values to the first control object to cause the first multimedia time-based media sequence to be presented in lockstep with the second multimedia time-based media sequence.