Architecture and application programming interfaces for Java-enabled MPEG-4 (MPEG-J) systems

• US 6,631,403 B1
• Filed: 05/11/1999
• Issued: 10/07/2003
• Est. Priority Date: 05/11/1998
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.), Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.), Sun Microsystems (Oracle Corporation)
• Inventors: Puri, Atul, Swaminathan, Viswanathan, Fernando, Gerard M., Shah, Pallavi K., Deutsch, Keith R., Schmidt, Robert L.
• Primary Examiner(s): Sheikh, Ayaz
• Assistant Examiner(s): Tran, Philip B.
• Application Number:
  US09/309,895
• Time in Patent Office: 1,610 Days
• Field of Search: 709/203-205, 709/217-219, 709/231, 709/328
• US Class Current: 709/217
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 21/4437 Implementing a Virtual Machine [VM] virtual machines in general G06F9/45533
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

MPEG-J is a collection of Java application programming interfaces (APIS) with which applications can be developed to interact with the platform and the content. In the context of MPEG-J, the platform is a device like a set-top box or a PC with Java packages conforming to a well-defined Java platform. The Java-based application consists of Java byte code, which may be available from a local source, like a hard disk, or it may be loaded from a remote site over a network. The MPEG-J Java byte code will be available as a separate elementary stream. The MPEG-4 system is the “Presentation engine” of MPEG-J. MPEG-J provides programmatic control through an “Application engine” which enhances the MPEG-4 browser by providing added interactive capability.

Claims

1. A multimedia browsing system, comprising:

a multimedia browsing subsystem that generates a multimedia scene based on a scene description, the multimedia scene comprising at least one multimedia object and the scene description defining spatial and temporal positioning of each object within the scene; and

a multimedia browsing control subsystem that controls the multimedia browsing subsystem based on a control program associated with the at least one multimedia object;

wherein;

the multimedia browsing control subsystem controls a behavior of the multimedia browsing subsystem based on behavior control commands of the control program associated with the at least one multimedia object;

the multimedia browsing system is executing on a data processing apparatus having at least one system resource;

the behavior control commands associated with the at least one multimedia object controllably modifying the multimedia scene based at least on a status of the one system resource;

the multimedia browsing subsystem is controlled by the multimedia browsing control subsystem through at least one application programming interface, which includes at least one each of an application manager application programming interface, a scene graph application programming interface, a resource manager application programming interface, a media decoder application programming interface, a functionality application programming interface, a network application programming interface, a device application programming interface, and an interactivity application programming interface;

wherein;

the multimedia browsing control subsystem comprises;

an application controller; and

a scene graph manager;

the scene graph application programming interface connects the application controller and the scene graph manager;

the scene graph manager is connected to a scene graph controller of the multimedia browsing subsystem;

wherein the multimedia browsing subsystem further comprises a scene graph decoder that decodes encoded scene graph data, and that outputs the decoded scene graph data to the scene graph controller;

the multimedia browsing control subsystem further comprises a scene encoder connected to the scene graph decoder of the multimedia browsing subsystem and controllably connectable to the scene graph manager; and

the scene graph manager controls the scene encoder to create additional encoded scene graph data based at least on the multimedia scene being controllably modified based at least on a status of the one system resource, the scene encoder outputting the additional encoded scene graph data to the scene graph decoder.

2. The multimedia browsing system of claim 1, wherein the multimedia browsing control subsystem further comprises a switch that selectively connects the scene graph manager to one of the scene encoder and the scene graph controller.

3. The multimedia browsing system according to claim 1, wherein the behavior control commands associated with the at least one multimedia object controllably degrade the multimedia scene based at least on a status of the one system resource.

4. A method for browsing an MPEG-4 multimedia scene comprising at least one MPEG-4 multimedia object, comprising:

generating, using a multimedia browsing subsystem, the MPEG-4 multimedia scene based on an MPEG-4 scene description, the MPEG-4 scene description defining spatial and temporal positioning of each MPEG-4 multimedia object within the scene, including executing the MPEG-4 scene description on a data processing apparatus having at least one system resource;

controlling the generation of the MPEG-4 multimedia scene based on an MPEG-J control program associated with the at least one MPEG-4 multimedia object by controlling the multimedia browsing subsystem using a multimedia browsing control subsystem, which includes at least one each of an application manager application programming interface, a scene graph application programming interface, a resource manager application programming interface, a media decoder application programming interface, a functionality application programming interface, a network application programming interface, a device application programming interface, and an interactivity application programming interface, including;

connecting the multimedia browsing subsystem to the multimedia browsing control subsystem through at least one application programming interface;

connecting an application controller and a scene graph manager through the scene graph application programming interface;

connecting the scene graph manager to a scene graph controller of the multimedia browsing subsystem;

controllably modifying the MPEG-4 multimedia scene based at least on a status of the one system resource;

decoding encoded scene graph data using a scene graph decoder of the multimedia browsing subsystem;

outputting decoded scene graph data to the scene graph controller;

connecting a scene graph encoder to the scene graph decoder of the multimedia browsing subsystem;

controllably connecting the scene graph encoder to the scene graph manager;

controlling the scene graph encoder using the scene graph manager to create additional encoded scene graph data based at least on the multimedia scene being controllably modified based at least on a status of the one system resource; and

outputting the additional encoded scene graph data from the scene graph encoder to the scene graph decoder.

5. The method of claim 4, further comprising:

selectively connecting the scene graph manager to one of the scene encoder and the scene graph controller using a switch.

6. The method according to claim 4, wherein controllably modifying the MPEG-4 multimedia scene includes controllably degrading the MPEG-4 multimedia scene based at least on a status of the one system resource.