Methods and apparatus for integrating external applications into an MPEG-4 scene

• US 6,934,906 B1
• Filed: 07/10/2000
• Issued: 08/23/2005
• Est. Priority Date: 07/08/1999
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Cheok, Lai-tee
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): Huynh, Thu V.
• Application Number:
  US09/613,292
• Time in Patent Office: 1,870 Days
• Field of Search: 715/500.1, 715/500, 709/204, 709/217, 709/229, 345/419
• US Class Current: 715/204
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A method of decoding, composing and rendering a scene. First information is obtained, the first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream. The first information is decoded by invoking a BIFS scene decoder and one or more specific media decoders that are required by the scene. Second information is obtained, the second information including a second part of a BIFS scene description stream that contains a reference to an external application. The second information is decoded by invoking the BIFS scene decoder and an external application decoder. An integrated scene is composed, the integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded scene descriptions streams. The composed integrated scene is rendered on a display.

Claims

1. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to an external application, wherein the reference to the external application includes a coded representation of a Universal Resource Locator relating to a location where software for an external application decoder resides;

decoding the second information using the BIFS scene decoder and the external application decoder;

composing an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded MPEG-4 BIFS scene description stream and the decoded BIFS scene description stream; and

rendering the composed integrated scene.

2. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to an external application, wherein the reference to the external application includes a start time and a stop time;

decoding the second information using the BIFS scene decoder and an external application decoder;

composing an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded MPEG-4 BIFS scene description stream and the decoded BIFS scene description stream;

rendering the composed integrated scene;

starting the external application decoder at the start time; and

stopping the external application decoder at the stop time.

3. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to an external application, wherein the reference to the external application includes description information;

decoding the second information using the BIFS scene decoder and an external application decoder;

composing an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded MPEG-4 BIFS scene description stream and the decoded BIFS scene description stream;

rendering the composed integrated scene; and

displaying a prompt in the scene based on the description information.

4. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to an external application;

decoding the second information using the BIFS scene decoder and an external application decoder;

composing an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded MPEG-4 BIFS scene description stream and the decoded BIFS scene description stream;

rendering the composed integrated scene;

receiving information signaling that the external application decoder should run in the background; and

signaling the external application decoder to relinquish an application window and continue to run.

5. An article of manufacture comprising a computer-readable medium having stored thereon instructions adapted to be executed by a processor, the instructions which, when executed, cause the processor to:

obtain information including a part of a MPEG-4 BIFS scene description stream, at least one coded MPEG-4 media stream, and a reference to an external application, wherein the reference to the external application includes a coded representation of a Universal Resource Locator relating to a location where software for the external application resides;

decode said information obtained;

compose an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded information; and

render the composed integrated scene.

6. An article of manufacture comprising a computer-readable medium having stored thereon instructions adapted to be executed by a processor, the instructions which, when executed, cause the processor to:

obtain information including a part of a MPEG-4 BIFS scene description stream, at least one coded MPEG-4 media stream, and a reference to an external application, wherein the reference to the external application integration includes a start time and a stop time;

decode said information obtained;

compose an integrated scene including one or more decoded MPEG-4 media objects and one or more external application objects specified in the decoded information;

render the composed integrated scene;

start the external application at the start time; and

stop the external application at the stop time.

7. The article of manufacture of claim 6, wherein the reference to an external application includes description information, and wherein the instructions stored on the computer-readable medium further include instructions adapted to be executed the processor to display a prompt in the scene based on the description information.

8. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

composing and rendering a scene including one or more decoded MPEG-4 media objects specified in the decoded MPEG-4 BIFS scene description stream;

detecting user interaction with the scene requesting activation of an external application;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to the external application, wherein the reference to the external application includes information corresponding to an ApplicationWindow node, wherein the information corresponding to the ApplicationWindow node includes information identifying a location for an application window within the scene and information identifying dimensions of an application window, and wherein the information corresponding to the ApplicationWindow node includes a start time and a stop time;

decoding the second information using the BIFS scene decoder and an external application decoder;

re-composing the scene by integrating a sub-scene specified by the decoded second information into the scene;

rendering the re-composed scene,

starting the external application at the start time; and

stopping the external application at the stop time.

9. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

composing and rendering a scene including one or more decoded MPEG-4 media objects specified in the decoded MPEG-4 BIFS scene description stream;

detecting user interaction with the scene requesting activation of an external application;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to the external application, wherein the reference to the external application includes information corresponding to an ApplicationWindow node, wherein the information corresponding to the ApplicationWindow node includes information identifying a location for the application window within the scene and information identifying dimensions of the application window, and, wherein the information corresponding to the ApplicationWindow node includes description information;

decoding the second information using the BIFS scene decoder and an external application decoder;

re-composing the scene by integrating a sub-scene specified by the decoded second information into the scene;

rendering the re-composed scene; and

displaying a prompt in the scene based on the description information.

10. A method of rendering a scene, the method comprising:

obtaining first information including a part of a MPEG-4 BIFS scene description stream and at least one coded MPEG-4 media stream;

decoding the first information using a BIFS scene decoder and one or more specific application decoders associated with the scene description;

composing and rendering a scene including one or more decoded MPEG-4 media objects specified in the decoded MPEG-4 BIFS scene description stream;

detecting user interaction with the scene requesting activation of an external application;

obtaining second information including a second part of a BIFS scene description stream that contains a reference to the external application, wherein the reference to the external application includes information corresponding to an ApplicationWindow node, wherein the information corresponding to the ApplicationWindow node includes information identifying a location for an application window within the scene and information identifying dimensions of the application window;

decoding the second information using the BIFS scene decoder and an external application decoder;

re-composing the scene by integrating a sub-scene specified by the decoded second information into the scene;

rendering the re-composed scene;

receiving information signaling that the external application should run in the background; and

signaling the external application to relinquish the application window and continue to run.