Generation of a bit stream containing binary image/audio data that is multiplexed with a code defining an object in ascii format

• US 6,611,262 B1
• Filed: 09/11/1998
• Issued: 08/26/2003
• Est. Priority Date: 09/22/1997
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Suzuki, Teruhiko
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/152,003
• Time in Patent Office: 1,810 Days
• Field of Search: 375/240.26, 375/240.13, 348/425.1, 348/425.3, 709/217, 345/419, 345/440, 345/619
• US Class Current: 345/419
• CPC Class Codes: H04N 21/234309 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo conversion of standards in analog television systems H04N7/01
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

A system controller directs a storage device to output a scene description data corresponding to a user request signal. A parsing circuit extracts a URL (Uniform Resource Locator) included in the scene description data, and causes storage devices to output elementary stream and object stream corresponding to the URL. After extracting an object descriptor from the object stream, a generator generates an ID of this object descriptor and supplies it to an encoder. Further, the generator adds the ID to the object descriptor and outputs the ID-added object descriptor to a multiplexer. The multiplexer multiplexes the scene description data including the ID that has been converted into binary format, the object descriptor and the elementary streams into a multiplexed stream for output thereof.

Claims

1. A method for producing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, said method comprising:

extracting a respective position from a node of said three dimensional space modeling data;

converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

replacing said position with said stream ID; and

multiplexing said image/audio data and said three dimensional space modeling data including said stream ID such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

2. The method according to claim 1, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

3. The method according to claim 1, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said position is represented by Uniform Resource Locator (URL) expressed in ASCII format, and said stream ID is expressed in binary format.

4. The method according to claim 3, further comprising converting said stream ID into a character string, and determining whether to replace said position of said image/audio data with said stream ID or said character string depending on whether said image/audio data is supplied by the same server or different servers.

5. The method according to claim 4, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

6. A method for producing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, said method comprising:

extracting a respective position from a node of said three dimensional space modeling data;

converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

converting said stream ID into a character string;

replacing said position with said character string; and

multiplexing said image/audio data and said three dimensional space modeling data including said character string such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

7. The method according to claim 6, wherein said position is replaced with said character string depending on whether said image/audio data is supplied by the same server or different servers.

8. The method according to claim 6, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said position is represented by Uniform Resource Locator (URL) expressed in ASCII format, and said stream ID is expressed in binary format.

9. The method according to claim 6, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

10. Apparatus for producing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, comprising:

means for extracting a respective position from a node of said three dimensional space modeling data;

means for converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

means for replacing said position with said stream ID; and

means for multiplexing said image/audio data and said three dimensional space modeling data including said stream ID such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

11. The apparatus according to claim 10, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

12. The apparatus according to claim 10, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said position is represented by Uniform Resource Locator (URL) expressed in ASCII format, and said stream ID is expressed in binary format.

13. The apparatus according to claim 12, further comprising means for converting said stream ID into a character string, and means for determining whether to replace said position of said image/audio data with said stream ID or said character string depending on whether said image/audio data is supplied by the same server or different servers.

14. The apparatus according to claim 13, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

15. Apparatus for producing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, comprising:

means for extracting a respective position from a node of said three dimensional space modeling data;

means for converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

means for converting said stream ID into a character string;

means for replacing said position with said character string; and

means for multiplexing said image/audio data and said three dimensional space modeling data including said character string such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

16. The apparatus according to claim 15, wherein said position is replaced with said character string depending on whether said image/audio data is supplied by the same server or different servers.

17. The apparatus according to claim 15, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said position is represented by Uniform Resource Locator (URL) expressed in ASCII format, and said stream ID is expressed in binary format.

18. The apparatus according to claim 15, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream.

19. A method for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, said method comprising:

receiving said bit stream;

demultiplexing the received bit stream into a stream ID, said three dimensional space modeling data and image/audio data; and

providing a correspondence between said image/audio data and a respective node in accordance with said stream ID such that said display image is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

20. The method according to claim 19, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML) and said stream ID information is expressed in binary format.

21. The method according to the claim 19, wherein the correspondence between said image/audio data and node information is in accordance with either said stream ID expressed by a first expression or a character string corresponding to said stream ID expressed by a second expression;

and wherein said node includes a flag for indicating whether said first or second expression has been used depending on whether said image/audio data has been supplied by the same server or different servers.

22. The method according to claim 19, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream;

and wherein said stream ID included in said node is collated with said stream ID included in said additional information.

23. A method for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, said method comprising:

receiving said bit stream including said image/audio data and said three dimensional space modeling data comprising the nodes, and outputting said three dimensional space modeling data and image/audio data;

converting character string information into stream ID information, said character string information being information indicating a correlation between a node and said image/audio data; and

linking said image/audio data and said node in accordance with the converted stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

24. The method according to the claim 23, wherein said information indicating the correlation is one of said character string information corresponding to said stream ID information and position designation information;

wherein said image/audio data is linked to said node in accordance with the converted stream ID information if said information indicating the correlation is said character string information, and said image/audio data is linked from a supplying portion designated by said position designation information to said node in case that said information indicating the correlation is said position designation information.

25. The method according to claim 23, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said character string information is expressed in ASCII format, and the converted stream ID information is expressed in binary format.

26. The method according to claim 23, wherein said bit stream includes information defining said image/audio data and including said stream ID;

and wherein the converted stream ID information is collated with the stream ID information included in the information defining said image/audio data, and said image/audio data is linked to said node in accordance with the result of collation.

27. Apparatus for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, comprising:

means for receiving said bit stream;

means for demultiplexing the received bit stream into a stream ID, said three dimensional space modeling data and image/audio data; and

means for providing a correspondence between said image/audio data and a respective node in accordance with said stream ID such that said display image is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

28. The apparatus according to claim 27, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML) and said stream ID information is expressed in binary format.

29. The apparatus according to the claim 27, wherein the correspondence between said image/audio data and node information is in accordance with either said stream ID expressed by a first expression or a character string corresponding to said stream ID expressed by a second expression;

and wherein said node includes a flag for indicating whether said first or second expression has been used depending on whether said image/audio data has been supplied by the same server or different servers.

30. The apparatus according to claim 27, wherein said bit stream includes additional information that includes said stream ID and that defines said image/audio data, said additional information having been multiplexed with said image/audio data and three dimensional space modeling data in said bit stream;

and wherein said stream ID included in said node is collated with said stream ID included in said additional information.

31. Apparatus for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, comprising:

means for receiving said bit stream including said image/audio data and said three dimensional space modeling data comprising the nodes;

means for outputting said three dimensional space modeling data and image/audio data;

means for converting character string information into stream ID information, said character string information being information indicating a correlation between a node and said image/audio data; and

means for linking said image/audio data and said node in accordance with the converted stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

32. The apparatus according to the claim 31, wherein said information indicating the correlation is one of said character string information corresponding to said stream ID information and position designation information;

wherein said image/audio data is linked to said node in accordance with the converted stream ID information if said information indicating the correlation is said character string information, and said image/audio data is linked from a supplying portion designated by said position designation information to said node in case that the information indicating the correlation is said position designation information.

33. The apparatus according to claim 31, wherein said three dimensional space modeling data is described by Virtual Reality Modeling Language (VRML), said character string information is expressed in ASCII format, and the converted stream ID information is expressed in binary format.

34. The apparatus according to claim 31, wherein said bit stream includes information that defines said image/audio data and that includes said stream ID;

and wherein the converted stream ID information is collated with the stream ID information included in said information defining said image/audio data, and said image/audio data is linked to said node in accordance with the result of collation.

35. A recording medium having recorded thereon a data producing program for producing three dimensional space modeling data comprising a plurality of nodes and image/audio data designated by position designation information included in the nodes of said three dimensional space modeling data, said data producing program being executed to perform the steps of:

extracting position designation information included in a node of said three dimensional space modeling data;

converting the extracted position designation information into stream ID information corresponding to said image/audio data designated by the extracted position designation information;

replacing the position designation information included in said node with said stream ID information; and

multiplexing said image/audio data and three dimensional space modeling data including said stream ID information such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

36. The recording medium according to claim 35, wherein said data producing program being further executed to perform the steps of:

converting said stream ID information into character string information being expressed by a first expression; and

determining whether to replace said position designation information included in said node with said stream ID information expressed by a second expression or with said character string information expressed by said first expression, wherein said position designation information included in said node is replaced in accordance with the determined result.

37. A recording medium having recorded thereon a data producing program for producing three dimensional space modeling data comprising a plurality of nodes and image/audio data designated by position designation information included in the nodes of said three dimensional space modeling data, said data producing program being executed to perform the steps of:

extracting position designation information included in a node of said three dimensional space modeling data;

converting the extracted position designation information into stream ID information corresponding to image/audio data designated by the extracted position designation information;

converting said stream ID information into character string information;

replacing said position designation information included in said node with said character string information; and

multiplexing said image/audio data and three dimensional space modeling data including said character string information such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

38. A recording medium having recorded thereon a data processing program for producing a display image from a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes, said data processing program being executed to perform the steps of:

receiving said bit stream including said image/audio data and said three dimensional space modeling data comprising a plurality of nodes;

outputting said three dimensional space modeling data and image/audio data; and

linking said image/audio data to a node in accordance with information indicating a correlation between said node and said image/audio data, said information indicating the correlation being stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

39. The recording medium according to the claim 38, wherein said information indicating the correlation is one of said stream ID information expressed by a first expression and character string information corresponding to said stream ID information, said character string information being expressed by a second expression;

and wherein said node includes flag information indicating said first or second expression of said information indicating the correlation, said data processing program being further executed to perform the steps of;

determining an expression of said information indicating the correlation in accordance with said flag information; and

converting said character string information expressed by said second expression into said stream ID information expressed by said first expression form, wherein said image/audio data is linked to said node in accordance with said stream ID information if said information indicating the correlation is said stream ID information, and said image/audio data is linked to said node in accordance with the converted stream ID information if said information indicating the correlation is said character string information.

40. A recording medium having recorded thereon a data processing program for producing a display image from a stream including a three dimensional space modeling data comprising a plurality of nodes and image/audio data, said data processing program comprising the steps of:

receiving said stream including said three dimensional space modeling data comprising a plurality of nodes and image/audio data and outputting said three dimensional space modeling data and image/audio data;

converting character string information into stream ID information, said character string information being information indicating a correlation between a node and image/audio data; and

linking said image/audio data to said node in accordance with the converted stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

41. A recording medium having recorded thereon a bit stream including three dimensional space modeling data comprising a plurality of nodes and image/audio data designated by position designation information included in the nodes of said three dimensional space modeling data, said bit stream being prepared by the steps of:

extracting position designation information included in a node of said three dimensional space modeling data;

converting the extracted position designation information into stream ID information corresponding to said image/audio data designated by the extracted position designation information;

replacing the position designation information included in said node with said stream ID information; and

multiplexing said image/audio data and three dimensional space modeling data including said stream ID data such that said bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

42. The recording medium according to claim 41, wherein said bit stream is further prepared by the steps of:

converting said stream ID information into character string information being expressed by a first expression; and

determining whether to replace said position designation information included in said node with said stream ID expressed by a second expression or with said character string information expressed by said first expression, wherein said position designation information included in said node is replaced in accordance with the determined result and information representative of the determined expression is inserted in place of the replaced information into said node.

43. A recording medium having recorded thereon a bit stream including three dimensional space modeling data comprising a plurality of nodes and image/audio data designated by position designation information included in the nodes of said three dimensional space modeling data, said bit stream being prepared by the steps of:

extracting position designation information included in a node of said three dimensional space modeling data;

converting the extracted position designation information into stream ID information corresponding to image/audio data designated by the extracted position designation information;

converting said stream ID information into character string information;

replacing said position designation information included in said node with said character string information; and

multiplexing said image/audio data and three dimensional space modeling data including said character string information such that said bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

44. A recording medium manufactured by a producing device, said recording medium having recorded thereon a signal having a data stream that includes three dimensional space modeling data comprising a plurality of nodes and image/audio data, said recording medium having the recorded signal processed by the steps of:

receiving said data stream including said three dimensional space modeling data comprising the nodes and said image/audio data;

outputting said three dimensional space modeling data and image/audio data; and

linking said image/audio data to a node in accordance with information indicating a correlation between said node and image/audio data, said information indicating the correlation being stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

45. The recording medium according to the claim 44, wherein said information indicating the correlation is one of said stream ID information expressed by a first expression and character string information corresponding to said stream ID information, said character string information expressed by a second expression;

and wherein said node includes flag information indicating an expression of said information indicating the correlation, said recording medium having the recorded signal further processed by the steps of;

determining an expression of said information indicating the correlation in accordance with said flag information; and

converting said character string information expressed by said second expression into said stream ID information expressed by said first expression, wherein said image/audio data is linked to said node in accordance with said stream ID information if said information indicating the correlation is said stream ID information, said image/audio data being linked to said node in accordance with the converted stream ID information if said information indicating the correlation is said character string information.

46. A recording medium producible by a producing device, said recording medium having recorded thereon a signal having a stream including three dimensional space modeling data that comprises a plurality of nodes and image/audio data, said recording medium having the recorded signal processed by the steps of:

receiving said stream including said three dimensional space modeling data comprising the nodes and said image/audio data;

outputting said three dimensional space modeling data and image/audio data;

converting character string information into stream ID information, said character string information being information indicating a correlation between a node and image/audio data; and

linking said image/audio data to said node in accordance with the converted stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

47. Apparatus for providing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, comprising:

a parsing circuit for extracting a respective position from a node of said three dimensional space modeling data;

a converter for converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

an encoder for replacing said position with said stream ID; and

a multiplexer for multiplexing said image/audio data and said three dimensional space modeling data including said stream ID such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

48. Apparatus for providing three dimensional space modeling data defined by a plurality of nodes and image/audio data specified by a position included in the nodes, comprising:

a parsing circuit for extracting a respective position from a node of said three dimensional space modeling data;

a converter for converting the extracted position into a stream ID corresponding to image/audio data associated with said position;

a changing circuit for converting said stream ID into character string information;

an encoder for replacing said position with said character string; and

a multiplexer for multiplexing said image/audio data and said three dimensional space modeling data including said character string such that a bit stream is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

49. Apparatus for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, comprising:

a demultiplexer for receiving said bit stream and for demultiplexing the received bit stream into a stream ID, said three dimensional space modeling data and image/audio data; and

a reconstruction circuit for providing a correspondence between said image/audio data and a respective node in accordance with said stream ID such that said display image is produced;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.

50. Apparatus for processing a bit stream including image/audio data and three dimensional space modeling data comprising a plurality of nodes to produce a display image, comprising:

a demultiplexer for receiving said bit stream including said image/audio data and said three dimensional space modeling data comprising the nodes and for outputting said three dimensional space modeling data and image/audio data;

a convertor for converting character string information into stream ID information, said character string information being information indicating a correlation between a node and said image/audio data; and

a reconstruction circuit for linking said image/audio data and said node in accordance with the converted stream ID information;

whereby said three dimensional space modeling data, said image/audio data and said stream ID are multiplexed according to a uniform format such that the format of said stream is determined prior to said multiplexing.