System and method of streaming 3-D wireframe animations

• US 8,421,804 B2
• Filed: 02/16/2005
• Issued: 04/16/2013
• Est. Priority Date: 02/16/2005
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Ostermann, Joern, Varakliotis, Sokratis
• Primary Examiner(s): Wu, Xiao M.
• Assistant Examiner(s): Tseng, Charles
• Application Number:
  US11/059,118
• Publication Number:
  US 20060181536A1
• Time in Patent Office: 2,981 Days
• Field of Search: 345/428, 345473-475
• US Class Current: 345/473
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/67 involving unequal error protection [UEP], i.e. providing protection according to the importance of the data
• CPC Class Codes: H04N 19/895 in combination with error concealment

Abstract

Optimal resilience to errors in packetized streaming 3-D wireframe animation is achieved by partitioning the stream into layers and applying unequal error correction coding to each layer independently to maintain the same overall bitrate. The unequal error protection scheme for each of the layers combined with error concealment at the receiver achieves graceful degradation of streamed animation at higher packet loss rates than approaches that do not account for subjective parameters such as visual smoothness.

Claims

1. A method comprising:

computing, via a processor, visual smoothness values for each node in a wireframe mesh, each node comprising an indexed set of vertices in the wireframe mesh;

partitioning the indexed set of vertices to produce a partitioned mesh having more nodes than the wireframe mesh;

computing a visual smoothness for each node in the partitioned mesh;

layering data describing the wireframe mesh into a plurality of layers by ordering nodes into a respective layer according to whether the nodes have a similar visual smoothness; and

allocating first nodes associated with visible parts of the wireframe mesh to a first layer of the plurality of layers and allocating second nodes associated with non-visible parts to a second layer of the plurality of layers, wherein the second layer is only transmitted to a decoder where a user at the decoder is able to rotate the wireframe mesh to view the nodes associated with the non-visible parts, wherein average visual smoothness values associated with each of the plurality of layers reflect a respective layer'"'"'s importance in the animation sequence;

applying different parameters n, FR, Sp, and F_BOP to a channel bitrate algorithm that applies to a block-of-packets structure for each layer of the plurality of layers, the channel bitrate algorithm comprising;

R=(N*FR*Sp)/F_BOP

wherein R is a bitrate, n is a number of lines, and Sp is a number of columns in the block-of-packets structure, FR is a sequence frame rate, and F_BOP is a number of data frames in the block-of-packets structure for a respective layer; and

applying unequal error protection to each respective layer of the plurality of layers according to a result of the channel bitrate algorithm.

2. The method of claim 1, wherein each layer of the plurality of layers comprises one of a node and a group of nodes within the wireframe mesh.

3. The method of claim 1, wherein a most important portion of the plurality of layers comprises the average visual smoothness value with a highest value and is error coded to be most resilient to packet errors.

4. The method of claim 1, wherein a number of nodes within a particular portion of the plurality of layers is associated with an output bit rate of the particular portion of the plurality of layers.

5. The method of claim 1, further comprising producing a 3-D packetized streaming signal representative of a scene comprising animation associated with the wireframe mesh.

6. The method of claim 1, wherein partitioning further comprises partitioning the wireframe mesh into arbitrary sized sub-meshes that will be allocated to a same layer.

7. A method comprising:

receiving streamed data from a source;

generating, from the streamed data, a representation of a wireframe mesh partitioned to produce a partitioned mesh having more nodes than the wireframe mesh, wherein the wireframe mesh is partitioned to yield partitions corresponding to different objects within a scene represented by the nodes of the partitioned mesh and a corresponding motion of the different objects in an animation sequence, the partitions being balanced based on a number of nodes in each partition, encoding parameters, and the corresponding motion associated with each partition, the streamed data based on an unequal error protection scheme, wherein;

the streamed data comprises a portion of a dataset at the source describing the wireframe mesh, the dataset comprising a plurality of layers associated with different average visual smoothness values that reflect a respective layer'"'"'s importance in an animation sequence;

a layer of the plurality of layers in the dataset comprises a plurality of partitions that identifies data in the layer that is visible and invisible when an animation in the animation sequence is in a static mode;

the portion of the dataset comprises a portion of a plurality of layers and a portion of the plurality of partitions unassociated with the portion of the plurality of layers; and

the portion of the dataset is selected to be supportable by the current bandwidth and to provide largest ones of the average visual smoothness values; and

applying, via a processor, applying different parameters n, FR, Sp, and F_BOP to a channel bitrate algorithm that applies to a block-of-packets structure for each layer of the plurality of layers, the channel bitrate algorithm comprising;

R=(N*FR*Sp)/F_BOP

wherein R is a bitrate, n is a number of lines, and Sp is a number of columns in the block-of-packets structure, FR is a sequence frame rate, and F_BOP is a number of data frames in the block-of-packets structure for a respective layer; and

applying an unequal error concealment scheme to each respective layer in the pluarilty of layers according to a result of the channel bitrate algorithm.

8. The method of claim 7, wherein the unequal error protection scheme comprises optimizing a distribution of a bit budget allocation amongst the plurality of layers.

9. The method of claim 7, wherein the plurality of layers are partitioned according to visual importance.

10. The method of claim 7, wherein interpolation-based concealment is applied to residual packets not recovered from error-concealment.