Delivering and controlling streaming interactive media comprising rendered geometric, texture and lighting data

• US 9,489,762 B2
• Filed: 09/13/2011
• Issued: 11/08/2016
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Primal Space Systems Inc.
• Original Assignee: Primal Space Systems Inc.
• Inventors: Jenkins, Barry Lynn
• Primary Examiner(s): BROOME, SAID A
• Application Number:
  US13/822,863
• Publication Number:
  US 20130215115A1
• Time in Patent Office: 1,883 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics

Abstract

In an exemplary embodiment, a method, conducted on a server, sends renderable graphics information to a client device, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region. The method includes determining a likelihood that a viewpoint undergoes movement from said first view region to said second view region. The method further includes sending said at least one set of graphics information upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold, said movement determined according to a predetermined motion path.

Claims

1. A method, conducted on a server, of sending renderable graphics information to a client device, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, the method comprising:

a) determining, using processing circuitry, a likelihood that a viewpoint undergoes movement from said first view region to said second view region;

b) sending said at least one set of graphics information upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold, said movement determined according to a predetermined motion path or by user input to said client device; and

c) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, and said set of mesh polygons or fragments of said mesh polygons visible from said second view region is determined by;

c1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, said at least one first-order silhouette edge being an edge of said polygon meshes having;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices of said second view region,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices of said second view region, and

the first and second polygons having a backfacing orientation with respect to each other,

c2) determining, using the processing circuitry, at least one supporting polygon between said second view region and said polygon meshes, said at least one supporting polygon being determined between said second view region and at least one vertex of said at least one first-order silhouette edge;

c3) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond at least said at least one first-order silhouette edge;

c4) determining, using the processing circuitry, one or more intersections of said at least one wedge with said mesh polygons; and

c5) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

2. A method, conducted on a server, of sending renderable graphics information to a client device, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, the method comprising:

a) determining, using processing circuitry, a likelihood that a viewpoint undergoes movement from said first view region to said second view region;

b) sending said at least one set of graphics information upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold, said movement determined according to a predetermined motion path or by user input to said client device; and

c) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, and said set of mesh polygons or fragments of said mesh polygons visible from said second view region determined by;

c1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices,

the first and second polygons having a backfacing orientation with respect to each other;

c2) determining, using the processing circuitry, if said at least one first-order silhouette edge and another first-order silhouette edge share a vertex forming an inside corner vertex of said polygon mesh;

c3) upon determination that said at least one first-order silhouette edge and another first-order silhouette edge share a vertex forming an inside corner vertex of said polygon mesh, determining at least one supporting polygon between said second view region and said at least one first-order silhouette edge, wherein said at least one supporting polygon is formed by said inside corner vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said inside corner vertex forming a plane,

said plane having a same sidedness orientation as said first polygon sharing said at least one first-order silhouette edge, and

said plane not front-facing with respect to each view region vertex from said plurality of view region vertices;

c4) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least one first-order silhouette edge;

c5) determining, using the processing circuitry, one or more intersections of said wedges with said mesh polygons; and

c6) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

3. A method, conducted on a server, of sending renderable graphics information to a client device, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, the method comprising:

a) determining, using processing circuitry, a likelihood that a viewpoint undergoes movement from said first view region to said second view region;

b) sending said at least one set of graphics information upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold, said movement determined according to a predetermined motion path or by user input to said client device; and

c) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, and said set of mesh polygons or fragments of said mesh polygons visible from said second view region determined by;

c1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices, and

the first and second polygons having a backfacing orientation with respect to each other,

c2) determining, using the processing circuitry, at least one supporting polygon between said second view region and said at least one first-order silhouette edge;

c3) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least said at least one first-order silhouette edge;

c4) determining, using the processing circuitry, one or more intersections of said at least one wedge with said mesh polygons;

c5) determining, using the processing circuitry, if said at least one wedge intersects at least one second first-order silhouette edge, said intersection forming a compound silhouette vertex;

c6) upon determining that said at least one wedge intersects said at least one second first-order silhouette edge forming a compound silhouette vertex, forming at least one second wedge, wherein said at least one second wedge is formed by said compound silhouette vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said compound silhouette vertex forming a plane, said plane not front-facing with respect to each view region vertex from said plurality of view region vertices; and

c7) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

4. A method, conducted on a server, of sending renderable graphics information to a client device, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, and said graphics information including a second set of graphics information visible from said second view region and not visible from said first view region, said second set of graphics information representing advertising, the method comprising:

a) sending said first set of graphics information upon determination, using processing circuitry, that the likelihood that a viewpoint moves from said first view region to said second view region is greater than a predetermined movement threshold, said movement threshold determined according to user input to said client device or by a predetermined motion path; and

b) sending said second set of graphics information upon determination, using the processing circuitry, that the likelihood that the viewpoint moves from said first view region to said second view region is greater than the predetermined movement threshold, said movement threshold determined according to the user input to said client device or by a predetermined motion path, and upon determination that an available transmission bandwidth from said server to said client device exceeds a predetermined threshold.

5. The method of claim 4, wherein said first set of graphics information and said second set of graphics information are determined by:

c) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices, and

the first and second polygons having a backfacing orientation with respect to each other;

d) determining, using the processing circuitry, if said at least one first-order silhouette edge and another first-order silhouette edge share a vertex forming an inside corner vertex of said polygon mesh;

e) upon determination that said at least one first-order silhouette edge and another first-order silhouette edge share a vertex forming an inside corner vertex of said polygon mesh, determining at least one supporting polygon between said second view region and said at least one first-order silhouette edge, wherein said at least one supporting polygon is formed by said inside corner vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said inside corner vertex forming a plane,

said plane having a same sidedness orientation as said first polygon sharing said at least one first-order silhouette edge, and

said plane not front-facing with respect to each view region vertex from said plurality of view region vertices;

f) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least one first-order silhouette edge;

g) determining, using the processing circuitry, one or more intersections of said wedges with said mesh polygons; and

h) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

6. The method of claim 4, wherein said first set of graphics information and said second set of graphics information are determined by:

c) determining, using processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices,

the first and second polygons having a backfacing orientation with respect to each other;

d) determining, using the processing circuitry, at least one supporting polygon between said second view region and at least one first-order silhouette edge;

e) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least one first-order silhouette edge;

f) determining, using the processing circuitry, one or more intersections of said wedges with said mesh polygons;

g) determining, using the processing circuitry, if said at least one wedge intersects at least one second first-order silhouette edge, said intersection forming a compound silhouette vertex;

h) upon determining that said at least one wedge intersects said at least one second first-order silhouette edge forming a compound silhouette vertex, forming at least one second wedge, wherein said at least one second wedge is formed by said compound silhouette vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said compound silhouette vertex forming a plane, said plane not front-facing with respect to each view region vertex from said plurality of view region vertices; and

i) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

7. The method of claim 5, wherein said graphics information is targeting information, which includes information about age or gender of said user of said client device.

8. The method according to claim 6, wherein said graphics information is targeting information, which includes information indicating previous responses to advertising made by said user of said client device.

9. The method of claim 4, wherein said graphics information comprises graphics information selectable by a viewer, and further comprising:

c) receiving information from said client device, indicating a selection of said second set of graphics information representing said advertising; and

d) sending, upon receiving information from said client device indicating a selection of said second set of graphics information representing said advertising, a third set of graphics information, said second and third sets of graphics information representing related products or services associated with said advertising.

10. The method of claim 4, wherein said graphics information comprises graphics information selectable by a viewer, and further comprising:

c) receiving information from said client device, indicating a selection of said second set of graphics information representing said advertising; and

d) sending, upon receiving information from said client device indicating a selection of said second set graphics information representing said advertising, a third set of graphics information, said third set of graphics information representing an object that can be used by a client user in a modeled environment.

11. The method of claim 10, further comprising:

e) charging the user in accordance with said selection.

12. A method, conducted on a client device, of receiving renderable graphics information from a server, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, the method comprising:

a) determining, using processing circuitry, a likelihood that a user viewpoint moves from said first view region to said second view region, said user viewpoint movement determined according to a predetermined motion path or by user input to a client device;

b) sending data to the server, said data representing a request for said at least one set of graphics information, upon determination that a likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined movement threshold;

c) receiving at least one set of graphics information from the server;

d) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, and said set of mesh polygons or fragments of said mesh polygons visible from said second view region determined by;

d1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, said at least one first-order silhouette edge being an edge of said polygon meshes having;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices of said second view region,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices of said second view region, and

the first and second polygons having a backfacing orientation with respect to each other,

d2) determining, using the processing circuitry, at least one supporting polygon between said second view region and said polygon meshes, said at least one supporting polygon being determined between said second view region and at least one vertex of said at least one first-order silhouette edge;

d3) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least one first-order silhouette edge;

d4) determining, using the processing circuitry, one or more intersections of said at least one wedge with said mesh polygons; and

d5) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

15. The method according to claim 12, wherein step c) of receiving said at least one set of graphics information from the server is performed upon determination that said user input to the client device is detected.

16. The method according to claim 12, wherein step b of receiving said at least one set of graphics information from the server is performed upon determination that said user input to the client device is not detected for a predetermined period of time.

13. A method, conducted on a client device, of receiving renderable graphics information from a server, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, the method comprising:

a) determining, using processing circuitry, a likelihood that a viewpoint undergoes movement from said first view region to said second view region, said movement determined according to a predetermined motion path or by user input to said client device;

b) sending data to the server, said data representing a request for at least one set of graphics information, upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold;

c) receiving said at least one set of graphics information from the server;

d) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, said first set of graphics information determined from said mesh polygons or fragments of said mesh polygons visible from said second view region, and said mesh polygons or fragments of said mesh polygons visible from said second view region determined by;

d1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices, and

the first and second polygons having a backfacing orientation with respect to each other;

d2) determining, using the processing circuitry, at least one supporting polygon between said second view region and said polygon meshes, said at least one supporting polygon being determined between said second view region and at least one vertex of said at least one first-order silhouette edge;

d3) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond at least said one first-order silhouette edge;

d4) determining, using the processing circuitry, one or more intersections of said at least one wedge with said mesh polygons;

d5) determining, using the processing circuitry, if said at least one wedge intersects at least one second silhouette edge, said intersection forming a compound silhouette vertex;

d6) upon determining that said at least one wedge intersects said at least one second first-order silhouette edge forming a compound silhouette vertex, forming at least one second wedge, wherein said at least one second wedge is formed by said compound silhouette vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said compound silhouette vertex forming a plane, said plane not front-facing with respect to each view region vertex from said plurality of view region vertices; and

d7) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.

14. A method, conducted on a client device, of receiving renderable graphics information from a server, said graphics information including at least one set of graphics information visible from a second view region and not visible from a first view region, and said at least one set of graphics representing advertising, the method comprising:

a) determining, using processing circuitry, a likelihood that a viewpoint undergoes movement from said first view region to said second view region, said movement determined according to a predetermined motion path or by user input to said client device;

b) sending data to the server, said data representing a request for at least one set of graphics information, upon determination that the likelihood that the viewpoint undergoes movement from said first view region to said second view region is greater than a predetermined threshold;

c) receiving said at least one set of graphics information from the server;

d) said at least one set of graphics information includes a set of mesh polygons or fragments of said mesh polygons visible from said second view region and not visible from said first view region, said first and second view regions having a plurality of view region vertices, said mesh polygons forming polygon meshes, and said mesh polygons or fragments of said mesh polygons visible from said second view region determined by steps of;

d1) determining, using the processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices, and

the first and second polygons having a backfacing orientation with respect to each other;

d2) determining, using the processing circuitry, at least one supporting polygon between said second view region and at least one first-order silhouette edge;

d3) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond at least one first-order silhouette edge;

d4) determining, using the processing circuitry, one or more intersections of said wedges with said mesh polygons;

d5) determining, using the processing circuitry, if said at least one wedge intersects at least one second silhouette edge, said intersection forming a compound silhouette vertex;

d6) upon determining that said at least one wedge intersects said at least one second first-order silhouette edge forming a compound silhouette vertex, forming at least one second wedge, wherein said at least one second wedge is formed by said compound silhouette vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said compound silhouette vertex forming a plane, said plane not front-facing with respect to each view region vertex from said plurality of view region vertices; and

d7) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes;

e) displaying the at least one set of graphics information with a visual indication that the set of graphics information is selectable with only a single button press; and

f) transmitting a selection result of the displayed second set of graphics information to the server.

17. A method, conducted on a client device, of receiving renderable graphics information from a server, said graphics information including a first set of graphics information visible from a second view region and not visible from a first view region, said first set of graphics information representing entertainment content, and said graphics information including a second set of graphics information visible from said second view region and not visible from said first view region, said second set of graphics information including graphics information representing advertising, the method comprising:

a) receiving said first set of graphics information upon determination, using processing circuitry, that a likelihood that a viewpoint moves from said first view region to said second view region is greater than a predetermined movement threshold, said movement determined according to user input to said client device or by a predetermined motion path; and

b) receiving said second set of graphics information upon determination, using the processing circuitry, that the likelihood that the viewpoint moves from said first view region to said second view region, said movement being determined according to the user input to said client device or by a predetermined motion path, and upon determination that user of said client device has selected to receive said second set of graphics information.

18. The method of claim 17, wherein said first of graphics information and said second set of graphics information are determined by:

a) determining, using processing circuitry, at least one first-order silhouette edge of said polygon meshes, wherein said at least one first-order silhouette edge includes;

first and second polygons sharing said at least one first-order silhouette edge,

the first polygon backfacing to each view region vertex from the plurality of view region vertices,

the second polygon front-facing to at least one view region vertex from the plurality of view region vertices, and

the first and second polygons having a backfacing orientation with respect to each other;

b) determining, using the processing circuitry, at least one supporting polygon between said second view region and at least one first-order silhouette edge;

c) constructing, using the processing circuitry, at least one wedge from said at least one supporting polygon, said at least one wedge extending away from said second view region beyond said at least one first-order silhouette edge;

d) determining, using the processing circuitry, one or more intersections of said wedges with said mesh polygons;

e) determining, using the processing circuitry, if said at least one wedge intersects at least one second first-order silhouette edge, said intersection forming a compound silhouette vertex;

f) upon determining that said at least one wedge intersects said at least one second first-order silhouette edge forming a compound silhouette vertex, forming at least one second wedge, wherein said at least one second wedge is formed by said compound silhouette vertex and at least two view region vertices from said plurality of view region vertices, said at least two view region vertices and said compound silhouette vertex forming a plane, said plane not front-facing with respect to each view region vertex from said plurality of view region vertices; and

g) determining, using the processing circuitry, said set of said mesh polygons or fragments of said mesh polygons visible from said second view region using said determined one or more intersections of said at least one wedge with said polygon meshes.