Method and system for real-time rendering within a gaming environment

• US 7,285,047 B2
• Filed: 10/17/2003
• Issued: 10/23/2007
• Est. Priority Date: 10/17/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Malzbender, Thomas, Gelb, Daniel G.
• Primary Examiner(s): Thai; Xuan M.
• Assistant Examiner(s): THOMASSON, MEAGAN J
• Application Number:
  US10/688,148
• Publication Number:
  US 20050085296A1
• Time in Patent Office: 1,467 Days
• Field of Search: 463/7, 463 30- 34, 463/35, 463/42, 463/1, 463/36, 345418-422, 345/581, 345/585
• US Class Current: 463/31
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: G06T 15/02 Non-photorealistic rendering

Abstract

A method and system for real-time rendering within a gaming environment. Specifically, one embodiment of the present invention discloses a method of rendering a local participant within an interactive gaming environment. The method begins by capturing a plurality of real-time video streams of a local participant from a plurality of camera viewpoints. From the plurality of video streams, a new view synthesis technique is applied to generate a rendering of the local participant. The rendering is generated from a perspective of a remote participant located remotely in the gaming environment. The rendering is then sent to the remote participant for viewing.

Claims

1. A method of real-time rendering in a gaming environment to create an interactive experience, comprising:

capturing a plurality of real-time video streams of a local participant from a plurality of camera viewpoints;

generating real-time renderings of said local participant based on said plurality of real-time video streams by applying a new view synthesis technique, said real-time renderings taken from a perspective of a remote participant located remotely in said gaming environment, wherein said new view synthesis technique comprises an image-based visual hull technique that comprises approximating a visual hull of said local participant by projecting contours associated with said local participant into three-dimensional space and computing an intersection of resulting frusta; and

sending said real-time renderings to said remote participant for viewing within said gaming environment.

2. The method of claim 1, further comprising:

incorporating said real-time renderings comprising real-time photo-realistic features of said local participant into a graphically generated avatar, said graphically generated avatar representing said local participant in said gaming environment; and

sending said graphically generated avatar incorporating said real-time renderings to said local participant for viewing within said gaming environment.

3. The method of claim 2, further comprising:

directing movement of said graphically generated avatar according to said local participant'"'"'s eye gaze.

4. The method as described in claim 1, wherein said generating real-time renderings of said local participant further comprises:

generating a three-dimensional geometric model of said local participant from said plurality of real-time video streams as said real-time renderings.

5. The method as described in claim 1, wherein said generating real-time renderings of said local participant further comprises:

rendering said real-time renderings from a second location of said remote participant with respect to a first location of said local participant within a coordinate space of said gaming environment.

6. The method as described in claim 5, further comprising:

enabling a change in at least one of said first and second locations by allowing said local participant and said remote participant to navigate through said coordinate space; and

modifying said real-time renderings to reflect said change in at least one of said first and second locations.

7. The method as described in claim 1, further comprising:

blending said real-time renderings within a synthetic rendering of said gaming environment; and

displaying said real-time renderings that are blended within said gaming environment at a display viewable by said second participant to enable real-time video communication for interactive gaming between said local participant and said remote participant within said gaming environment.

8. The method as described in claim 1, wherein said gaming environment comprises a three dimensional gaming environment.

9. The method as described in claim 1, further comprising:

personifying said graphically generated avatar to said local participant by illustrating real-time emotion of said local participant through said video image stream comprising representative said real-time photo-realistic features of said local participant.

10. The method as described in claim 1, further comprising:

generating other real-time renderings of said local participant based on said plurality of real-time video streams by applying said new view synthesis technique, said other real-time renderings taken from a perspective of another participant located remotely in said gaming environment; and

sending said other real-time renderings to said another participant for viewing within said gaming environment.

11. The method of claim 1, further comprising:

obtaining parameters using a single uncalibrated target to calibrate cameras used for said capturing.

12. A method of real-time rendering in a gaming environment to create an interactive experience, comprising:

capturing a plurality of real-time video streams of a local participant from a plurality of camera viewpoints;

generating a video image stream in real-time of said local participant based on said plurality of real-time video streams by applying a new view synthesis technique, said video image stream rendered from a perspective of a remote participant located remotely in said gaming environment, wherein said new view synthesis technique comprises an image-based visual hull technique that comprises approximating a visual hull of said local participant by projecting contours associated with said local participant into three-dimensional space and computing an intersection of resulting frusta; and

incorporating said video image stream comprising real-time photo-realistic features of said local participant into a graphically generated avatar, said graphically generated avatar representing said local participant in said gaming environment.

13. The method as described in claim 12, wherein said generating said video image stream further comprises:

rendering said video image stream from a second location of said remote participant with respect to a first location of said local participant within a coordinate space of said gaming environment.

14. The method as described in claim 13, further comprising:

enabling a change in at least one of said first and second locations by allowing said local participant and said remote participant to navigate through said coordinate space; and

modifying said video image stream to reflect said change in at least one of said first and second locations.

15. The method as described in claim 12, further comprising:

blending said graphically generated avatar incorporating said video image stream within a synthetic rendering of said gaming environment; and

displaying said graphically generated avatar incorporating said video image stream within said gaming environment at a display viewable by said remote participant to enable real-time video communication for interactive gaming between said local participant and said remote participant within said gaming environment.

16. The method as described in claim 12, further comprising:

personifying said graphically generated avatar to said local participant by illustrating real-time emotion of said local participant through said video image stream comprising representative said real-time photo-realistic features of said local participant.

17. The method as described in claim 12, wherein said generating said video image stream further comprises:

generating a three dimensional geometric model of said local participant based on said plurality of real time video streams by applying said new view synthesis technique; and

generating said video image stream of said local participant from said three dimensional geometric model.

18. The method as described in claim 12, further comprising:

generating a real-time audio output from audio of said local participant to enable real-time audio communication in said gaming environment between said local participant and said remote participant.

19. The method as described in claim 12, further comprising:

varying a total of said plurality of camera viewpoints when capturing said plurality of real-time video streams of said local participant to vary image quality in said plurality of video image streams.

20. The method as described in claim 12, further comprising:

performing non-photorealistic processing of said plurality of video image streams resulting in a non-photorealistic cartoon rendering of said plurality of video image streams.