APPARATUS AND METHOD FOR PROCESSING STAGE PERFORMANCE USING DIGITAL CHARACTERS

• US 20150030305A1
• Filed: 04/12/2013
• Published: 01/29/2015
• Est. Priority Date: 04/12/2012
• Status: Abandoned Application

• Current Assignee: Dongguk University Industry-Academic Cooperation Foundation
• Original Assignee: Dongguk University Industry-Academic Cooperation Foundation
• Inventors: Moon, Bong Kyo
• Application Number:
  US14/379,952
• Publication Number:
  US 20150030305A1
• Time in Patent Office: Days
• US Class Current: 386/201
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/822 Strategy games; Role-playing games A63F13/825, A63F13/828 take precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

The present invention relates to an apparatus and method for processing a stage performance using digital characters. According to one embodiment of the present invention, an apparatus for processing a virtual video performance using a performance of an actor includes a motion input unit for receiving an input motion from the actor through a sensor attached to the body of the actor, a performance processor for creating a virtual space and reproducing a performance in real time according to a pre-stored scenario, a playable character (PC) played by the actor and acting based on the input motion, a non-playable character (NPC) acting independently without being controlled by the actor, an object, and a background being arranged and interacting with one another in the virtual space, and an output unit for generating a performance image from the performance reproduced by the performance processor and outputting the performance image to a display device.

Claims

1. An apparatus for processing a virtual video performance using a performance of an actor, the apparatus comprising:

a motion input unit for receiving an input motion from the actor through a sensor attached to the body of the actor;

a performance processor for creating a virtual space and reproducing a performance in real time according to a pre-stored scenario, a playable character (PC) played by the actor and acting based on the input motion, a non-playable character (NPC) acting independently without being controlled by the actor, an object, and a background being arranged and interacting with one another in the virtual space; and

an output unit for generating a performance image from the performance reproduced by the performance processor and outputting the performance image to a display device.

2. The apparatus according to claim 1, wherein the motion input unit comprises at least one of a sensor attached to a body part of the actor to sense a motion of the body part and a sensor marked on the face of the actor to sense a change in a facial expression of the actor.

3. The apparatus according to claim 2, wherein the motion input unit senses three-dimensional (3D) information about a motion or a facial expression of the actor, and the performance processor generates a 3D digital character controlled in response to the motion or facial expression of the actor based on the sensed 3D information.

4. The apparatus according to claim 1, wherein the performance processor guides the actor to perform a scene by providing a script of the scenario suitable for the scene to the actor in real time with the passage of time.

5. The apparatus according to claim 1, wherein the motion input unit is provided as many as the number of actors, the motion input units are electrically connected to separate sensing spaces and receive motions of the actors in the respective sensing spaces through sensors attached to the bodies of the actors in the respective sensing spaces, and the performance processor arranges a plurality of PCs played by the actors, the NPC, the object, and the background in one virtual space to generate a joint performance image of the actors.

6. The apparatus according to claim 1, further comprising an NPC processor for determining an action of the NPC based on input information of the PC and environment information about the object or the virtual space, wherein the NPC processor dynamically changes the action of the NPC in the virtual space according to an input motion from the actor or an interaction between the PC and the NPC.

7. The apparatus according to claim 6, wherein the NPC processor adaptively selects an action of the NPC based on the input information or the environment information referring to a knowledgebase of actions of the NPC, and the NPC processor determines that the selected action of the NPC matches the scenario.

8. The apparatus according to claim 1, further comprising a synchronizer for synchronizing the PC, the NPC and the object in the virtual space by providing the actor in real time with information about an interaction and relationship between the PC and the NPC or the object according to the performance of the actor.

9. The apparatus according to claim 8, wherein the interaction and relationship information comprises the magnitude of a force calculated from a logical position relationship between the PC and the NPC or the object in the virtual space and is visually provided to the actor through the display device.

10. The apparatus according to claim 8, wherein the interaction and relationship information comprises the magnitude of a force calculated from a logical position relationship between the PC and the NPC or the object in the virtual space and is provided to the actor in the form of at least one of shock or vibration through a tactile means attached to the body of the actor.

11. The apparatus according to claim 1, further comprising a communication unit having at least two separate channels, wherein a first channel of the communication unit receives a speech from the actor and is inserted into the performance, and a second channel of the communication unit is used for communication between the actor and another actor or person without being exposed in the performance.

12. An apparatus for processing a virtual video performance using a performance of an actor, the apparatus comprising:

a motion input unit for receiving an input motion from the actor through a sensor attached to the body of the actor;

a performance processor for creating a virtual space and reproducing a performance in real time according to a pre-stored scenario, a playable character (PC) played by the actor and acting based on the input motion, a non-playable character (NPC) acting independently without being controlled by the actor, an object, and a background being arranged and interacting with one another in the virtual space; and

an output unit for generating a performance image from the performance reproduced by the performance processor and outputting the performance image to a display device,

wherein the scenario comprises a plurality of scenes having at least one branch and the scenes are changed or extended by accumulating composition information thereof according to the performance of the actor or an external input.

13. The apparatus according to claim 12, wherein the performance processor guides the actor to perform a scene by providing a script of the scenario suitable for the scene to the actor in real time with the passage of time and determines a next scene of the scenario by identifying the branch based on the performance of the actor according to the selected script.

14. The apparatus according to claim 12, wherein the performance processor changes or extends the scenario by collecting a speech improvised by the actor during the performance and registering the collected speech to a database storing the script.

15. The apparatus according to claim 12, further comprising an NPC processor for determining an action of the NPC based on input information of the PC and environment information about the object or the virtual space,

wherein the NPC processor identifies the branch in consideration of an input motion from the actor or an interaction between the PC and the NPC to dynamically change the action of the NPC so as to be suitable for the identified branch.

16. A method for processing a virtual video performance using a performance of an actor, the method comprising:

receiving an input motion from the actor through a sensor attached to the body of the actor;

creating a virtual space in which a playable character (PC) played by the actor and acting based on the input motion, a non-playable character (NPC) acting independently without being controlled by the actor, an object, and a background are arranged and interact with one another;

reproducing a performance in real time in the virtual space according to a pre-stored scenario; and

generating a performance image from the performance reproduced by the performance processor and outputting the performance image to a display device.

17. The method according to claim 16, wherein the creation of a virtual space comprises determining an action of the NPC based on input information of the PC and environment information about the object or the virtual space, and dynamically changing the action of the NPC in the virtual space according to an input motion from the actor or an interaction between the PC and the NPC.

18. The method according to claim 16, wherein the reproduction of a performance in real time comprises providing the actor in real time with information about an interaction and relationship between the PC and the NPC or the object according to the performance of the actor, and synchronizing the PC, the NPC, and the object in the virtual space by visually providing the interaction and relationship information to the actor through the display device or in the form of at least one of shock and vibration through a tactile means attached to the body of the actor.