CAPTURING VIEWS AND MOVEMENTS OF ACTORS PERFORMING WITHIN GENERATED SCENES

• US 20110181601A1
• Filed: 01/22/2010
• Published: 07/28/2011
• Est. Priority Date: 01/22/2010
• Status: Abandoned Application

• Current Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America (Sony Group Corp.)
• Inventors: Murrant, David, Mumbauer, Michael
• Application Number:
  US12/692,518
• Publication Number:
  US 20110181601A1
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

Generating scenes for virtual environment of a visual entertainment program, comprising: capturing views and movements of an actor performing within the generated scenes, comprising: tracking movements of a headset camera and a plurality of motion capture markers worn by the actor within a physical volume of space; translating the movements of the headset camera into head movements of a virtual character operating within the virtual environment; translating the movements of the plurality of motion capture markers into body movements of the virtual character; generating first person point-of-view shots using the head and body movements of the virtual character; and providing the generated first person point-of-view shots to the headset camera worn by the actor.

Claims

1. A method of generating scenes for virtual environment of a visual entertainment program, comprising:

capturing views and movements of an actor performing within the generated scenes, comprising;

tracking movements of a headset camera and a plurality of motion capture markers worn by the actor within a physical volume of space;

translating the movements of the headset camera into head movements of a virtual character operating within the virtual environment;

translating the movements of the plurality of motion capture markers into body movements of the virtual character;

generating first person point-of-view shots using the head and body movements of the virtual character; and

providing the generated first person point-of-view shots to the headset camera worn by the actor.

2. The method of claim 1, wherein the visual entertainment program is a video game.

3. The method of claim 1, wherein the visual entertainment program is a motion picture.

4. The method of claim 1, wherein the movements of the headset camera are tracked by computing positions and orientations of the headset camera.

5. The method of claim 4, wherein the positions of the headset camera are tracked using position trackers positioned within the physical volume of space.

6. The method of claim 4, wherein the orientations of the headset camera are tracked using at least one of accelerometers and gyroscopes attached to the headset camera.

7. The method of claim 1, wherein translating the movements of the headset camera into head movements of a virtual character to generate the first person point-of-view shots comprises

translating the movements of the headset camera into changes in fields of view of the virtual character operating within the virtual environment.

8. The method of claim 1, wherein generating scenes for a visual entertainment program comprises

performing a motion capture session in which views and movements of an actor performing within the generated scenes are captured.

9. The method of claim 8, further comprising

performing multiple motion capture sessions to select a take that provides best camera movement and angle.

10. The method of claim 9, wherein the multiple motion capture sessions are recorded so that each session can be evaluated and compared.

11. The method of claim 1, wherein providing the generated first person point-of-view shots to the headset camera comprises

feeding back the first person point-of-view shots to a display of the headset camera.

12. The method of claim 1, further comprising

storing the generated first person point-of-view shots in a storage unit for a later use.

13. A method of capturing views and movements of an actor performing within generated scenes, comprising:

tracking positions and orientations of an object worn on a head of the actor within a physical volume of space;

tracking positions of motion capture markers worn on a body of the actor within a physical volume of space;

translating the positions and orientations of the object into head movements of a virtual character operating within virtual environment;

translating the positions of the plurality of motion capture markers into body movements of the virtual character; and

generating first person point-of-view shots using the head and body movements of the virtual character.

14. The method of claim 13, further comprising

providing the generated first person point-of-view shots to a display of the object worn on the head of the actor.

15. The method of claim 13, wherein the virtual environment in which the virtual character is operating comprises

virtual environment generated for a video game.

16. The method of claim 13, wherein the virtual environment in which the virtual character is operating comprises

hybrid environment generated for a motion picture in which virtual scenes are integrated with live action scenes.

17. A system of generating scenes for virtual environment of a visual entertainment program, comprising:

a plurality of position trackers configured to track positions of a headset camera object and a plurality of motion capture markers worn by an actor performing within a physical volume of space;

an orientation tracker configured to track orientations of the headset camera object;

a processor including a storage medium storing a computer program comprising executable instructions that cause the processor to;

receive a video file including virtual environment;

receive tracking information about the positions of the headset camera object and the plurality of motion capture markers from the plurality of position trackers;

receive tracking information about the orientations of the headset camera object from the orientation tracker;

translate the positions and the orientations of the headset camera object into head movements of a virtual character operating within the virtual environment;

translate the positions of the plurality of motion capture markers into body movements of the virtual character;

generate first person point-of-view shots using the head and body movements of the virtual character; and

provide the generated first person point-of-view shots to the headset camera object worn by the actor.

18. The system of claim 17, wherein the visual entertainment program is a video game.

19. The system of claim 17, wherein the visual entertainment program is a motion picture.

20. The system of claim 17, wherein the orientation tracker comprises

at least one of accelerometers and gyroscopes attached to the headset camera object.

21. The system of claim 17, wherein the processor including executable instructions that cause the processor to translate the positions and the orientations of the headset camera object into head movements of a virtual character comprises executable instructions that cause the processor to

translate the positions and the orientations of the headset camera object into changes in fields of view of the virtual character operating within the virtual environment.

22. The system of claim 17, further comprising

a storage unit for storing the generated first person point-of-view shots for a later use.

23. The system of claim 17, wherein the processor is a game console configured to receive inputs from the headset camera object, the plurality of position tracker, and the orientation tracker.

24. The system of claim 17, wherein the headset camera object includes a display for displaying the provided first person point-of-view shots.

25. A computer-readable storage medium storing a computer program for generating scenes for virtual environment of a visual entertainment program, the computer program comprising executable instructions that cause a computer to:

receive a video file including virtual environment;

receive tracking information about the positions of the headset camera object and the plurality of motion capture markers from the plurality of position trackers;

receive tracking information about the orientations of the headset camera object from the orientation tracker;

translate the positions and the orientations of the headset camera object into head movements of a virtual character operating within the virtual environment;

translate the positions of the plurality of motion capture markers into body movements of the virtual character;

generate first person point-of-view shots using the head and body movements of the virtual character; and

provide the generated first person point-of-view shots to the headset camera object worn by the actor.

26. The storage medium of claim 25, wherein executable instructions that cause a computer to translate the positions and the orientations of the headset camera object into head movements of a virtual character to generate the first person point-of-view shots comprise executable instructions that cause a computer to

translate the positions and the orientations of the headset camera object into changes in fields of view of the virtual character operating within the virtual environment.

27. The storage medium of claim 25, wherein executable instructions that cause a computer to provide the generated first person point-of-view shots to the headset camera object comprise executable instructions that cause a computer to

feed back the first person point-of-view shots to a display of the headset camera object.

28. The storage medium of claim 25, further comprising executable instructions that cause a computer to

store the generated first person point-of-view shots in a storage unit for a later use.