Scheme for inserting a mimicked performance into a scene and providing an evaluation of same

• US 8,177,611 B2
• Filed: 12/11/2008
• Issued: 05/15/2012
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Kang, Peter
• Primary Examiner(s): Graybill, David E
• Application Number:
  US12/333,160
• Publication Number:
  US 20090163262A1
• Time in Patent Office: 1,251 Days
• Field of Search: None
• US Class Current: 463/7
• CPC Class Codes: A63B 2024/0071 Distinction between different activities, movements, or kind of sports performed
• CPC Class Codes: A63B 2024/0096 using performance related parameters for controlling electronic or video games or avatars
• CPC Class Codes: A63B 2071/0638 Displaying moving images of recorded environment, e.g. virtual environment
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2244/10 Combat sports
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 71/0622 Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video

Abstract

A method for use in relation to a performance includes receiving a selection of a reference video segment, capturing a performance with a camera, generating a new video segment that is representative of the reference video segment having a portion thereof that is replaced with at least a portion of the captured performance, and displaying the new video segment on a display. Another method includes displaying a representation of a new video segment in the online community and providing feedback relating to the captured performance. And another method includes comparing one or more aspects of a performance with corresponding aspects of a reference performance, and providing an indication of how closely the performance matches the reference performance. A storage medium stores a computer program adapted to cause a processor based system to execute the steps of any one or more of these methods.

Claims

1. A storage medium storing a computer program adapted to cause a processor based system to execute steps comprising:

receiving a selection of a reference video segment;

capturing a performance with a camera;

generating a new video segment that is representative of the reference video segment having a portion thereof that is replaced with at least a portion of the captured performance; and

displaying the new video segment on a display;

wherein the camera and the display are coupled to the processor based system that performs the steps of receiving and generating;

wherein the reference video segment comprises a video segment from a work, and a camera view in the video segment from the work has been modified in the reference video segment.

2. The storage medium of claim 1, wherein the computer program is further adapted to cause the processor based system to execute steps comprising:

capturing audio provided by the performance with a microphone that is coupled to the processor based system; and

replacing at least a portion of original audio in the reference video segment with corresponding portions of the captured audio.

3. The storage medium of claim 1, wherein:

the reference video segment includes at least one character;

the performance comprises a performance by a user; and

the step of generating a new video segment comprises replacing at least a portion of the character with a corresponding portion of the user.

4. The storage medium of claim 1, wherein:

the work comprises one of a previously published movie, television show, television broadcast, video game, sporting event, and music video.

5. A storage medium storing a computer program adapted to cause a processor based system to execute steps comprising:

displaying a reference video segment in an online community;

receiving data from a camera capturing a performance, wherein the camera is coupled to a client device;

generating a new video segment that is representative of the reference video segment having a portion thereof that is replaced with at least a portion of the captured performance;

displaying a representation of the new video segment in the online community; and

providing to the client device feedback relating to the portion of the captured performance.

6. The storage medium of claim 5, wherein the computer program is further adapted to cause the processor based system to execute steps comprising:

receiving data from one or more additional cameras that each capture a different performance, wherein each camera is coupled to a different client device;

generating additional video segments that are each representative of the reference video segment having a portion thereof that is replaced with at least a portion of one of the captured different performances;

displaying a representation of each additional video segment in the online community; and

providing to each client device feedback relating to the portion of its respective captured performance.

7. A storage medium storing a computer program adapted to cause a processor based system to execute steps comprising:

capturing a performance with a camera;

receiving output data from the camera that is representative of the performance;

processing the output data to compare one or more aspects of the performance with corresponding aspects of a reference performance; and

providing an indication of how closely the performance matches the reference performance.

8. The storage medium of claim 7, wherein:

the performance comprises a performance by a user;

the reference performance comprises a performance by a character; and

the comparing one or more aspects of the performance with corresponding aspects of the reference performance comprises comparing gestures by the user with gestures by the character.

9. The storage medium of claim 7, wherein the computer program is further adapted to cause the processor based system to execute steps comprising:

capturing audio provided by the performance; and

wherein the comparing one or more aspects of the performance with corresponding aspects of the reference performance comprises comparing the captured audio provided by the performance with audio provided by the reference performance.

10. The storage medium of claim 7, wherein the computer program is further adapted to cause the processor based system to execute steps comprising:

generating a new video segment that is representative of the reference performance having a portion thereof that is replaced with at least a portion of the captured performance; and

modifying the portion of the captured performance in the new video segment so that the portion of the captured performance more closely matches a corresponding portion in the reference performance.