CENTRALIZED DATABASE FOR 3-D AND OTHER INFORMATION IN VIDEOS

• US 20120102023A1
• Filed: 10/25/2010
• Published: 04/26/2012
• Est. Priority Date: 10/25/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven, Stamate, Vlad
• Granted Patent: US 9,542,975 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/036 Insert-editing

Abstract

Methods and systems for a centralized database for 3-D and other information in videos are presented. A centralized database contains video metadata such as camera, lighting, sound, object, depth, and annotation data that may be queried for and used in the editing of videos, including the addition and removal of objects and sounds. The metadata stored in the centralized database may be open to the public and admit contributor metadata.

Claims

1. A method comprising:

receiving video data to be displayed on a display, the video data comprising a video identifier;

querying a server, with the video identifier, for video metadata stored on the server, the video identifier identifying the video metadata and the video metadata comprising video lighting location metadata describing a location of a light source with respect to a camera that captured the video data;

receiving the video metadata from the server and editing the video data with the received video lighting location metadata to produce edited video data that relights the video data consistent with the video lighting location metadata; and

outputting the edited video data to the display.

2. A method as in claim 1 wherein the video metadata comprises lighting data, sound data, camera data, object segmentation data, depth map data, composition data, and annotation data.

3. A method as in claim 1 wherein the editing of the video data removes an object from the video data and where the video lighting location metadata is applied to determine a background behind the removed object that is consistent with the lighting location metadata for the remainder of the video data.

4. A method as in claim 1 wherein the editing of the video data adds an object to the video data and where the video lighting location metadata is applied to the added object so that it appears as if it was lighted by the light source.

5. A method as in claim 1 wherein the editing of the video data relights the video data by analyzing the video lighting location metadata to light the video data with regards to the light source.

6. A method as in claim 2 wherein the editing of the video data analyzes the camera data and location data to combine the video data captured by two or more cameras for the same scene to create a pair of stereoscopic images to present a three dimensional image.

7. A method as in claim 2 wherein the editing of the video removes a sound from the video and where the video metadata is applied so that the removed sound is consistent with the sound data.

8. A method as in claim 3 wherein object segmentation data, depth map data, and annotation data is used to identify the object to be removed.

9. A system comprising:

a player receiving video data to be displayed on a display, the video data comprising a video identifier, querying a server with the video identifier for video metadata, receiving the video metadata from the server and editing the video data with the received video lighting location metadata to produce edited video data that relights the video data consistent with the video lighting location metadata;

a server storing the video metadata, the video metadata comprising video lighting location metadata describing a location of a light source with respect to a camera capturing the video data; and

a display for displaying the edited video.

10. A method as in claim 9 wherein the video metadata comprises lighting data, sound data, camera data, object segmentation data, depth map data, and annotation data.

11. A method as in claim 9 wherein the player edits the video data to remove an object from the video data and where the video lighting location metadata is applied to determine a background behind the removed object that is consistent with the lighting location metadata for the remainder of the video data.

12. A method as in claim 9 wherein the player edits the video data to add an object to the video data and where the video lighting location metadata is applied to the added object so that it appears as if it was lighted by the light source.

13. A method as in claim 9 wherein the player relights the video data by analyzing the video lighting location metadata to light the video data with regards to the light source.

14. A method as in claim 10 wherein the player edits the video data by analyzing the camera data and location data to combine the video data captured by two or more cameras for the same scene to create a pair of stereoscopic images to present a three dimensional image.

15. A method as in claim 10 wherein the player edits a movie to remove a sound from the movie and where the video metadata is applied so that the removed sound is consistent with the sound data.

16. A method as in claim 11 wherein the player analyzes object segmentation data, depth map data, and annotation data to identify the object to be removed.

17. A method comprising:

receiving from a first contributor a first metadata contribution for a video;

storing the first metadata contribution in a video metadata database;

receiving from a second contributor a second metadata contribution for a video, the second metadata contribution being received after the first metadata contribution;

determining that the second metadata contribution is more accurate than the first metadata contribution in describing the video; and

replacing at least a portion of the first metadata contribution stored on the video metadata database with the second metadata contribution.

18. A method of claim 17 wherein the first contributor and the second contributor are internet users.

19. A method of claim 17 wherein the metadata stored in the video metadata database is accessible to internet users.

20. A method of claim 17 wherein the metadata stored in the video metadata database comprises camera data, lighting data, sound data, object segmentation data, depth map data, and annotation data.