Method and apparatus for supporting storage of multiple camera views

• US 20060047674A1
• Filed: 09/01/2004
• Published: 03/02/2006
• Est. Priority Date: 09/01/2004
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Tabatabai, Ali, Visharam, Mohammed Zubair
• Application Number:
  US10/932,502
• Publication Number:
  US 20060047674A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Metadata describing multimedia data associated with multiple camera views of a scheme is created. Further, the metadata describing the multimedia data associated with multiple camera views is stored in separate metadata tracks of a media format file. Each of the separate metadata tracks corresponds to one of the multiple camera views of the scene.

Claims

1. A method comprising:

creating metadata describing multimedia data associated with a plurality of camera views of a scene; and

storing the metadata describing the multimedia data associated with the plurality of camera views of the scene in separate metadata tracks of a media format file, each of the separate metadata tracks corresponding to one of the plurality of camera views of the scene.

2. The method of claim 1 wherein the media format file has an International Organization for Standardization (ISO) base file format.

3. The method of claim 1 further comprising:

assigning a unique identifier to each of the separate metadata tracks; and

associating the unique identifier to a camera position used for a corresponding one of the plurality of camera views.

4. The method of claim 1 wherein each of the separate metadata tracks comprises a random access table with a list of random access frames within multimedia data of a corresponding one of the plurality of camera views.

5. The method of claim 1 wherein contents of random access tables in the separate metadata tracks enable switching between the plurality of camera views.

6. The method of claim 4 wherein each of the separate metadata tracks further comprises a timestamp table storing a timestamp for all frames including the random access frames from the list.

7. A method comprising:

receiving a request to switch from a first camera view of a scene to a second camera view of the scene at run time;

identifying a current frame of the first camera view;

accessing a metadata track associated with the second camera view to find a random access frame in the second camera view that is close in time to the current frame of the first camera view; and

switching, at run time, to the second camera view at the found random access frame.

8. The method of claim 7 wherein the request to switch identifies a camera position used for the second camera view.

9. The method of claim 7 wherein the metadata track associated with the second camera view stores metadata describing multimedia data pertaining to the second camera view.

10. The method of claim 9 wherein the metadata comprises a random access table providing a list of random access frames within the multimedia data pertaining to the second camera view.

11. The method of claim 10 wherein the metadata further comprises a timestamp table providing a timestamp for all frames including the random access frames in the list.

12. The method of claim 7 wherein the request to switch pertains to a real-time mode.

13. The method of claim 12 wherein accessing the metadata track associated with the second camera view comprises:

searching a random access table associated with the second camera view for a random access frame following the current frame of the first camera view;

determining whether any intermediate frames exist between the current frame of the first camera view and the following random access frame of the second camera view; and

switching to the second camera view at the following random access frame if no intermediate frames exist between the current frame of the first camera view and the following random access frame of the second camera view.

14. The method of claim 13 further comprising:

decoding frames of the second camera view, starting with the following random access frame; and

presenting the decoded frames to a user.

15. The method of claim 13 further comprising:

determining that at least one intermediate frame exists between the current frame of the first camera view and the following random access frame of the second camera view;

searching the random access table associated with the second camera view for a random access frame preceding the current frame of the first camera view; and

switching to the second camera view at the preceding random access frame.

16. The method of claim 15 further comprising:

decoding frames of the second camera view, starting with the preceding random access frame;

refraining from presenting decoded frames to a user until reaching a timestamp of the current frame of the first camera view; and

upon reaching the timestamp of any one of the current frame and a frame following the current frame, beginning presentation of decoded frames to the user.

17. The method of claim 7 wherein the request to switch pertains to a pause mode.

18. The method of claim 17 wherein accessing the metadata track associated with the second camera view comprises:

searching a random access table associated with the second camera view for a random access frame matching the current frame of the first camera view; and

if a matching random access frame exists in the random access table, switching to the second camera view at the matching random access frame.

19. The method of claim 18 further comprising:

decoding the matching random access frame; and

presenting the decoded frame to a user.

20. The method of claim 18 further comprising:

determining that a matching random access frame does not exist in the random access table associated with the second camera view;

searching the random access table associated with the second camera view for a random access frame preceding the current frame of the first camera view;

decoding intermediate frames of the second camera view, starting with the preceding random access frame and until reaching a frame that matches the current frame of the first camera view;

decoding the matching frame of the second camera view; and

presenting the decoded matching frame to a user.

21. An apparatus comprising:

a metadata generator to create metadata describing multimedia data associated with a plurality of camera views of a scene; and

a file creator to form a media format file storing the metadata describing the multimedia data associated with the plurality of camera views of the scene in separate metadata tracks, each of the separate metadata tracks corresponding to one of the plurality of camera views of the scene.

22. The apparatus of claim 21 wherein the media format file has an International Organization for Standardization (ISO) base file format.

23. The apparatus of claim 21 wherein the file creator is further to assign a unique identifier to each of the separate metadata tracks, and to associate the unique identifier to a camera position used for a corresponding one of the plurality of camera views.

24. The apparatus of claim 21 wherein each of the separate metadata tracks comprises a random access table with a list of random access frames within multimedia data of a corresponding one of the plurality of camera views.

25. The apparatus of claim 21 wherein contents of random access tables in the separate metadata tracks enable switching between the plurality of camera views.

26. The apparatus of claim 24 wherein each of the separate metadata tracks further comprises a timestamp table storing a timestamp for all frames including the random access frames from the list.

27. An apparatus comprising:

a request receiver to receive a request to switch from a first camera view of a scene to a second camera view of the scene at run time, and to identify a current frame of the first camera view; and

a media data stream processor to access a metadata track associated with the second camera view to find a random access frame in the second camera view that is close in time to the current frame of the first camera view, and to switch, at run time, to the second camera view at the found random access frame.

28. The apparatus of claim 27 wherein the request to switch identifies a camera position used for the second camera view.

29. The apparatus of claim 27 wherein the metadata track associated with the second camera view stores metadata describing multimedia data pertaining to the second camera view.

30. The apparatus of claim 29 wherein the metadata comprises a random access table providing a list of random access frames within the multimedia data pertaining to the second camera view.

31. The apparatus of claim 30 wherein the metadata further comprises a timestamp table providing a timestamp for all frames including the random access frames in the list.

32. The apparatus of claim 27 wherein the request to switch pertains to a real-time mode.

33. The apparatus of claim 32 wherein the media data stream processor is to access the metadata track associated with the second camera view by searching a random access table associated with the second camera view for a random access frame following the current frame of the first camera view, determining whether any intermediate frames exist between the current frame of the first camera view and the following random access frame of the second camera view, and switching to the second camera view at the following random access frame if no intermediate frames exist between the current frame of the first camera view and the following random access frame of the second camera view.

34. The apparatus of claim 33 wherein the media data stream processor is further to decode frames of the second camera view, starting with the following random access frame, and to present the decoded frames to a user.

35. The apparatus of claim 33 wherein the media data stream processor is further to determine that at least one intermediate frame exists between the current frame of the first camera view and the following random access frame of the second camera view, to search the random access table associated with the second camera view for a random access frame preceding the current frame of the first camera view, and to switch to the second camera view at the preceding random access frame.

36. The apparatus of claim 35 wherein the media data stream processor is further to decode frames of the second camera view, starting with the preceding random access frame, to refrain from presenting decoded frames to a user until reaching a timestamp of the current frame of the first camera view, and to begin presentation of decoded frames to the user upon reaching the timestamp of any one of the current frame and a frame following the current frame.

37. The apparatus of claim 27 wherein the request to switch pertains to a pause mode.

38. The apparatus of claim 37 wherein the media data stream processor is to access the metadata track associated with the second camera view by searching a random access table associated with the second camera view for a random access frame matching the current frame of the first camera view, and switching to the second camera view at a matching random access frame if the matching random access frame exists in the random access table.

39. The apparatus of claim 38 wherein the media data stream processor is further to decode the matching random access frame, and to present the decoded frame to a user.

40. The apparatus of claim 38 wherein the media data stream processor is further to determine that a matching random access frame does not exist in the random access table associated with the second camera view, to search the random access table associated with the second camera view for a random access frame preceding the current frame of the first camera view, to decode intermediate frames of the second camera view, starting with the preceding random access frame and until reaching a frame that matches the current frame of the first camera view, to decode the matching frame of the second camera view, and to present the decoded matching frame to a user.

41. An apparatus comprising:

means for creating metadata describing multimedia data associated with a plurality of camera views of a scene; and

means for storing the metadata describing the multimedia data associated with the plurality of camera views of the scene in separate metadata tracks of a media format file, each of the separate metadata tracks corresponding to one of the plurality of camera views of the scene.

42. The apparatus of claim 41 wherein the media format file has an International Organization for Standardization (ISO) base file format.

43. The apparatus of claim 41 wherein each of the separate metadata tracks comprises a random access table with a list of random access frames within multimedia data of a corresponding one of the plurality of camera views.

44. An apparatus comprising:

means for receiving a request to switch from a first camera view of a scene to a second camera view of the scene at run time;

means for identifying a current frame of the first camera view;

means for accessing a metadata track associated with the second camera view to find a random access frame in the second camera view that is close in time to the current frame of the first camera view; and

means for switching, at run time, to the second camera view at the found random access frame.

45. The apparatus of claim 44 wherein the request to switch identifies a camera position used for the second camera view.

46. The apparatus method of claim 44 wherein the metadata track associated with the second camera view stores metadata describing multimedia data pertaining to the second camera view.

47. A computer readable medium that provides instructions, which when executed on a processor cause the processor to perform a method comprising:

creating metadata describing multimedia data associated with a plurality of camera views of a scene; and

storing the metadata describing the multimedia data associated with the plurality of camera views of the scene in separate metadata tracks of a media format file, each of the separate metadata tracks corresponding to one of the plurality of camera views of the scene.

48. The computer readable medium of claim 47 wherein the media format file has an International Organization for Standardization (ISO) base file format.

49. The computer readable medium of claim 47 wherein each of the separate metadata tracks comprises a random access table with a list of random access frames within multimedia data of a corresponding one of the plurality of camera views.

50. A computer readable medium that provides instructions, which when executed on a processor cause the processor to perform a method comprising:

receiving a request to switch from a first camera view of a scene to a second camera view of the scene at run time;

identifying a current frame of the first camera view;

accessing a metadata track associated with the second camera view to find a random access frame in the second camera view that is close in time to the current frame of the first camera view; and

switching, at run time, to the second camera view at the found random access frame.

51. The computer readable medium of claim 50 wherein the request to switch identifies a camera position used for the second camera view.

52. The computer readable medium of claim 50 wherein the metadata track associated with the second camera view stores metadata describing multimedia data pertaining to the second camera view.

53. A system comprising:

a memory; and

at least one processor coupled to the memory, the processor executing a set of instructions which cause the processor to

create metadata describing multimedia data associated with a plurality of camera views of a scene, and

to store the metadata describing the multimedia data associated with the plurality of camera views of the scene in separate metadata tracks of a media format file, each of the separate metadata tracks corresponding to one of the plurality of camera views of the scene.

54. The system of claim 53 wherein the media format file has an International Organization for Standardization (ISO) base file format.

55. The system of claim 53 wherein each of the separate metadata tracks comprises a random access table with a list of random access frames within multimedia data of a corresponding one of the plurality of camera views.

56. A system comprising:

a memory; and

at least one processor coupled to the memory, the processor executing a set of instructions which cause the processor to

receive a request to switch from a first camera view of a scene to a second camera view of the scene at run time,

to identify a current frame of the first camera view,

to access a metadata track associated with the second camera view to find a random access frame in the second camera view that is close in time to the current frame of the first camera view, and

to switch, at run time, to the second camera view at the found random access frame.

57. The system of claim 56 wherein the request to switch identifies a camera position used for the second camera view.

58. The system of claim 56 wherein the metadata track associated with the second camera view stores metadata describing multimedia data pertaining to the second camera view.