Network distribution and management of interactive video and multi-media containers

• US RE42,728 E1
• Filed: 05/31/2005
• Issued: 09/20/2011
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Madrane, Nabil
• Primary Examiner(s): BONSHOCK, DENNIS G
• Application Number:
  US11/140,518
• Time in Patent Office: 2,303 Days
• Field of Search: 715/716, 715/719, 715/720, 715/721, 715/723, 715/205, 715/206, 715/208
• US Class Current: 715/716
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G11B 2220/213 Read-only discs
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: Y10S 715/976 3-D Icon

Abstract

Interactive interfaces to video information provide a displayed view of a quasi-object called a root image. The root image consists of a plurality of basic frames selected from the video information, arranged such that their respective x and y directions are aligned with the x and y directions in the root image and the z direction in the root image corresponds to time, such that base frames are spaced apart in the z direction of the root image in accordance with their time separation. The displayed view of the root image changes in accordance with a designated viewing position, as if the root image were a three-dimensional object. The user can manipulate the displayed image by designating different viewing positions, selecting portions of the video information for playback and by special effects, such as cutting open the quasi-object for a better view. A toolkit permits interface designers to design such interfaces, notably so as to control the types of interaction which will be possible between the interface and an end user. Implementations of the interfaces including editors and viewers are also disclosed.

Claims

1. A method of delivering video over a network, comprising:

receiving video data representing a video sequence at a first device;

generating a hyper-media container containing including data associated with the video data;

storing the video data in a memory at the first device;

storing the hyper-media container in the memory in a primary storage format;

and

providing the video data and the hyper-media container available over the network to a remote user second device, the hyper-media container being provided in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the second device,

wherein the hyper-media container in the secondary storage format includes address information of annotation data.

2. The method as set forth in claim 1, wherein generating the hyper-media container comprises includes providing annotations to the video data.

3. The method as set forth in claim 1, wherein generating the hyper-media container includes providing segmentation data associated with the video data.

4. The method as set forth in claim 1, further comprising controlling access to the video data and the hyper-media container.

5. The method as set forth in claim 4 55, wherein controlling access comprises includes controlling access to annotations.

6. The method as set forth in claim 4 55, wherein controlling access comprises includes controlling access to annotation packs.

7. The method as set forth in claim 4 55, wherein controlling access comprises includes controlling access to versions of annotations.

8. The method as set forth in claim 1, wherein the providing the video data and the hyper-media container available to the remote user comprises second device includes publishing the video data and the hyper-media container.

9. The method as set forth in claim 1, wherein the providing the video data and the hyper-media container available to the remote user second device includes distributing at least the hyper-media container.

10. The method as set forth in claim 9, wherein the distributing comprises includes providing the hyper-media container available on-demand.

11. The method as set forth in claim 9, wherein the distributing comprises includes streaming the video data to the remote user second device over the network.

12. The method as set forth in claim 9, wherein the distributing comprises includes immerse streaming the video data to the remote user second device over the network.

13. The method as set forth in claim 9, wherein the distributing comprises includes broadcasting the hyper-media container over the network to the remote user second device.

14. The method as set forth in claim 1, further comprising indexing the video data.

15. The method as set forth in claim 1, further comprising receiving modifications to one of the hyper-media container and the video data from the remote user second device and the modifying the corresponding one of the hyper-media container and video data.

16. The method as set forth in claim 1, further comprising allowing the remote user second device to collaborate with other remote users devices on at lest least one of the video data and the hyper-media container.

17. The method as set forth in claim 16, further comprising maintaining version control of modifications to at least one of the video data and the hyper-media container.

18. The method as set forth in claim 1, wherein generating the hyper-media container comprises including includes providing an identification of a location for the video data associated with the hyper-media container.

19. The method as set forth in claim 1, wherein generating the hyper-media container comprises including includes providing an identifier for the video data associated with the hyper-media container.

20. The method as set forth in claim 1, wherein the data in the hyper-media container that is associated with the video data comprises includes an identifier for a data object.

21. The method as set forth in claim 1, wherein the receiving comprises includes receiving the video data over the network.

22. The method as set forth in claim 1, wherein the receiving comprises includes receiving the video data from a second remote user fourth device.

23. The method as set forth in claim 1, further comprising enabling the remote user second device to send the hyper-media container directly to a second remote user fourth device.

54. The method as set forth in claim 1, wherein the hyper-media container is stored and managed as a database object.

55. The method as set forth in claim 1, further comprising controlling access to the video data and the hyper-media container.

24. A method of delivering video over a network, comprising:

receiving video data representing a video sequence at a first device;

generating a hyper-media container containing including data associated with the video data;

storing the video data in a memory at the first device;

storing the hyper-media container in the memory in a primary storage format;

and

providing the video data and the hyper-media container available over the network to a remote user second device, the hyper-media container being provided in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the second device,

wherein generating the hyper-media container includes analyzing the video data and associating results of the analyzing with the hyper-media container, and the hyper-media container in the secondary storage format includes address information of annotation data.

25. The method as set forth in claim 24, wherein the analyzing comprises includes selecting an object from within the video.

26. The method as set forth in claim 24, wherein the analyzing includes extracting an object from within the video.

27. The method as set forth in claim 24, wherein the analyzing includes ranking frames of the video.

28. The methods method as set forth in claim 24, wherein the analyzing includes analyzing camera motion.

29. The method as set forth in claim 24, wherein the analyzing includes generating zooming effects.

30. The method as set forth in claim 24, wherein the analyzing includes generating scripts effects.

31. The method as set forth in claim 24, wherein the analyzing includes generating special effects.

56. The method as set forth in claim 24, wherein the hyper-media container is stored and managed as a database object.

32. A method of delivering video over a network, comprising:

receiving video data representing a video sequence at a first device;

generating a hyper-media container containing including data associated with the video data;

storing the video data in a memory at the first device;

storing the hyper-media container in the memory in a primary storage format;

providing the video data and the hyper-media container available over the network to a remote user second device, the hyper-media container being provided in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the second device;

the method further comprising;

receiving modifications to one of the hyper-media container and the video data from the remote user second device and modifying the corresponding one of the hyper-media container and video data; and

publishing versions of the modifications from the remote user second device to other remote users devices,

wherein the hyper-media container in the secondary storage format includes address information of annotation data.

57. The method as set forth in claim 32, wherein the hyper-media container is stored and managed as a database object.

33. A method of providing video data over a network comprising:

receiving video data representing a video sequence at a first device;

generating a hyper-media container including data associated with the video data;

storing the video data in a memory at the first device;

storing the hyper-media container in the memory in a primary storage format; and

providing the video data and the hyper-media container available over the network to a second device, the hyper-media container being provided in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the second device,

wherein the hyper-media container is separated from the video data and the hyper-media container in the secondary storage format includes address information of annotation data.

34. The method as set forth in claim 33, wherein the annotation data is described in terms of mark-up language.

35. The method as set forth in claim 33, wherein the annotation data is associated with the video data through an identifier.

36. The method as set forth in claim 33, wherein the annotation data is manually generated by a user.

37. The method as set forth in claim 33, wherein the annotation data is automatically generated.

38. The method as set forth in claim 33, wherein the video data is associated with one or more annotation data.

39. The method as set forth in claim 33, wherein the annotation data includes an identification of a location for the video data associated with the annotation data.

40. The method as set forth in claim 33, further comprising designating at least one object in a frame of the video data, and tracking the object in the other frames.

41. The method as set forth in claim 33, wherein the video data is received in streaming format.

42. The method as set forth in claim 33, wherein the video data is downloaded.

43. The method as set forth in claim 33, wherein the hyper-media container is stored and managed as a database object.

44. A method of providing annotation data over a network comprising:

receiving video data representing a video sequence at a first device;

generating annotation data including data associated with the video data;

storing the annotation data in a memory at the first device; and

providing the digital data and annotation data available over the network to a second device, wherein the annotation data is separated from the video data and is associated with the video data through an identifier,

wherein the annotation data is associated with a hyper-media container, the hyper-media container is stored at the first device in a primary storage format and provided to the second device in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the second device, and the hyper-media container in the secondary storage format includes address information of the annotation data.

45. The method as set forth in claim 44, wherein the annotation data is described in terms of mark-up language.

46. The method as set forth in claim 44, wherein the annotation data is manually generated by a user.

47. The method as set forth in claim 44, wherein the annotation data is automatically generated.

48. The method as set forth in claim 44, wherein the video data is associated with one or more annotation data.

49. The method as set forth in claim 44, wherein the annotation data includes an identification of a location for the video data associated with the annotation data.

50. The method as set forth in claim 44, further comprising designating at least one object in a frame of the video data, and tracking the object in the other frames.

51. The method as set forth in claim 44, wherein the video data is received in streaming format.

52. The method as set forth in claim 44, wherein the video data is downloaded.

53. The method as set forth in claim 44, wherein the hyper-media container is stored and managed as a database object.

58. An apparatus for delivering video over a network, comprising:

a receiving unit configured to receive video data representing a video sequence;

a generating unit configured to generate a hyper-media container including data associated with the video data;

a memory unit which stores the video data and the hyper-media container, the hyper-media container being stored in a primary storage format;

a providing unit configured to provide the video data and the hyper-media container available over the network to a first device, the hyper-media container being provided in a secondary storage format, the secondary storage format being a format different than the primary storage format and being a format that is readable at the first device,

wherein the hyper-media container in the secondary storage format includes address information of annotation data.