Dynamic modification of multimedia content

• US 6,317,795 B1
• Filed: 07/22/1997
• Issued: 11/13/2001
• Est. Priority Date: 07/22/1997
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Snible, Edward Charles, Schloss, Robert Jeffrey, Yu, Philip Shi-Lung, Malkin, Peter Kenneth, Willebeek-LeMair, Marc Hubert
• Primary Examiner(s): Geckil, Mehmet B.
• Assistant Examiner(s): VU, THONG H
• Application Number:
  US08/898,220
• Time in Patent Office: 1,575 Days
• Field of Search: 709/217, 709/225, 709/223, 709/201, 709/206, 709/203, 395/615, 455/5.1, 455/4.2, 348/5.5, 348/463, 348/151, 348/446, 380/20, 707/1, 707/181, 707/517, 707/500, 370/468, 345/335, 345/433-443, 345/473, 345/125, 345/127, 710/71, 710/5, 710/54, 434/43, 434/38, 434/20
• US Class Current: 709/246
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/6405 Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 7/162 Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

A method and system for manipulating or modifying identifiable objects in a standard broadcast or Internet-based multimedia stream according to a control specification and a content specification. Viewers and/or organizations can independently specify acceptable levels of content on multiple dimensions to satisfy the content specification while minimizing the filtering or blocking to the viewers. A “fuzz ball” control specification is provided for masking some portion of a video frame. Several fuzz ball specifications can be overlaid to address multidimensional content specifications or rating systems. The manipulation of the multimedia stream can take place at the client (set-top box or computer), intermediate node, the content server or a combination thereof. Proxy servers can modify content specifications for outgoing requests, enabling organizations to specify intranet-wide policies. Multicasting can be supported by using a single stream delivered to multiple clients, each modifying the video using a different specification. The specification to facilitate modification can be done at different granularity levels: the video, a group of frames, or individual frame level and can also be time-based. Various protocols can be used to provide the content and/or control specification, including the VBI of a standard broadcast, PICS, RTSP and MPEG protocols.

Claims

1. In a multimedia network including a multimedia stream, a method of modifying objects associated with content of the multimedia stream, comprising the steps of:

transmitting a content request including a content specification from at least one of a user and a proxy to a remotely located server, wherein the content request includes a multidimensional content specification;

requesting, from the remotely located server, a control specification from a control specification provider based on a comparison of the content request and the content of the multimedia stream;

dynamically modifying in the proxy one or more objects on one or more dimensions of the multimedia stream based on the content specification and the control specification, said dynamically modifying further comprising the step of dynamically modifying a frame of a video according to multiple control specifications and the multidimensional content specification; and

transmitting the modified multimedia stream from the proxy to the user.

2. The method of claim 1, wherein said dynamically modifying comprises the steps of:

generating a first stream including the content;

generating a second stream including the control specification for the content;

receiving a request for the content including the content specification

said dynamically modifying includes dynamically modifying the content of the first stream according to the control specification and the content specification; and

transmitting the modified multimedia stream.

3. The method of claim 2, further comprising the steps of determining and notifying a content requester of a percentage of the content that would be modified according to the content specification, in response to said receiving.

4. The method of claim 2, further comprising the step of communicating to the requester a blocking indicator, without showing the video, when the percentage exceeds a threshold.

5. The method of claim 2, wherein the control specification includes a multidimensional control specification.

6. The method of claim 1, wherein the content includes video and wherein the control specification includes a fuzz ball specification, said step of dynamically modifying further comprising the steps of:

generating the fuzz-ball specification corresponding to one or more content specifications; and

receiving a request for the content including the content specification; and

dynamically overlaying at least a part of a frame of the video based on the fuzz-ball specification and the content specification, in response to said receiving.

7. The method of claim 6, wherein the content specification and the control specification include a PICS protocol, said method further comprising the steps of:

generating separate fuzz-ball specifications corresponding to different content specifications; and

selecting a fuzz-ball specification based on a PICS specification.

8. The method of claim 6, wherein the content specification is time-based.

9. The method of claim 6, wherein said generating comprises the step of

generating the fuzz-ball specififcation as one or more fuzz balls having a size, location, and temporal relation to a frame of the video.

10. The method of claim 6, further comprising the step of overlaying multiple fuzz ball specifications to meet the content specification.

11. The method of claim 1, said dynamically modifying further comprising the step of combining multiple content specifications covering one of multiple dimensions and rating systems.

12. The method of claim 11, wherein the content includes video, further comprising the step of overlaying multiple fuzz-ball filters, in response to said combining step.

13. The method of claim 1, wherein the content includes video, further comprising the step of dynamically modifying a frame of the video according to the multiple control specifications and the multidimensional content specification.

14. The method of claim 13, wherein the content includes video, said dynamically modifying the frame of the video further comprising the steps of masking a frame of the video according to the multiple control specifications and the multidimensional content specification.

15. The method of claim 13, wherein said dynamically modifying further comprises the step of dynamically modifying the frame of the video at one or more of:

a content server, a client, a set top box; and

a proxy node.

16. The method of claim 1, further comprising the step of communicating one of the content specifications and the control specification according to one of:

a PICS protocol;

a RSTP protocol; and

an MPEG protocol.

17. The method of claim 16, wherein the content includes video and the PICS protocol includes a plurality of PICS labels, further comprising the steps of:

said communicating including communicating a V label indicating a content rating of a video and an overlay label to indicate the effect of a modification to the content rating; and

updating a category value of the V label, in response to said dynamically modifying.

18. The method of claim 1, wherein the network comprises a World Wide Web including a hierarchy of servers, further comprising the step of:

an intermediate proxy server modifying content specifications for an outgoing content request.

19. The method of claim 18, wherein the hierarchy includes a heterogeneous proxy hierarchy wherein said modifying is not performed by the client or all servers in the hierarchy.

20. The method of claim 1, further comprising the steps of:

multicasting a single multimedia stream to multiple requesters; and

rendering the video by said requesters, at least two of said requesters rendering the video according to different content specifications.

21. The method of claim 20, further comprising the steps of:

generating one or more separate fuzz ball specifications for the different content specifications; and

selecting one or more fuzz ball specifications according to a PICS protocol.

22. The method of claim 1, said dynamically modifying further comprising the step of dynamically bypassing, masking, blocking and substituting objects.

23. The method of claim 22, wherein said content includes video, further comprising the step of substituting one or more frames or segments of the video with one or more alternative frames or segments.

24. The method of claim 22 wherein the content comprises video, further comprising the step of skipping one of video frames and video segments based on the control specification and the content specification.

25. The method of claim 22, wherein the content includes video, further comprising one or more steps of:

masking or blocking some portion of one of a video frame and a video stream at one of a server, proxy, set top box, and client.

26. The method of claim 1, said dynamically modifying further comprising the step of generating a second stream including the control specification for the content;

wherein the control specification is generated at one of the video header, a group of frames of the video, and an individual frame level.

27. The method of claim 1, further comprising the steps of communicating the control specification and the content as a single stream.

28. The method of claim 27, wherein the content includes video and wherein said step of communicating comprises the step of communicating the control specification during the vertical blanking interrupt of the multimedia stream.

29. The method of claim 27, wherein said step of communicating comprises the step of associating a frame label with one or more frames of the video.

30. The method of claim 1, further comprising the steps of:

multicasting a single multimedia stream to multiple requesters; and

each requester rendering the video using a at least two different content specifications.

31. The method of claim 1, wherein the objects include one of an identifiable object in the multimedia stream;

at least a portion of a frame of video; and

a sample of audio.

32. The method according to claim 1, wherein the transmitting step transmits the multimedia stream through at least one proxy.

33. A computer program product comprising:

a computer usable medium having computer readable program code means embodied therein for modifying objects associated with the content of a multimedia stream, the computer readable program code means in said computer program product comprising;

computer readable program code means for causing the computer to effect, receiving a content request, by a remotely located server, including a content specification from at least one of a user and a proxy, wherein the content request includes a multidimensional content specification;

computer readable program code means for causing the computer to effect, requesting, from the remotely located server, a control specification from a control specification provider based on a comparison of the content request and the content of the multimedia stream;

computer readable program code means for causing the computer to effect, dynamically modifying at the proxy one or more objects on one or more dimensions of the multimedia stream based on the content specification and the control specification, said dynamically modifying further comprising the step of dynamically modifying a frame of a video according to multiple control specifications and the multidimensional content specification; and

computer readable program code means for causing the computer to effect, transmitting the modified multimedia stream from the proxy to the user.

34. The computer program product of claim 33, wherein said computer readable program code means for causing the computer to effect dynamically modifying comprises:

computer readable program code means for causing the computer to effect, generating a first stream including the content;

computer readable program code means for causing the computer to effect, generating a second stream including the control specification for the content;

computer readable program code means for causing the computer to effect, receiving a request for the content including the content specification;

computer readable program code means for causing the computer to effect said dynamically modifying includes computer readable program code means for causing the computer to effect dynamically modifying the content of the first stream according to the control specification and the content specification; and

computer readable program code means for causing the computer to effect, transmitting the modified multimedia stream.

35. The computer program product of claim 34, further comprising computer readable program code means for causing the computer to effect, determining and notifying a content requester of a percentage of the content that would be modified according to the content specification, in response to said receiving.

36. The computer program product of claim 34, further comprising computer readable program code means for causing the computer to effect, communicating to the requester a blocking indicator, without showing the video, when the percentage exceeds a threshold.

37. The computer program product of claim 34, wherein the control specification includes a multidimensional control specification.

38. The computer program product of claim 33, wherein the content includes video and wherein the control specification includes a fuzz ball specification, said computer readable program code means for causing the computer to effect, dynamically modifying further comprising:

computer readable program code means for causing the computer to effect, generating the fuzz-ball specification corresponding to one or more content specifications;

computer readable program code means for causing the computer to effect, receiving a request for the content including the content specification; and

computer readable program code means for causing the computer to effect, dynamically overlaying at least a part of a frame of the video based on the fuzz-ball specification and the content specification, in response to said receiving.

39. The computer program product of claim 38, wherein the content specification and the control specification include a PICS protocol, said computer program product further comprising computer readable program code means for causing the computer to effect, generating separate fuzz-ball specifications corresponding to different content specifications;

and computer readable program code means for causing the computer to effect, selecting a fuzz-ball specification based on a PICS specification.

40. The computer program product of claim 39, wherein said computer readable program code means for causing the computer to effect, generating comprises computer readable program code means for causing the computer to effect, generating the fuzz-ball specififcation as one or more fuzz balls having a size, location, and temporal relation to a frame of the video.

41. The computer program product of claim 38, wherein the content specification is time-based.

42. The computer program product of claim 38, further comprising computer readable program code means for causing the computer to effect, overlaying multiple fuzz ball specifications to meet the content specification.

43. The computer program product of claim 33, said computer readable program code means for causing the computer to effect, dynamically modifying further comprising computer readable program code means for causing the computer to effect, combining multiple content specifications covering one of multiple dimensions and rating systems.

44. The computer program product of claim 43, wherein the content includes video, further comprising computer readable program code means for causing the computer to effect, overlaying multiple fuzz-ball filters, in response to said computer readable program code means for causing the computer to effect said combining.

45. The computer program product of claim 33, further comprising computer readable program code means for causing the computer to effect, communicating one of the content specifications and the control specification according to one of:

a PICS protocol;

a RSTP protocol; and

an MPEG protocol.

46. The computer program product of claim 45, wherein the content includes video and the PICS protocol includes a plurality of PICS labels, further comprising computer readable program code means for causing the computer to effect said communicating including computer readable program code means for causing the computer to effect, communicating a V label indicating a content rating of a video and an overlay label to indicate the effect of a modification to the content rating;

and

computer readable program code means for causing the computer to effect, updating a category value of the V label, in response to said computer readable program code means for causing the computer to effect dynamically modifying.

47. The method of claim 33, wherein the content includes video, further comprising computer readable program code means for causing the computer to effect, dynamically modifying a frame of the video according to the multiple control specifications and the multidimensional content specification.

48. The computer program product of claim 47, wherein the content includes video, said computer readable program code means for causing the computer to effect dynamically modifying the frame of the video further comprising computer readable program code means for causing the computer to effect, masking a frame of the video according to the multiple control specifications and the multidimensional content specification.

49. The computer program product of claim 47, wherein said computer readable program code means for causing the computer to effect dynamically modifying further comprises computer readable program code means for causing the computer to effect, dynamically modifying the frame of the video at one or more of:

a content server, a client, a set top box; and

a proxy node.

50. The computer program product of claim 33, wherein the network comprises a World Wide Web including a hierarchy of servers, further comprising computer readable program code means for causing the computer to effect an intermediate proxy server to modify content specifications for an outgoing content request.

51. The computer program product of claim 50, wherein the hierarchy includes a heterogeneous proxy hierarchy wherein said computer readable program code means for causing the computer to effect modifying is not performed by the client or all servers in the hierarchy.

52. The computer program product of claim 33, further comprising:

computer readable program code means for causing the computer to effect, multicasting a single multimedia stream to multiple requesters; and

computer readable program code means for causing the computer to effect, rendering the video by said requesters, at least two of said requesters rendering the video according to different content specifications.

53. The computer program product of claim 52, further comprising:

computer readable program code means for causing the computer to effect, generating one or more separate fuzz ball specifications for the different content specifications; and

computer readable program code means for causing the computer to effect, selecting one or more fuzz ball specifications according to a PICS protocol.

54. The computer program product of claim 33, said computer readable program code means for causing the computer to effect dynamically modifying further comprising computer readable program code means for causing the computer to effect, dynamically bypassing, masking, blocking and substituting objects.

55. The computer program product of claim 54, wherein said content includes video, further comprising computer readable program code means for causing the computer to effect, substituting one or more frames or segments of the video with one or more alternative frames or segments.

56. The computer program product of claim 54 wherein the content comprises video, further comprising computer readable program code means for causing the computer to effect, skipping one of video frames and video segments based on the control specification and the content specification.

57. The computer program product of claim 54, wherein the content includes video, further comprising computer readable program code means for causing the computer to effect one or more of:

masking or blocking some portion of one of a video frame and a video stream at one of a server, proxy, set top box, and client.

58. The computer program product of claim 33, said computer readable program code means for causing the computer to effect dynamically modifying further comprising computer readable program code means for causing the computer to effect, generating a second stream including the control specification for the content;

wherein the control specification is generated at one of the video header, a group of frames of the video, and an individual frame level.

59. The computer program product of claim 33, further comprising computer readable program code means for causing the computer to effect, communicating the control specification and the content as a single stream.

60. The computer program product of claim 59, wherein the content includes video and wherein said computer readable program code means for causing the computer to effect communicating comprises computer readable program code means for causing the computer to effect communicating the control specification during the vertical blanking interrupt of the multimedia stream.

61. The computer program product of claim 59, wherein computer readable program code means for causing the computer to effect, communicating comprises computer readable program code means for causing the computer to effect, associating a frame label with one or more frames of the video.

62. The computer program product of claim 33, further comprising:

computer readable program code means for causing the computer to effect, multicasting a single multimedia stream to multiple requesters; and

computer readable program code means for causing the computer to effect, each requester rendering the video using a at least two different content specifications.

63. The computer program product of claim 33, wherein the objects include one of an identifiable object in the multimedia stream;

at least a portion of a frame of video; and

a sample of audio.

64. The computer program product of claim 33, wherein the computer readable program code effects transmitting the modified multimedia stream through at least one proxy.

65. In a multimedia network including a multimedia stream, a method of modifying objects associated with content of the multimedia stream, comprising the steps of:

(a) transmitting a content request including a content specification from at least one of a user and a proxy to a remotely located server, wherein the content request includes a multidimensional content specification;

(b) determining, in the remotely located server, whether the content of the multimedia stream satisfies the content specification;

(c) requesting, from the remotely located server, a control specification from a control specification provider based on the result of step (b);

(d) transmitting the control specification to the proxy responsive to the request of step (c);

(e) transmitting the multimedia stream to the proxy;

(f) modifying, in the proxy, one or more objects on one or more dimensions of the multimedia stream based on the content specification and the control specification, said modifying further comprising the step of dynamically modifying a frame of a video according to multiple control specifications and the multidimensional content specification; and

(g) transmitting the modified multimedia stream to a further proxy.

66. The multimedia network according to claim 65, wherein the further proxy is the user.

67. The multimedia network according to claim 65, further including the step:

(a1) transmitting a further content specification from a further proxy to the server based on a further content request, the server combing the content specification and the further content specification.

68. In a multimedia network including a multimedia stream, a method of modifying objects associated with content of the multimedia stream, comprising the steps of:

(a) transmitting a content request including a content specification from at least one of a user and a proxy to a remotely located server, wherein the content request includes a multidimensional content specification;

(b) determining, in the remotely located server, whether the content of the multimedia stream satisfies the content specification;

(c) requesting, from the remotely located server, a control specification from a control specification provider based on the result of step (b);

(d) transmitting the control specification from the control specification provider to the user responsive to the request of step (c);

(e) transmitting the multimedia stream from the server to the user; and

(f) modifying, in the user, one or more objects on one or more dimensions of the multimedia stream based on the content specification and the control specification, said modifying further comprising the step of dynamically modifying a frame of a video according to multiple control specifications and the multidimensional content specification.