System for providing on-line virtual reality movies by transmitting partial resolution frames through a subtraction process

• US 7,814,520 B2
• Filed: 12/24/2002
• Issued: 10/12/2010
• Est. Priority Date: 02/14/1996
• Status: Expired due to Term

• Current Assignee: Distributed Media Solutions, LLC (IP Investments Group LLC)
• Original Assignee: KWOK CHU & Shindler LLC (Intellectual Ventures LLC)
• Inventors: Guedalia, Jacob Leon
• Primary Examiner(s): Lonsberry; Hunter B.
• Application Number:
  US10/330,466
• Publication Number:
  US 20030106063A1
• Time in Patent Office: 2,849 Days
• Field of Search: 725 90- 96, 725/100, 725/134
• US Class Current: 725/90
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/23106 involving caching operations prefetching while addressing of a memory level in which the access to the desired data or data block requires associative addressing means within memory systems or architectures G06F12/0862; caching at an intermediate stage in a data network H04L67/568
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/26216 involving the channel capacity, e.g. network bandwidth admission control, resource allocation in open networks H04L12/5692; flow control in packet networks H04L47/10; establishing a schedule or organising the servicing of application requests taking into account QoS H04L67/61
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

A system for transmitting digital data representing an original over plural transmission links, at least some of which have limited bandwidth, including a digital data source storing digital data representing the original, a digital data receiver receiving the digital data representing the original via one of the plural transmission links having limited bandwidth and a digital data transmitter operative to transmit the digital data representing the original to the receiver over a transmission link having a limited bandwidth in plural blocks which are sequentially transmitted at a rate determined by the limited bandwidth, each block being an incomplete collection of data which includes parts of multiple frames, each frame being viewable in a selectable order by the receiver even when less than all of the plural blocks have been received, receipt of subsequent blocks by the receiver being used to cumulatively improve the quality of the digital data viewed by the receiver.

Claims

1. A system for transmitting digital data representing an original over plural transmission links, at least some of which have limited bandwidth, comprising:

a digital data storage storing digital data representing the original;

a digital data receiver receiving said digital data representing the original via one of said plural transmission links having limited bandwidth; and

a digital data transmitter operative to transmit said digital data representing the original to said receiver over a transmission link having a limited bandwidth in plural blocks which are sequentially transmitted at a rate determined by said limited bandwidth, each block being an incomplete collection of data which includes parts of multiple frames, each frame being viewable in a selectable order by said receiver even when less than all of said plural blocks have been received, receipt of subsequent blocks by the receiver being used to cumulatively improve the quality of the digital data viewed by the receiver.

2. A system according to claim 1, wherein a first one of said plural blocks contains digital data which represents a first approximation to said original.

3. A system according to claim 2, wherein additional ones of said plural blocks, when combined with said first one of said plural blocks provide additionally accurate approximations to said original.

4. A system according to claim 1, wherein each of said multiple frames comprises a portion of data which can be independently and interactively manipulated.

5. A system according to claim 1 further comprising a block generator operative to receive digital data representing said original and to provide said plural blocks.

6. A system according to claim 5, wherein said block generator is operative to provide plural blocks which are distinguished from each other by their respective frequency bands.

7. A system according to claim 5, wherein said block generator comprises a fractal compression engine.

8. A system according to claim 7, wherein said block generator is operative to decompose said digital data representing the original in relatively high frequency and relatively low frequency digital data portions, and wherein a first plurality of blocks containing said relatively low frequency portion is transmitted by said data transmitter prior to transmission of a second plurality of blocks containing said relatively high frequency portion.

9. A system according to claim 5, wherein said block generator is operative to provide plural blocks by sampling said received digital data.

10. A system according to claim 9, wherein the sampling rate of a plurality of blocks is equal to the sum of the sampling rates of individual ones of said plurality of blocks.

11. A system according to claim 6, wherein said block generator comprises a wavelet encoder.

12. A system according to claim 5, wherein said block generator comprises a quantizer which produces blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

13. A block generator comprising:

a producer interface; and

a digital data compressor, operative in response to producer control parameters received via said producer interface for receiving digital data representing the original and providing plural blocks, each block being an incomplete collection of data which includes parts of multiple frames.

14. A block generator according to claim 13 and being operative to provide plural blocks which are distinguished from each other by their respective frequency bands.

15. A block generator according to claim 13 further comprising a fractal compression engine.

16. A block generator according to claim 15 and being operative to decompose said digital data representing the original in relatively high frequency and relatively low frequency digital data portions, and wherein a first plurality of blocks containing said relatively low frequency portion is transmitted by said data transmitter prior to transmission of a second plurality of blocks containing said relatively high frequency portion.

17. A block generator according to claim 13 and being operative to provide plural blocks by sampling said received digital data.

18. A block generator according to claim 17, wherein the sampling rate of a plurality of blocks is equal to the sum of the sampling rates of individual ones of said plurality of blocks.

19. A block generator according to claim 14 further comprising a wavelet encoder.

20. A block generator according to claim 13 further comprising a quantizer which produces blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

21. A block generator according to claim 15 further comprising a quantizer which produces blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

22. A block generator according to claim 17 further comprising a quantizer which produces blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

23. A block generator according to claim 20 further comprising a quantizer which produces blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

24. A digital data receiver comprising:

a data receipt interface for receiving digital data representing an original in a plurality of sequential blocks, each block being an incomplete collection of data which includes parts of multiple frames;

a block accumulator for combining plural blocks as they are received for viewing by the recipient; and

a viewer including a recipient interface which permits each frame to be viewed in an order selected by the recipient, even when less than all of said plural blocks have been received, combining of plural blocks by said block accumulator being used to improve the quality of the digital data viewed by said recipient.

25. A digital data receiver according to claim 24, wherein said block accumulator is operative to combine plural blocks which are distinguished from each other by their respective frequency bands.

26. A digital data receiver according to claim 24 further comprising a fractal decompression engine.

27. A digital data receiver according to claim 26, wherein said data receipt interface is operative to initially receive a first plurality of blocks containing relatively low frequency data and thereafter receive a second plurality of blocks containing relatively high frequency data and wherein said block accumulator is operative to reconstitute said digital data representing the original from said blocks representing relatively high frequency and relatively low frequency data.

28. A digital data receiver according to claim 24, wherein said block accumulator is operative to combine plural blocks having different sampling.

29. A digital data receiver according to claim 28, wherein the sampling rate of a combined plurality of blocks is equal to the sum of the sampling rates of individual ones of said plurality of blocks.

30. A digital data receiver according to claim 25 further comprising a wavelet decoder.

31. A digital data receiver according to claim 24, wherein said block accumulator comprises a dequantizer which combines blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

32. A digital data receiver according to claim 26, wherein said block accumulator comprises a dequantizer which combines blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

33. A digital data receiver according to claim 30, wherein said block accumulator comprises a dequantizer which combines blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

34. A digital data transmitter actuator comprising:

an organizer configured to access digital data representing an original that is organized in plural blocks for subsequent transmission, each block being an incomplete collection of data which includes parts of multiple frames, each frame being viewable in a selectable order by a receiver even when less than all of said plural blocks have been received; and

a receiver instruction interface responsive to interactive inputs from said receiver for actuating said organizer to select a given block and at least one given partial frame within said given block for transmission.

35. A method for transmitting digital data representing an original over plural transmission links at least some of which have limited bandwidth comprising:

storing digital data representing the original; and

transmitting said digital data representing the original to said a receiver over a transmission link selected from a set of multiple transmission links, wherein each transmission link has a limited bandwidth, in plural blocks which are sequentially transmitted at a rate determined by said limited bandwidth, each block being an incomplete collection of data which includes parts of multiple frames, each frame being viewable in a selectable order by said receiver even when less than all of said plural blocks have been received, receipt of subsequent blocks by the receiver being used to cumulatively improve the quality of the digital data viewed by the receiver.

36. A method according to claim 35, wherein a first one of said plural blocks contains digital data which represents a first approximation to said original.

37. A method according to claim 36, wherein additional ones of said plural blocks, when combined with said first one of said plural blocks provide additionally accurate approximations to said original.

38. A method according to claim 35, wherein each of said multiple frames comprises a portion of data which can be independently and interactively manipulated.

39. A method according to claim 35 further comprising generating said plural blocks.

40. A method according to claim 39, wherein said plural blocks are distinguished from each other by their respective frequency bands.

41. A method according to claim 39, wherein said generating employs fractal compression.

42. A method according to claim 41, wherein said generating includes decomposing said digital data representing the original into first plural blocks containing a relatively low frequency portion and second plural blocks containing a relatively high frequency portion, and transmitting the first plural blocks before transmitting the second plural blocks.

43. A method according to claim 39, wherein said generating provides plural blocks by sampling said received digital data.

44. A method according to claim 43, wherein the sampling rate of any set of plural blocks is equal to the sum of the sampling rates of individual ones of said plural blocks.

45. A method according to claim 40, wherein said generating comprises wavelet encoding.

46. A method according to claim 39, wherein said generating comprises quantizing into blocks each of which contain quantized data of a different order, such that accumulation of multiple blocks provides combined data of greater precision than that contained in any single block.

47. A method for digital data transmission comprising:

organizing digital data representing the original into plural blocks for subsequent transmission, each block being an incomplete collection of data which includes parts of multiple frames, each frame being viewable in a selectable order by said a receiver even when less than all of said plural blocks have been received;

responsive to interactive inputs from a receiver for actuating said organizer, selecting a given block and at least one given partial frame within said given block for transmission; and

transmitting the selected given block and at least one given partial frame.

48. A method for encoding original digital video data to be stored on a server computer for on-line delivery to client computers, comprising:

encoding the digital video into a database comprising a series of encoded data blocks, each block comprising a sequence of encoded frames, with the property that successive blocks when decoded and integrated together provide successively higher bandwidth versions of the video for online playback;

storing the database on a server computer,

processing a request by a client computer for on-line delivery of the video in order to determine which data blocks to transmit, so as to accommodate the client bandwidth; and

transmitting the necessary data blocks to the client computer thereby enabling the client computer to decode

the transmitted data blocks on the client computer, integrate

the transmitted data blocks together on the client computer to reconstruct an appropriate version of the original digital video, and

play the reconstructed video on the client computer.

49. A method according to claim 48 wherein said encoding includes a bit-rate control device enabling the producer to pre-select the sequence of bandwidths or quality levels for the database.

50. A method according to claim 48 wherein said encoding is performed in such a way that the first blocks of the database correspond to previews of the video.

51. A method according to claim 48 wherein said transmitting is repeated in succession a number of times in order to transmit additional data blocks to the client computer, thereby upgrading the quality of the video while it is replayed at the client computer.

52. A method for encoding original digital object movie data to be stored on a server computer for on-line delivery to client computers, comprising:

encoding the digital object movie into a database comprising a series of encoded data blocks, each block comprising a sequence of encoded frames, with the property that successive blocks when decoded and integrated together provide successively higher bandwidth versions of the object movie for on-line playback;

storing the database on a server computer;

processing a request by a client computer for on-line delivery of the object movie in order to determine which data blocks to transmit, so as to accommodate the client bandwidth; and

transmitting the necessary data blocks to the client computer so that the client computer can decode

the data blocks on the client computer, integrate

the data blocks together on the client computer to reconstruct an appropriate version of the original digital object movie and

play the reconstructed object movie on the client computer.

53. A method according to claim 52 wherein said encoding includes a bit-rate control device enabling the producer to pre-select the sequence of bandwidths or quality levels for the database.

54. A method according to claim 52 wherein said transmitting is repeated in succession a number of times in order to transmit additional data blocks to the client computer, thereby upgrading the quality of the object movie while it is replayed at the client computer.

55. A video processing system operative on digital video data for receiving and decoding the digital video, comprising:

a transmitter for transmitting a request for a digital video to a server so that the server can determine which data blocks to transmit so as to accommodate a client bandwidth;

a receiver for receiving data blocks associated with the digital video from the server, wherein the data blocks are part of a series of data blocks that have previously been encoded by an encoder at the server from a sequence of digital video frames with the property that successive blocks when decoded and integrated together provide successively higher bandwidth versions of the video for on-line playback;

a decoder for decompressing the data blocks back into video data on the client computer;

an accumulator for integrating the data blocks together on the client computer to reconstruct an appropriate version of the original digital video; and

a player for playing the reconstructed digital video.

56. The system as claimed in claim 55 wherein said encoder includes a bit-rate controller for enabling pre-selection of a sequence of bandwidths or quality levels for.

57. The system as claimed in claim 55 wherein said encoder compresses the digital video in such a way that a first set of blocks correspond to previews of the video.

58. The system as claimed in claim 55 wherein the transmitter, decoder, accumulator and player repeatedly operate in succession a number of times in order to receive additional data blocks, thereby upgrading the quality of the video while it is being replayed.

59. A multi-casting unit (MCU) system operative on a broadcasting network for caching of data which gets transmitted from stations to viewers, comprising:

an encoder for compressing digital multimedia data into databases comprising a series of encoded data blocks, each block comprising a sequence of encoded frames, with the property that successive blocks when decoded and integrated together provide successively higher bandwidth versions of the media for on-line playback;

station communication lines from the stations to the MCU for sending data blocks;

viewer communication lines from the MCU to the viewers for sending data;

viewer receiver units for receiving the data sent by the MCU;

storage devices for archiving the databases on a multitude of station computers;

a management unit within the MCU to process requests by viewers for on-line delivery of media stored on station computers in order to determine which data blocks to transmit, so as to accommodate the viewer bandwidth;

a transmitter for delivering the necessary data blocks on the station communication lines from the station to the MCU, and on the viewer communication lines from the MCU to the viewer receiver units;

a storage device for saving the data blocks delivered by the station communication lines in the cache residing in the MCU;

a processing unit within the MCU for processing the data blocks which the MCU receives;

a decoder for decompressing the data blocks on the viewer receiver;

an accumulator for integrating the data blocks together on the viewer receiver unit to reconstruct an appropriate version of the original digital media; and

a player for playing the reconstructed media from the viewer receiver unit.

60. The system as claimed in claim 59 wherein said management unit operates by setting inventory flags to indicate which data blocks are currently stored in the MCU.

61. The system as claimed in claim 60 wherein said management unit operates by monitoring from the stations which media data is outdated, removing from cache the blocks corresponding to the media data which is outdated, and resetting the inventory flags to indicate that the above blocks are no longer stored in the cache.

62. The system as claimed in claim 59 wherein said processing unit within the MCU comprising:

a decoder for decompressing the data blocks received;

an accumulator for integrating the data blocks together to reconstruct appropriate versions of the original digital media; and

an encoder for compressing the reconstructed media versions into an intermediate database for future transmission to the viewers.

63. A method for streaming multimedia data over a network, comprising the steps of:

encoding the media into a progressive database indexed according to frame and progressive block numbers;

serializing the encoded database;

storing the serialized database on a server; and

streaming the serialized database to a client computer upon request so that the client computer can create

a mirror copy of the encoded database on the client computer from the streamed database and

decode the encoded database into a sequence of frames for real time display.

64. A multimedia network streaming system, comprising:

an encoder for compressing the media into a progressive database indexed according to frame and progressive block numbers;

a sequencer for serializing the encoded database;

a storage device for archiving the serialized database on a server;

a transmitter for streaming the serialized database to a client upon request;

a processor for creating a mirror copy of the encoded database on the client computer from the data which streams in; and

a decoder for decompressing the encoded database on the client computer into a sequence of frames for real time display.