Method and system for encoding movies, panoramas and large images for on-line interactive viewing and gazing

• US 6,721,952 B1
• Filed: 03/07/1997
• Issued: 04/13/2004
• Est. Priority Date: 08/06/1996
• Status: Expired due to Term

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: ROXIO Incorporated (MelodyVR Group Plc)
• Inventors: Guedalia, Jacob Leon, Guedalia, Isaac David
• Primary Examiner(s): Srivastava, Vivek
• Application Number:
  US08/813,181
• Time in Patent Office: 2,594 Days
• Field of Search: H04/N.2173, H04/N.5445, G06/F.300, G06/F 1.300, 725/38, 725/86, 725/89, 725/44, 725/40, 725/140, 725/132, 346/578, 346/580, 346/581, 346/588, 346/589, 345/783, 345/788
• US Class Current: 725/38
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94

Abstract

This invention discloses apparatus and a method for producing a moving picture including providing a digital image including the content of a multiplicity of individual images defining a view window which may be selectably superimposed over at least a portion of the digital image at a given time, and moving the view window over the digital image such as to view at least portions of the multiplicity of the individual images in a desired sequence thereby to define the moving picture.

Claims

1. A method for producing a moving picture comprising:

providing a digital image including the content of a multiplicity of individual images, the multiplicity of individual images comprising a plurality of individual images of an object in different orientations;

defining a view window which may be selectably superimposed over at least a portion of said digital image at a given time; and

moving said view window over said digital image such as to view at least portions of said multiplicity of said individual images in a desired sequence thereby to define said moving picture,

wherein said view window is superimposed sequentially over a portion of each of said plurality of individual images, so as to view generally the same portion of said object in each of said plurality of individual images, and

whereby moving said view window over said plurality of individual images in different orientations provides a sensation of viewing movement of said object.

2. Apparatus for producing a moving picture comprising:

a digital image generator providing a digital image including the content of a multiplicity of individual images, the multiplicity of individual images comprising a plurality of individual images of an object in different orientations;

a view window which may be selectably superimposed over at least of portion of said digital image at a given time; and

a user operated window controller for moving said view window over said digital image such as to view at least portion of said multiplicity of said individual images in a desired sequence thereby to define said moving picture,

wherein said view window is superimposed sequentially over a portion of each of said plurality of individual images so as to view generally the same portion of said object in each of said plurality of individual images, and

whereby moving said view window over said plurality of individual images in different orientation provides a sensation of viewing movement of said object.

3. A digital image display system comprising:

a memory storing digital images in an interactively accessible data structure, which stores pre-manipulated image content in multiple image content units, wherein said data structure joins all of the image content units together into a large digital image, and wherein the large digital image comprises a movie and each image content unit constitutes a frame of the movie; and

a downloader downloading data stored in said memory in a manner such that any of the stored image content units may be accessed at at least a minimum quality prior to completion of downloading.

5. A digital image display system according to claim 3 and also comprising a user-operated controller for advancing frames in the movie by moving a view window within the large digital image.

6. A digital image display system according to claim 5 and wherein said view window moves in such a way that it maintains the same relative position within each frame within the movies, so that advancement of the view window makes it seem as if the frames are being advanced.

4. A digital image display system comprising:

a memory storing digital images in an interactively accessible data structure, which stores pre-manipulated image content in multiple image content units, wherein said data structure joins all of the image content units together into a large digital image, and

wherein the large digital image comprises a large still image and each image content unit constitutes a tile of the large still image at a given resolution, said multiple image content units comprising multiple at least partially redundant tiles of said large still image at varying resolutions; and

a downloader downloading data stored in said memory in a manner such that any of the stored image content units may be accessed at at least a minimum quality prior to completion of downloading.

7. A digital image display system according to claim 4 and also comprising a user-operated controller for user selectable gazing in said large still image by moving a view window within the large digital image.

8. A digital image display method comprising:

storing digital images in an interactively accessible data structure, which stores pre-manipulated image content in multiple image content units, wherein said data structure joins all of the image content units together into a large digital image, and wherein the large digital image comprises a movie and each image content unit constitutes a frame of the movie; and

downloading said digital images in a manner such that any of the stored image content units may be accessed at at least a minimum quality prior to completion of downloading.

10. A digital image display method according to claim 8 and also comprising advancing frames in the movie by moving a view window within the large digital image.

11. A digital image display method according to claim 10 and wherein said view window moves in such a way that it maintains the same relative position within each frame within the movies, so that advancement of the view window makes it seem as if the frames are being advanced.

9. A digital image display method comprising:

storing digital images in an interactively accessible data structure, which stores pre-manipulated image content in multiple image content units, wherein said data structure joins all of the image content units together into a large digital image, and wherein the large digital image comprises a large still image and each image content unit constitutes a tile of the large still image at a given resolution, said multiple image content units comprising multiple at least partially redundant tiles of said large still image at varying resolutions; and

downloading said digital images in a manner such that any of the stored image content units may be accessed at at least a minimum quality prior to completion of downloading.

12. A digital image display method according to claim 9 and also comprising user selectable gazing in said large still image by moving a view window within the large digital image.

13. A method for simulating an interactive movie on a computer display from a sequence of digital frames, as data is received on-line, comprising the steps of:

joining the digital frames together to form a large digital image;

encoding said large digital image to generate encoded image data;

storing said encoded image data;

transmitting said encoded image data to a client computer upon request;

decoding said encoded image data to generate a reconstructed large digital image; and

displaying moving window views of said reconstructed large digital image in a fixed display window, in response to keyboard presses and mouse clicks by a user, to exhibit the same effect as if advancing forward and backward through the digital frames, in order to simulate interactive motion on said client computer display device.

14. The method according to claim 13, wherein said steps of joining, encoding, storing, transmitting, and decoding arc applied to a plurality of digital frame sequences, corresponding to a plurality of movies, linked together through hot spots.

15. The method according to claim 14, wherein said step of storing includes storing hot spot locations and link information, along with said encoded image data.

16. The method according to claim 15, wherein said step of transmitting includes transmitting hot spot locations and link information along with the plurality of said encoded image data.

17. The method according to claim 16, wherein said step of displaying also uses said hot spot locations, and further comprises the steps of:

processing mouse clicks by the user in hot spot locations to link to another of said plurality of reconstructed large digital images; and

moving a window view from a position within a first large digital image to a position within a second large digital image linked thereto at a hot spot.

18. The method according to claim 17, wherein a portion of said second large digital image is stored within the encoded data for the first large digital image, and wherein said moving a window view comprises the steps of:

moving a window view from a position within the first large digital image to another position within said first large digital image corresponding to the location of the portion of said second large digital image contained therein; and

moving a window view from said position within said first large digital image to a corresponding position within said second large digital image.

19. The method according to claim 13, wherein said large digital image is a long horizontal image containing all of the individual movie frames.

29. A method according to claim 13 wherein said encoding and decoding operate according to a progressive JPEG algorithm.

20. An image processing system, operative on a sequence of individual digital frames to produce an on-line interactive movie, comprising:

a stitching unit for assembling the digital frames together to form a large digital image;

an encoder for compressing said large digital image into encoded image data;

a storage device for archiving said encoded image data;

a transmitter for delivering said encoded image data to a client computer upon request;

a decoder for decompressing said encoded image data, to generate a reconstructed large digital image; and

a monitor for displaying moving window views of said reconstructed large digital image in a fixed display window, in response to keyboard presses and mouse clicks by a user, exhibiting the same effect as if advancing forward and backward through said digital frames, in order to simulate interactive motion on said client computer display device.

21. The system as claimed in claim 20, wherein said stitching unit, encoder, storage device, transmitter and decoder operate on a plurality of digital frame sequences, corresponding to a plurality of movies, linked together through hot spots.

22. The system as claimed in claim 21, wherein said storage device archives the location and link information of said hot spots along with the plurality of said encoded image data.

23. The system as claimed in claim 22, wherein said transmitter delivers the hot spot and link information, along with said encoded image data.

24. The system as claimed in claim 23, wherein said monitor uses hot spot locations to display moving window views, by means of:

a processing unit for processing mouse clicks by the user in hot spot locations in order to link to another of said plurality of reconstructed large digital images; and

a pointer in memory for moving said window view from a position within a first large digital image to a position within a second large digital image which is linked thereto at said hot spot locations.

25. The system as claimed in claim 24, wherein a portion of said second large digital image is stored within the encoded data for said first large digital image, and wherein said pointer in memory comprises:

a first pointer in memory for moving said window view from a position within said first large digital image to another position within said first large digital image corresponding to the location of the portion of said second large digital image contained therein; and

a second pointer in memory for moving said window view from said position within said first large digital image to a corresponding position within second large digital image.

26. The system as claimed in claim 20, wherein said large digital image is a long horizontal image containing individual movie frames.

30. An image processing system according to claim 20 wherein said encoder and decoder operate according to a progressive JPEG algorithm.

27. A method for simulating interactive audio from a sequence of digital audio clips, as data is received on-line, comprising the steps of:

joining said digital clips together to form a long digital signal;

encoding said long signal to generate encoded audio data;

storing said encoded audio data;

transmitting said encoded audio data to a client computer upon request;

decoding said encoded audio data to generate a reconstructed long digital signal; and

playing changing segments of said reconstructed long signal, in response to keyboard presses and mouse clicks by a user, to exhibit the same effect as if advancing through said digital audio clips.

28. An audio processing system operative on a sequence of digital audio clips, to produce on-line interactive audio, said system comprising:

a stitching unit for joining said digital audio clips together to form a long digital signal;

an encoder for compressing said long digital signal into encoded audio data;

a storage device for archiving said encoded audio data;

a transmitter for delivering said encoded audio data to a client computer upon request;

a decoder for decompressing said encoded audio data to generate a reconstructed long digital signal; and

a sound player for playing changing segments of said reconstructed signal, in response to keyboard presses and mouse clicks by a user, exhibiting the same effect as if advancing through said digital audio clips.

31. A method for simulating an interactive three-dimensional object movie or panorama on a computer display, as data is received on-line, comprising the steps of:

arranging a sequence of individual digital frames from an object movie or panorama video strip in a sequence from top to bottom to form a long vertical image;

compressing said long vertical image;

storing the compressed data;

transmitting the compressed data to a client computer;

decompressing the compressed data on-line to reconstruct said long vertical image on the client computer, while the compressed data is being transmitted; and

progressively displaying selected window views to simulate interactive object motion or panorama navigation on the client computer display device, as the data for the complete construction of these views becomes available.

32. A method according to claim 31 and also comprising the steps of:

separating the individual frames from the long vertical image on the client computer, as they become available; and

joining the individual frames into a large digital image on the client computer and wherein;

said step of progressively displaying operating by displaying sliding window views of said large digital image.

33. The method according to claim 31, wherein the compressing of said long vertical image is performed by encoding horizontal bands of image data in succession.

34. The method according to claim 33 wherein said compressing is performed by a JPEG encoder.

35. The method according to claim 31, wherein the compressing of said long vertical image is carried out in a progressive mode, resulting in a series of compressed data slices.

36. The method according to claim 35, wherein the compressing is performed by a progressive mode JPEG encoder.

37. The method according to claim 31, wherein said storing of the compressed data creates a single stream database of compressed image data on the server, and the transmission of said compressed data uses a single data channel.

38. The method according to claim 31, wherein said storing of the compressed data creates a multiple stream database of compressed image data on the server, and the transmission of the compressed data uses multiple data channels.

39. The method according to claim 31, wherein the decompressing of the compressed data is performed by decoding horizontal bands of image data in succession.

40. The method according to claim 39, wherein said decompressing is performed by a JPEG decoder.

41. The method according to claim 31, wherein the decompressing of the compressed data is carried out in a progressive mode, resulting in a series of decompressed data slices.

42. The method according to claim 41, wherein said decompressing is performed by a progressive mode JPEG decoder.

43. The method according to claim 41, wherein the decompressing of said compressed data comprises the steps of decompressing the individual compressed image slices, and integrating the slices as they become available so as to obtain successively enhanced video frames.

44. The method according to claim 32, wherein said progressively displaying is carried out by a viewing window which jumps from frame to frame, as each successive frame is made available.

45. The method according to claim 32, wherein said progressively displaying is carried out by a viewing window which moves smoothly from frame to frame, as each successive frame is made available.

55. Apparatus according to claim 44, wherein said decompressor comprises a JPEG decoder.

46. Apparatus for simulating an interactive three-dimensional object movie or panorama on a computer display, as data is received on-line, comprising:

a frame composer arranging a sequence of individual digital frames from an object movie or panorama video strip in a sequence from top to bottom to form a long vertical image;

a compressor compressing said long vertical image;

a memory storing the compressed data;

a transmitter transmitting the compressed data to a client computer;

a decompressor decompressing the compressed data on-line to reconstruct said long vertical image on the client computer, while the compressed data is being transmitted; and

a display driver progressively displaying selected window views to simulate interactive object motion or panorama navigation on the client computer display device, as the data for the complete construction of these views becomes available.

47. Apparatus according to claim 46 and also comprising:

a frame separator separating the individual frames from the long vertical image on the client computer, as they become available; and

a stitching device joining the individual frames into a large digital image on the client computer and wherein;

said display driver operates by displaying sliding window views of said large digital image.

48. Apparatus according to claim 46, wherein the compressor operates on said long vertical image by encoding horizontal bands of image data in succession.

49. Apparatus according to claim 48 wherein said compressor comprises a JPEG encoder.

50. Apparatus according to claim 46, wherein said compressor operates on said long vertical image in a progressive mode, resulting in a series of compressed data slices.

51. Apparatus according to claim 50, wherein said compressor comprises a progressive mode JPEG encoder.

52. Apparatus according to claim 46, wherein said memory includes a single stream database of compressed image data on the server, and the transmitter employs a single data channel.

53. Apparatus according to claim 46, wherein said memory includes a multiple stream database of compressed image data on the server, and the transmitter employs multiple data channels.

54. Apparatus according to claim 46, wherein said decompressor operates by decoding horizontal bands of image data in succession.

56. Apparatus according to claim 46, wherein said decompressor operates on the compressed data in a progressive mode, resulting in a series of decompressed data slices.

57. Apparatus according to claim 56, wherein said decompressor comprises a progressive mode JPEG decoder.

58. Apparatus according to claim 56, wherein said decompressor operates on said compressed data by decompressing the individual compressed image slices, and integrating the slices as they become available so as to obtain successively enhanced video frames.

59. Apparatus according to claim 47, wherein said display driver operates by causing a viewing window to jump from frame to frame, as each successive frame is made available.

60. Apparatus according to claim 47, wherein said display driver operates by causing a viewing window to move smoothly from frame to frame, as each successive frame is made available.

61. An image processing system operative on an object movie or panorama film strip for providing interactive on-line simulation of a three-dimensional object movie or panorama comprising:

an A/D converter for converting the film strip into digital data;

a production unit for converting the digital data into a long vertical digital image;

an encoder for compressing said long vertical image so that it can be efficiently stored on a server computer storage device and further transmitted to client computers;

a transmitter for sending the compressed image data as a data stream to a client computer upon request; and

a decoder for decompressing the data stream on the client computer into image data, and arranging the image data into a form suitable for interactive display on-line, as the data is received.

62. The system as claimed in claim 61, wherein said encoder comprises a progressive compressor and a bit rate controller, for producing compressed image slices.

63. The system as claimed in claim 62, wherein said decoder comprises a progressive decompressor for the compressed slices and integrator for accumulating the slices into image data.

64. A method for producing interactive media comprising:

providing a digital file including content of a multiplicity of individual interactively accessible digital content units;

storing said digital file in an interactively accessible data structure;

downloading said digital file in a manner such that any of the stored multiplicity of individual interactively accessible digital content units may be accessed at at least a minimum quality prior to completion of downloading;

defining a pointer which may be selectably pointed at at least a portion of said digital file at a given time; and

moving said pointer over said digital file so as to access at least portions of said multiplicity of said individual interactively accessible digital content units in a desired sequence.

65. Apparatus for producing interactive media, said apparatus comprising:

a digital file generator providing a digital file including the content of a multiplicity of interactively accessible digital content units;

a memory storing said digital file in an interactively accessible data structure;

a downloader downloading data stored in said memory in a manner such that any of the stored multiplicity of individual interactively accessible digital content units may be accessed at at least a minimum quality prior to completion of downloading;

a pointer which may be selectably pointed to at least a portion of said digital file at a given time; and

a user-operated pointer controller for moving said pointer over said digital image such as to view at least potions of said multiplicity of said interactively accessible digital content units in a desired sequence.