System and methods for preparing multimedia data using digital video data compression

• US 6,675,387 B1
• Filed: 04/06/1999
• Issued: 01/06/2004
• Est. Priority Date: 04/06/1999
• Status: Expired due to Term

• Current Assignee: Comcast Cable Communications Management LLC (Comcast Corporation)
• Original Assignee: Liberate Technologies
• Inventors: Qiu, Tong, McRae, Paul Ernest, Boucher, Antoine
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): WONG, ALLEN C
• Application Number:
  US09/287,235
• Time in Patent Office: 1,736 Days
• Field of Search: 725/105, 709/218, 370/487, 375/240.01
• US Class Current: 725/105
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

Systems and methods are described for preparing multimedia data using digital video data compression. Various embodiments of the invention use pre-rendering and encoding techniques that format the video image data into fat macroblocks. The fat macroblocks include header information that identifies the beginning locations of strips of macroblocks representing slices of the image, and of the macroblocks disposed within the strips. The header information enables macroblocks disposed in a strip of fat macroblocks to be individually addressed thereby providing transmission bandwidth, data processing, and data storage advantages for preparing and transmitting data for updating video images including less than full screen motion. The advantages are achieved in part because updates only require that the first macroblock of each strip of macroblocks be transmitted. Further efficiencies can be realized by using a zero displacement motion vector instead of the actual values of the first macroblock if the first macroblock is not changing. Methods using fat macroblock formatted data sets provide advantages for encoding and transmitting large bitmaps of static content, scrolling the static content, enabling animation of small areas of the display, supporting picture-in picture functionality, and allowing for overlay and combination of different image data from different sources on the display. The methods of preparing the multimedia data include pre-rendering a bitmap to create a compressed form of the bitmap, and determining a viewable area of the video images to be displayed. The viewable area of the video images can be smaller than the full-sized area of the video images.

Claims

1. In a digital video communication system including a server, a method of preparing digital video data using digital video data compression in a computer at said server for transmission to a settop box at a user terminal coupled to said server by a digital video signal distribution system, the method comprising:

pre-rendering a bitmap to create a compressed form of the bitmap, the bitmap including content of a series of video images of the digital video data, the video images having a full-sized area; and

determining a viewable area of the video images to be displayed, the viewable area of the video images is smaller than the full-sized area, wherein;

the content includes at least one of fixed content and variable content, the fixed content having a lifetime in the series of video images, and portions of the variable content having lifetimes less than the lifetime of the fixed content; and

the method further comprises encoding the content as at least one data set formatted as at least one fat macroblock and a fat macroblock header, said fat macroblock header identifying a plurality of macroblocks within said full-sized area, said plurality of macroblocks forming a fat macroblock from an array of rows and columns of macroblocks of said full-sized area.

2. The method of preparing digital video data of claim 1, including:

designating screen areas for updating;

designating macroblocks corresponding to the screen areas;

responsive to a designation for updating an entire screen, generating an intra frame from corresponding macroblock data; and

responsive to a designation for updating a subset of the screen, creating one of a predicted frame and a bi-directional frame including macroblocks required for the updating of the subset of the screen.

3. The method of preparing digital video data of claim 2, wherein:

the content includes multimedia elements; and

the method further comprises, responsive to user input, changing the content, the changing including collecting macroblocks to be changed for each element to be changed.

4. The method of preparing digital video data of claim 3, further comprising rendering the content of the video image, the rendering including:

extracting selected sections of bitmaps for variable multimedia elements;

saving the selected sections; and

rendering the selected sections into updated macroblocks.

5. The method of preparing digital video data of claim 4, wherein:

the encoding is performed according to MPEG specifications; and

the method further comprises marking the bitmap with the updated macroblocks.

6. The method of preparing digital video data of claim 5, further comprising creating one of a predicted frame and a bi-directional frame including the updated macroblocks, the creating including determining any required motion vectors for the updated macroblocks.

7. The method of preparing digital video data of claim 4, wherein the video images include animation, the animation corresponds to fat macroblocks containing each frame of an animated sequence of images.

8. The method of preparing digital video data of claim 4, further comprising

updating macroblocks for a portion of the viewable area corresponding to a picture-in-picture display, the updated macroblocks including one or more of predicted frames and bi-directional frames including images corresponding to full motion for the picture-in-picture display.

9. The method of preparing digital video data of claim 8, wherein the updating includes one or more of:

using motion vectors for panning;

completely replacing the macroblocks corresponding to the picture-in-picture display; and

using data streams transmitted in real-time from at least one live video source.

10. The method of preparing digital video data of claim 4, wherein the video images include smooth scrolling the viewable area, the viewable area provided as a screen image on at least one of the displays, the smooth scrolling including:

moving the viewable region a distance corresponding to at least one macroblock, the moving creating a missing region on the screen image;

filling the missing region with macroblock data using motion vectors; and

replacing the screen image.

11. The method of preparing digital video data of claim 4, further comprising amalgamating an image between a first frame of data and a second frame of data, the amalgamating including generating at least one bi-directional frame corresponding to values disposed between first frame values and second frame pixel values.

12. The method of preparing digital video data of claim 11, wherein the encoding provides values within the macroblocks in compressed forms with different quantization factors.

13. The method of preparing digital video data of claim 11, wherein:

the first frame comprises an intra-frame having a first decoder time stamp value;

the second frame comprises a predicted frame having a second decoder time stamp value; and

the at least one bidirectional frame includes at least one zero frame having a decoder time stamp value greater than the first time stamp value and less than the second time.

14. The method of preparing digital video data of claim 11, wherein the amalgamating includes one of fading and wiping.

15. The method of preparing digital video data of claim 2, wherein:

the content includes multimedia elements; and

the method further comprises, responsive to at least one of predetermined timing, calculated timing, and random timing, changing the content, the changing including collecting macroblocks to be changed for each multimedia element to be changed.

16. The method of preparing digital video data of claim 1, wherein:

the content includes variable multimedia elements; and

the method further comprises, when transmitting, determining the variable content to update a group of the variable multimedia elements from selected sections disposed in at least one strip of fat macroblocks, each strip represents a horizontal area of one video image in the series of video images.

17. The method of preparing digital video data of claim 16, wherein:

each macroblock represents a square array of pixels including N pixels on each side of the square array, N is an even number;

the variable multimedia elements include passive areas and active areas, the active areas include content subject to change during the lifetime of the variable content; and

pre-rendering includes formatting the active areas to ensure that no active area of a first variable element shares a macroblock with any active area of another variable multimedia element.

18. The method of preparing digital video data of claim 17, wherein the pre-rendering includes placing a border enclosing each active area, the border enclosing each active area extends outward from the corresponding active area by at least approximately N/2 pixels.

19. The method of preparing digital video data of claim 17, wherein encoding includes saving a group of the active areas as bitmap files.

20. The method of preparing digital video data of claim 19, wherein the bitmap files and fat macroblocks are tracked using a specifier, the specifier including one of a look-up table, and a markup language.

21. The method of preparing digital video data of claim 17, wherein the frames of data correspond to the active areas.

22. The method of preparing digital video data of claim 16, including:

rendering new values for the selected sections; and

re-encoding the new values into macroblocks.

23. The method of preparing digital video data of claim 1, wherein each fat macroblock includes macroblock data and said fat macroblock header, said fat macroblock header including encoding parameters for said fat macroblock.

24. The method of preparing digital video data of claim 23, further comprising indexing a group of fat macroblocks as a data set including fat macroblock information organized into strips, each strip includes an indication of a location within the strip for each macroblock disposed in the strip.

25. The method of preparing digital video data of claim 24, wherein

encoding includes compressing data disposed in the macroblocks.

26. The method of preparing digital video data of claim 25, wherein:

each strip has the same number of macroblocks; and

the macroblocks vary in size due to the compression of data disposed in the macroblocks.

27. The method of preparing digital video data of claim 25, wherein:

compressing is performed according to a compression routine;

compressing occurs for one strip of macroblocks at a time; and

upon completion of compressing of all of the strips, the compression routine provides an array of integer offsets to a renderer, each of the integer offsets corresponds to a compressed strip.

28. The method of preparing digital video data of claim 23, wherein the encoding parameters include a quantization factor.

29. The method of preparing digital video data of claim 1, wherein the frames include portions from more different fat macroblock data sets.

30. The method of preparing digital video data of claim 1, wherein the encoding is performed according to MPEG techniques.

31. The method of preparing digital video data of claim 1, wherein encoding includes compressing data disposed in the macroblocks.

32. The method of preparing digital video data of claim 1, wherein:

the pre-rendering includes separating the content into regions of the video images, the separating performed according to a directive;

data processing steps performed after pre-rendering are applied by region; and

information obtained from the data processing steps performed after pre-rendering is maintained separately by region.

33. The method of preparing digital video data of claim 32, further comprising:

determining regions having higher resolution requirements according to the directive; and

applying a different quantization factor to macroblocks corresponding to the regions having higher resolution requirements.

34. The method of preparing digital video data of claim 1, further comprising rendering, the rendering including:

extracting selected sections of bitmaps for variable elements of the video image; and

saving the selected sections, the variable elements including content subject to change during a viewing period.

35. The method of preparing digital video data of claim 34, wherein:

the content includes fixed content having static elements, the static elements include at least one of a background color, and a bitmap corresponding to a portion of the video images; and

the variable elements include at least one of a form element, and an animated graphics file.

36. The method of preparing digital video data of claim 1, wherein the display has a viewing size, and the viewing size is different than the full size area.

37. The method of preparing digital video data of claim 1, wherein:

each of the video images is represented by macroblocks of data, each of the macroblocks is related to a plurality of pixels corresponding to a geometrical portion of the video image;

the video image includes variable content having active areas in which content is subject to change as the variable content changes, and passive areas in which content does not change as the variable content changes; and

pre-rendering includes formatting the active areas to ensure that no active area of a first variable element shares a macroblock with any active area of another variable element.

38. In a digital video data communication system including a server, a method for transmitting digital video data to a settop box at a user terminal coupled to said server by a digital video signal distribution system using digital video data compression, said method comprising:

pre-rendering a bitmap to create a compressed form of the bitmap, the bitmap including content of a series of video images of the digital video data, the video images having a full-sized area;

determining a viewable area of the video images to be displayed, the viewable area is smaller than the full-sized area, and

transmitting frames of data representing portions of the video images to processing equipment, the processing equipment is connected to at least one display, wherein;

the content includes at least one of fixed content and variable content, the fixed content having a lifetime in the series of video images, and portions of the variable content having lifetimes less than the lifetime of the fixed content; and

the method including, prior to transmitting, encoding the content as at least one data set formatted as at least one fat macroblock and a fat macroblock header, said fat macroblock header identifying a plurality of macroblocks within said full-sized area, said plurality of macroblocks forming a fat macroblock from an array of rows and columns of macroblocks of said full-sized area.

39. The method for transmitting digital video data of claim 38, wherein:

the content includes multimedia elements; and

the method further comprises, when transmitting, determining the variable content to update a first group of variable elements from selected sections disposed in at least one strip of fat macroblocks, each strip represents a horizontal area of one video image in the series of video images.

40. The method for transmitting digital video data of claim 39, wherein:

each macroblock corresponds to a square array of pixels including N pixels on each side of the square array, N is an even number;

the variable elements include passive areas and active areas, the active areas including content subject to change during the lifetime of the variable content; and

pre-rendering includes formatting the active areas to ensure that no active area of a first variable element shares a macroblock with any active area of another variable element.

41. The method for transmitting digital video data of claim 38, including:

designating screen areas for updating;

designating macroblocks corresponding to the screen areas;

responsive to a designation for updating an entire screen, generating an intra frame from corresponding macroblock data; and

responsive to a designation for updating a subset of the screen, creating one of a predicted frame and a bi-directional frame including macroblocks required for the updating of the subset of the screen.

42. The method for transmitting digital video data of claim 41, wherein:

the encoding is performed according to MPEG specifications; and

the method further comprises marking the bitmap with the updated macroblocks.

43. The method for transmitting digital video data of claim 42, further comprising creating one of a predicted frame and a bi-directional frame including the updated macroblocks, the creating including determining any required motion vectors for the updated macroblocks.

44. The method for transmitting digital video data of claim 42, wherein the video images include animation, the animation corresponds to fat macroblocks containing each frame of an animated sequence of images.

45. The method for transmitting digital video data of claim 38, wherein the frames include portions from more than one fat macroblock data set.

46. In a digital video data communication system including a server and a settop box at a user terminal coupled to said server by a digital video signal distribution system, said settop box responsive to digital video signals from said server to display an image at said user terminal, a method of distributing Internet data to digital cable devices comprising:

pulling initial content from the Internet;

pre-rendering a bitmap corresponding to the initial content to create a compressed form of the bitmap, the bitmap corresponding to a series of video images of the digital video data, the video images having a full-sized area;

determining a viewable area of the video images to be displayed, the viewable area of the video images is smaller than the full-size area; and

transmitting frames of data representing portions of the video images to digital cable processing equipment, the digital cable processing equipment is connected to displays for more than one user, wherein;

the initial content includes at least one of fixed content and variable content, the fixed content having a lifetime in the series of video images, and portions of the variable content having lifetimes less than the lifetime of the fixed content; and

the method further comprises encoding the content as at least one data set formatted as at least one fat macroblock and a fat macroblock header, said fat macroblock header identifying a plurality of macroblocks within said full-sized area, said plurality of macroblocks forming a fat macroblock from an array of rows and columns of macroblocks of said full-sized area.

47. In a digital video data communication system including a server and a settop box at a user terminal coupled to said server by a digital video signal distribution system, said settop box responsive to digital video signals from said server to display an image at said user terminal, a method of using a television channel to provide Internet access comprising:

pulling initial content from the Internet;

pre-rendering a bitmap corresponding to the initial content to create a compressed form of the bitmap, the bitmap corresponding to a series of video images of the digital video data, the video images having a full-sized area;

determining a viewable area of the video images to be displayed, the viewable area of the video images is smaller than the full-size area;

storing digital video data elements including, the bitmap in a server system;

multiplexing the digital video data elements into frames of data;

transmitting frames of data from the server system to a cable network;

transmitting streams of data representing portions of the video images in a television channel to digital cable processing equipment, the digital cable processing equipment is connected to at least one display for a user, wherein;

the initial content includes at least one of fixed content and variable content, the fixed content having a lifetime in the series of video images, and portions of the variable content having lifetimes less than the lifetime of the fixed content; and

the method further comprises encoding the content as at least one data set formatted as at least one fat macroblock and a fat macroblock header, said fat macroblock header identifying a plurality of macroblocks within said full-sized area, said plurality of macroblocks forming a fat macroblock from an array of rows and columns of macroblocks of said full-sized area.

48. In a digital video data communication system including a server, said server storing a full size image, and a settop box at a user terminal coupled to said server by a digital video signal distribution system, said settop box responsive to digital video signals from said server to display an image at said user terminal, a method comprising:

receiving said full size image at said server;

storing said full size image as MPEG macroblocks at said server;

forming a fat macroblock header, said fat macroblock header identifying a plurality of MPEG macroblocks within said full size image, said plurality of MPEG macroblocks forming a fat macroblock from an array of rows and columns of MPEG macroblocks of said stored image; and

storing said fat macroblock header at said server.

49. A method in accordance with claim 48, further comprising:

determining a viewable area of said full size image, wherein said viewable area is smaller than said full size image area;

retrieving said fat macroblock header identifying said plurality of MPEG macroblocks as being within said viewable area;

assembling said plurality MPEG macroblocks identified by said fat macroblock header as being within said viewable area as an MPEG digital video signal corresponding to said viewable area of said full size image; and

transmitting said MPEG digital video signal to said settop box at said user terminal.

50. A method in accordance with claim 48, further comprising:

inputting scroll information at said user terminal;

transmitting said scroll information to said server;

determining a viewable area of said full size image corresponding to said scroll information, wherein said viewable area is smaller than said full size image area;

retrieving said fat macroblock header identifying said plurality of MPEG macroblocks as being within said viewable area;

assembling said plurality MPEG macroblocks identified by said fat macroblock header as being within said viewable area as an MPEG digital video signal corresponding to said viewable area of said full size image; and

transmitting said MPEG digital video signal to said settop box at said user terminal.

51. A method in accordance with claim 48, wherein said fat macroblock header identifies a plurality of MPEG macroblocks within said full size image, said plurality of MPEG macroblocks corresponding to static content.

52. A method in accordance with claim 48, wherein said fat macroblock header identifies a plurality of MPEG macroblocks within said full size image, said plurality of MPEG macroblocks corresponding to variable content.

53. A method in accordance with claim 52, wherein said fat macroblock header corresponding to said variable content identifies a plurality of MPEG macroblocks within said full size image, said plurality of MPEG macroblocks corresponding to an animated GIF image.

54. A method in accordance with claim 52, wherein said fat macroblock header corresponding to said variable content identifies a plurality of MPEG macroblocks within said full size image, said plurality of MPEG macroblocks corresponding to a full motion picture-in-picture video.