Moving image composing system

• US 6,332,003 B1
• Filed: 11/12/1998
• Issued: 12/18/2001
• Est. Priority Date: 11/11/1997
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Kato, Hitoshi, Kato, Tsuyoshi, Yoshio, Hiroaki, Maruya, Kensuke, Hayakawa, Yoshihiro, Matsuura, Shun, Asai, Kayoko, Kageyu, Satoshi
• Primary Examiner(s): Le, Vu
• Application Number:
  US09/189,868
• Time in Patent Office: 1,132 Days
• Field of Search: 375/240.23, 375/240.24, 375/240.26, 375/240.29, 375/240.16, 375/240.1, 348/420.1, 348/425.1, 348/425.3, 348/575, 348/580-581, 382/245-246, 382/276, 382/284
• US Class Current: 375/240.23
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence

Abstract

A moving image composing system for displaying reduced and encoded images at arbitrary positions on a multi-image raster. The moving image composing system comprises a moving image reducing unit for reducing a moving image inputted, a moving image encoding unit for encoding the reduced moving image to produce an encoded moving image, an accumulating unit for storing and accumulating the encoded image, and a moving image composing unit for combining more-than-one encoded moving images accumulated to produce a multi-image combined moving image. The moving image encoding unit performs the encoding processing to facilitate the change of positional information the reduced moving image has, in consideration with the case that the change of the position information occurs due to the disposition of the reduced moving image at its combination. When there is a need to change the positional information of the reduced moving image due to the combination processing, the moving image composing unit changes the positional information to positional information required for the combination processing.

Claims

1. A moving image composing system comprising:

moving image receiving means for receiving a moving image;

moving image reducing means for reducing said moving image to produce a reduced moving image;

moving image encoding means for encoding said reduced moving image to produce an encoded moving image;

accumulating means for storing and accumulating said encoded moving image;

moving image composing means for combining at least one encoded moving image accumulated in said accumulating means into a multi-image combined moving image;

moving image decoding means for decoding said multi-image combined moving image; and

moving image displaying means for displaying the decoded moving image,

wherein said moving image encoding means conducts the encoding on the basis of positional information said reduced moving image has to facilitate a change of said positional information in consideration with a case that a variation of said positional information occurs due to a disposition of said reduced moving image at the moving image combination, and

said moving image composing means changes said positional information to positional information required for its combining processing when a need for the change of the positional information said reduced moving image has takes place.

2. A moving image composing system comprising:

moving image receiving means for receiving a moving image;

moving image reducing means for reducing said moving image to produce a thumbnail moving image;

moving image encoding means for encoding said thumbnail moving image according to an MPEG standard to produce an encoded thumbnail moving image;

accumulating means for storing and accumulating said encoded thumbnail moving image;

moving image composing means for combining one or more encoded moving images accumulated in said accumulating means into a multi-image combined moving image;

moving image decoding means for decoding said multi-image combined moving image; and

moving image displaying means for displaying the decoded moving image,

wherein said moving image encoding outputs a supplementary code string for making up for a wanted bit in consideration with a case that the number of bits of an MB_Addr_Inc varies due to a change of a horizontal coordinate of a slice, and

said moving image composing means rewrites a value of MB_Addr_Inc by supplying the wanted bit from said supplementary code string and produces a multi-image combined moving image code string.

3. A moving image composing system as defined in claim 2, wherein said moving image encoding means and said moving image composing means utilize, as said supplementary code string for making up for the wanted bit, extra_bit_slice and extra_information_slice of a slice header section in an MPEG video sequence.

4. A moving image composing system as defined in claim 3, wherein said moving image encoding means outputs 8 sets of extra_bit_slice and extra_information_slice as said supplementary code string, and said moving image composing means supplies an arbitrary number of wanted bits from said supplementary code string, and produces a macroblock_escape code string.

5. A moving image composing system as defined in claim 2, wherein said moving image composing means includes a layout table for, when a multi-image raster is divided in macroblock size into areas, retaining multi-image layout information on each of said areas, and layout table making means for drawing up said layout table, while said moving image composing means refers to said layout table to determine an encoded thumbnail moving image code string to be outputted as said multi-image combined moving image code string.

7. A moving image composing system as defined in claim 5, wherein said moving image encoding means and said moving image composing means utilize, as said supplementary code string to be used for making said head of said macroblock code string come into contact with said byte border, extra_bit_slice and extra_information_slice of a slice header section in an MPEG video sequence.

8. A moving image composing system as defined in claim 5, wherein said moving image encoding means and said moving image composing means inserts, as said supplementary code string to be used for making said end of said macroblock code string come into contact with said byte border, macroblock_stuffing in an MPEG video sequence into a head portion of said macroblock code string.

6. A moving image composing system comprising:

moving image receiving means for receiving a moving image;

moving image reducing means for reducing said moving image to produce a thumbnail moving image;

moving image encoding means for encoding said thumbnail moving image according to an MPEG standard to produce an encoded thumbnail moving image;

accumulating means for storing and accumulating said encoded thumbnail moving image;

moving image composing means for combining one or more encoded moving images accumulated in said accumulating means into a multi-image combined moving image;

moving image decoding means for decoding said multi-image combined moving image; and

moving image displaying means for displaying the decoded moving image,

wherein said moving image encoding means inserts a supplementary code string so that a head and end of each of macroblock code strings come into contact with a byte border, and

said moving image composing means successively arranges said macroblock code strings in a raster scanning order to produce a desired multi-image combined moving image code string.

9. A moving image composing system comprising:

moving image receiving means for receiving a multi-image combined moving image;

multi-image combined moving image dividing means for extracting a slice code string of each of image area sections from a code string of said multi-image combined moving image;

image layout table making means for drawing up an image layout table which retains a size of each of image areas in a multi-image raster and positional information in a corresponding relation;

moving image re-encoding means for converting said slice code string into a re-encoded slice code string which is allowed to be re-combined within a different multi-image raster;

accumulating means for storing and accumulating encoded thumbnail images;

moving image composing means for re-combining said re-encoded slice code strings to produce a multi-image combined moving image code string providing a different image construction;

moving image listing means for drawing up a list of a combination and order of moving images constituting a multi-image combined moving image;

moving image decoding means for decoding said multi-image combined moving image; and

moving image displaying means for displaying the decoded moving image,

wherein said multi-image combined moving image dividing means fetches an identification code of an encoded thumbnail moving image, positional information on said encoded thumbnail moving image in a multi-image raster and positional information on said encoded thumbnail moving image in each of said image areas, which are required for re-encoding the extracted slice code string, from said layout table, and

said moving image re-encoding means converts said slice code string into said code string to be re-combined within said different multi-image raster on the basis of the fetched results to produce said multi-image combined moving image different in image construction from the first-mentioned multi-image combined moving image.

10. A moving image composing system as defined in claim 9, wherein said moving image listing means selectively draws up a list of moving images from said encoded thumbnail moving images accumulated in said accumulating means and moving images composed of the re-encoded slice code strings, and said moving image composing means obtains a code string from said encoded thumbnail moving image code strings accumulated in said accumulating means and the re-encoded slice code strings on the basis of said moving image list drawn up by said moving image listing means.

11. A moving image composing system comprising:

image encoding means for encoding an inputted image on the basis of an image encoding format and for outputting an encoded image; and

image composing means for receiving said one or more encoded images to produce a multi-image combined image stream,

wherein said image encoding means outputs an extended code string, defined as a code string which does not affect an image in said image encoding format, into said encoded image in advance in consideration with a case that a change of an information value defined as a variable-length code in said image encoding format takes place, and

said image composing means changes said information value of said encoded image to a different value through the use of said extended code string for producing said multi-image combined image stream.

12. A moving image composing system as defined in claim 11, wherein said image composing means rewrites said information value of each of said encoded image inputted to output said multi-image combined image stream in which a code string of each of said encoded images is a correct information value as said image encoding format and a code string sequence of each of said encoded images is a correct sequence as said image encoding format.

13. A moving image composing system as defined in claim 11, wherein, when receiving only one encoded image, said image composing means does not produce a multi-image combined image but changes said information value defined as a variable-length code in said encoded image and outputs an encoded image after the change of said information value.

14. A moving image composing system as defined in claim 11, wherein, in changing said information value defined as a variable-length code, said image composing means, when a basic operation unit length of said image composing means is taken to be N bits, takes out a bit code string corresponding an arbitrary multiple of N from said extended code string to adjust a bit string corresponding to a variation of the information code length occurring due to the change of said information value.

15. A moving image composing system as defined in claim 14, wherein said operation unit length is one of 4 bits, 8 bits forming 1 byte, 16 bits, 32 bits and 64 bits which are operation unit lengths of a central processing unit of a general-purpose computer, respectively.

16. A moving image composing system as defined in claim 11, further comprising delay means for outputting inputted data after said inputted data temporarily or long undergoes processing including an inputted data accumulation and an inputted data retention, wherein said image composing means receives said encoded image produced through said delay means.

17. A moving image composing system as defined in claim 16, wherein a process in which said delay means is not placed exists in a process on one of an input side of said image encoding means, a process between said image encoding means and said image composing means and both the processes, and said image composing means receives one or more encoded images produced without a single passage through said delay means.

18. A moving image composing system as defined in claim 11, further comprising a layout table for, when a reproduced image of a multi-image combined image to be outputted is divided into one or more areas, retaining, as multi-image layout information, a construction of said areas, with said layout table retaining, as said multi-image layout information, at least positional information on disposition of each of said areas in a multi-image combined image and size information of each of said areas, and with said image composing means changing an information value defined as a variable-length code and an information value defined as a fixed-length code to values retained in said layout table in producing said multi-image combined image stream.

19. A moving image composing system as defined in claim 18, wherein said encoded images are distinguishable by single identifiers, and said encoded images having arbitrary identifiers are obtainable through the use of said identifiers, and said layout table further retains said identifiers of said encoded images to be displayed in said areas as area image information in said multi-layout information, and said image composing means refers to said area image information to select said encoded image to be displayed in said area from the inputted images or to receive it as one of the inputted encoded images to change said information value of said encoded image.

20. A moving image composing system as defined in claim 18, further comprising layout table changing means for changing a value of said multi-image layout information in said layout table, with said layout table changing means changing said value of said multi-image layout information at an arbitrary timing, and with said image composing means refers to said multi-image layout information value changed to dynamically change a multi-image layout of said multi-image combined image to be outputted.

21. A moving image composing system as defined in claim 11, wherein said image encoding format is for a static image, and said encoded image is a static image.

22. A moving image composing system as defined in claim 11, wherein said image encoding format relates to moving images, and said encoded image is a moving image.

23. A moving image composing system as defined in claim 11, wherein, in said image composing means, said encoded image to be inputted and said multi-image combined image stream to be outputted are obtained by the encoding using image encoding formats different from each other.

24. A moving image composing system comprising an image composing means for receiving at least one encoded image, produced by encoding processing based upon an image encoding format to produce a multi-image combined image stream, wherein, when there is a need to change information value defined as a variable-length code in said image encoding format, said image composing means utilizes an extended code string defined as a code string which does not affect an image in said image encoding format to output said extended code string into said multi-image combined image for changing said information value in said encoded image to a different value.

25. A moving image composing system as defined in claim 24, wherein said image composing means rewrites said information value of each of said encoded image inputted to output said multi-image combined image stream in which a code string of each of said encoded images is a correct information value as said image encoding format and a code string sequence of each of said encoded images is a correct sequence as said image encoding format.

26. A moving image composing system as defined in claim 24, wherein, when receiving only one encoded image, said image composing means does not produce a multi-image combined image, but changes said information value defined as a variable-length code in said encoded image and outputs an encoded image after the change of said information value.

27. A moving image composing system as defined in claim 24, wherein, in changing said information value defined as a variable-length code, said image composing means, when a basic operation unit length of said image composing means is taken to be N bits, inserts a bit code string corresponding an arbitrary multiple of N into said extended code string to adjust a bit string corresponding to a variation of the information code length occurring due to the change of said information value.

28. A moving image composing system as defined in claim 24, wherein said operation unit length is one of 4 bits, 8 bits forming 1 byte, 16 bits, 32 bits and 64 bits which are operation unit lengths of a central processing unit of a general-purpose computer, respectively.

29. A moving image composing system as defined in claim 24, further comprising delay means for outputting inputted data after said inputted data temporarily or long undergoes processing including an inputted data accumulation and an inputted data retention, wherein said image composing means receives said encoded image produced through said delay means.

30. A moving image composing system as defined in claim 29, wherein a process in which said delay means is not placed exists in a process on one of an input side of said image encoding means, a process between said image encoding means and said image composing means and both the processes, and said image composing means receives at least one encoded image produced without a single passage through said delay means.

31. A moving image composing system as defined in claim 24, further comprising a layout table for, when a reproduced image of a multi-image combined image to be outputted is divided into one or more areas, retaining, as multi-image layout information, a construction of said areas, with said layout table retaining, as said multi-image layout information, at least positional information on disposition of each of said areas in a multi-image combined image and size information of each of said areas, and with said image composing means changing an information value defined as a variable-length code and an information value defined as a fixed-length code to values retained in said layout table in producing said multi-image combined image stream.

32. A moving image composing system as defined in claim 31, wherein said encoded images are distinguishable by single identifiers, and said encoded images having arbitrary identifiers are obtainable through the use of said identifiers, and said layout table further retains said identifiers of said encoded images to be displayed in said areas as area image information in said multi-layout information, and said image composing means refers to said area image information to select said encoded image to be displayed in said area from the inputted images or to receive it as one of the inputted encoded images to change said information value of said encoded image.

33. A moving image composing system as defined in claim 31, further comprising layout table changing means for changing a value of said multi-image layout information in said layout table, with said layout table changing means changing said value of said multi-image layout information at an arbitrary timing, and with said image composing means refers to said multi-image layout information value changed to dynamically change a multi-image layout of said multi-image combined image to be outputted.

34. A moving image composing system as defined in claim 24, wherein said image encoding format is for a static image, and said encoded image is a static image.

35. A moving image composing system as defined in claim 24, wherein said image encoding format relates to moving images, and said encoded image is a moving image.

36. A moving image composing system comprising:

first and second image encoding means each for encoding an inputted image on the basis of an image encoding format to output an encoded image; and

image composing means for receiving at least one encoded image outputted from said first image encoding means and at least one encoded image outputted from said second image encoding means to produce a multi-image combined image,

wherein said first image encoding means outputs an extended code string, defined as a code string which does not affect an image in said image encoding format, into said encoded image in advance in consideration with a case that a change of an information value defined as a variable-length code in said image encoding format takes place, and said second image encoding means does not output said extended code string into said encoded image, and

said image composing means changes said information value to a different value through the use of said extended coded string for said encoded image outputted from said first image encoding means, and further, changes said information value to a different value by outputting said extended code string into said multi-image combined image for said encoded image outputted from said second image encoding means.