System for constructing mosaic images

• US 7,382,927 B2
• Filed: 04/04/2002
• Issued: 06/03/2008
• Est. Priority Date: 01/08/1999
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Crinon, Regis J., Sezan, M. Ibrahim
• Primary Examiner(s): Chen; Wenpeng
• Application Number:
  US10/116,977
• Publication Number:
  US 20020135672A1
• Time in Patent Office: 2,252 Days
• Field of Search: 382/243, 382/232, 382/233, 382/236, 382/286, 348/699, 375/240.08, 375/240.01, 375/240.12
• US Class Current: 382/243
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic

Abstract

A mosaic image construction system includes at least one of a decoder receiving or an encoder transmitting a sequence of pixel data frames. The sequence includes at least a first pixel data frame and a second pixel data frame. The second pixel data frame is preferably temporally later than the first pixel data frame. The second pixel data frame has an associated parameter indicating the motion of the second frame with respect to the first frame and is used in the construction of the sequence of the pixel data frames. The mosaic image is constructed using at least the first and second pixel data frames together with the associated parameter.

Claims

1. A method of constructing a mosaic image from a sequence of source frames, said method comprising:

(a) constructing a sequence of encoded pixel data frames at an encoder, said sequence including at least a first pixel data frame, a second pixel data frame temporally later than said first pixel data frame, said second pixel data frame having an associated parameter indicating the global motion of the entire said second pixel data frame with respect to the entire said first pixel data frame, where said associated parameter is used as a basis upon which said second pixel data frame is encoded;

(b) transmitting from said encoder said first pixel data frame, said second pixel data frame, and said associated parameter to a decoder, said decoder not used by said encoder to encode said sequence of encoded pixel data frames;

(c) wherein said transmitting from said encoder does not transmit an encoded, constructed said mosaic image to said decoder;

(d) constructing at said decoder said sequence of source frames including said first pixel data frame and said second pixel data frame based upon said associated parameter;

(e) constructing said mosaic image at said decoder using at least said first and second pixel data frames based upon said associated parameter, wherein said mosaic image has a spatial extent greater than said first pixel data frame and greater than said second pixel data frame, wherein a portion of said mosaic image consistent with a portion of said first pixel data frame and said second pixel data frame does not exist within said encoder until after said mosaic image is first constructed by said decoder based upon said first pixel data frame and said second pixel data frame, and does not exist within said decoder until after said mosaic image is first constructed by said decoder based upon said first pixel data frame and said second pixel data frame, and includes pixels representing image information in addition to that found in either individual one of said first and second pixel data frames.

2. The method of claim 1 wherein said first and second pixel data frames are encoded as an MPLG-4 bitstream.

3. The method of claim 2 wherein said MPLG-4 bitstream includes at least one of video object frames and fields encoded using global motion compensation.

4. The method of claim 3 wherein said global motion compensation is signaled by at least one bit flag.

5. The method of claim 1 wherein said first and second pixel data frames are received in said decoder.

6. The method of claim 5 wherein said first and second pixel data frames are transmitted to said decoder from said encoder.

7. The method of claim 6 wherein said encoder is remotely located from said decoder.

8. The method of claim 1 wherein said constructing said mosaic image includes a mosaic builder that merges said pixel data frames.

9. The method of claim 8 wherein said mosaic builder is activated by a mosaic controller in response to at least one of a user and a signal within the bitstream including said first and second pixel data frames.

10. The method of claim 8 wherein said constructing said mosaic image includes a blending factor to control the contribution from different images.

11. The method of claim 8 wherein said constructing said mosaic image is synthesized with respect to an arbitrary but fixed reference viewpoint.

12. The method of claim 1 wherein said mosaic image is used as a representation of said encoded sequence of pixel data frames.

45. The method of claim 8 wherein said first frame is selected as the initial mosaic image.

13. A mosaic image construction system comprising:

(a) a decoder that receives a sequence of encoded pixel data frames, said decoder not used by an encoder to encode said sequence of encoded pixel data frames, said sequence including at least a first pixel data frame, a second pixel data frame temporally later than said first pixel data frame, said second pixel data frame encoded to include an associated parameter indicating the global motion of the entire said second pixel data frame with respect to the entire said first pixel data frame and used by said decoder to decode said second pixel data frame so as to reconstruct the entire said second pixel data frame apart from said first pixel data frame;

(b) said decoder constructing said sequence of encoded pixel data frames including said first pixel data frame, said second pixel data frame constructed based upon said associated parameter;

(c) a mosaic builder in said decoder using at least said first and second pixel data frames together with said associated parameter to construct said mosaic image, wherein said mosaic image has a spatial extent greater than said first pixel data frame and greater than said second pixel data frame, wherein a portion of said mosaic image consistent with a portion of said first pixel data frame and said second pixel data frame does not exist within said sequence of encoded pixel data frames until after said mosaic image is first constructed by said decoder based upon said first pixel data frame and said second pixel data frame.

14. The system of claim 13 wherein said first and second pixel data frames are encoded as an MPLG-4 bitstream.

15. The system of claim 14 wherein said MPLG-4 bitstream includes at least one of video object frames and fields encoded using global motion compensation.

16. The system of claim 15 wherein said global motion is signaled by at least one bit flag.

17. The system of claim 13 wherein said first and second pixel data frames are transmitted to said decoder from said encoder.

18. The system of claim 17 wherein said encoder is remotely located from said decoder.

19. The system of claim 13 wherein said mosaic builder is activated by a mosaic controller in response to at least one of a user and a signal within the bitstream including said pixel data frames.

20. The system of claim 19 wherein said mosaic image is constructed in accordance with a blending factor to control contribution from different images.

21. The system of claim 13 wherein said mosaic image is synthesized with respect to an arbitrary but fixed reference viewpoint.

22. The method of claim 13 wherein said mosaic image is used as a representation of said encoded sequence of pixel data frames.

23. A mosaic image construction system comprising:

(a) an encoder that receives and encodes a sequence of pixel data frames each frame having a respective spatial extent when received by said encoder, said encoder transmitting an encoded said sequence of pixel data frames to a decoder not used by said encoder to encode said sequence of pixel data frames, each encoded said pixel data frame having said respective spatial extent, said encoded sequence including at least a first pixel data frame, a second pixel data frame temporally later than said first pixel data frame, said encoded second pixel data frame including an associated parameter indicating the global motion of the entire said second pixel data frame with respect to the entire said first pixel data frame where said associated parameter is used as a basis upon which said second pixel data frame is decoded; and

(b) a mosaic builder using at least said first and second pixel data frames together with said associated parameter to construct a mosaic image, wherein said mosaic image has a spatial extent greater than said first pixel data frame and greater than said second pixel data frame, wherein a decoder constructs said encoded pixel data frames including said first pixel data frame, said second pixel data frame based upon said associated parameter.

24. The system of claim 23 wherein said first and second pixel data frames are encoded as an MPLG-4 bitstream.

25. The system of claim 24 wherein said MPLG-4 bitstream includes at least one of video object frames and fields encoded using global motion compensation.

26. The system of claim 25 wherein said global motion compensation is signaled by at least one bit flag.

27. The system of claim 23 wherein said first and second pixel data frames are transmitted to said decoder from said encoder.

28. The system of claim 27 wherein said encoder is remotely located from said decoder.

29. The system of claim 23 wherein said mosaic builder is activated by a mosaic controller in response to at least one of a user and a signal within the bitstream including said pixel data frames.

30. The system of claim 29 wherein said mosaic image is constructed in accordance with a blending factor to control contribution from different images.

31. The system of claim 23 wherein said mosaic image is synthesized with respect to an arbitrary but fixed reference viewpoint.

32. The method of claim 23 wherein said mosaic image is used as a representation of said encoded sequence of pixel data frames.

33. A method of constructing a mosaic image comprising:

(a) constructing a sequence of encoded pixel data frames at an encoder, said sequence including at least a first pixel data frame, and a second pixel data frame, said second pixel data frame having an associated parameter indicating the global motion of the entire said second pixel data frame with respect to the entire said first pixel data frame, where said associated parameter is used as a basis upon which pixel values of said second pixel data frame are selectively not encoded; and

(b) constructing said mosaic image at a decoder using at least said first and second pixel data frames together with said associated parameter, wherein said mosaic image has a spatial extent greater than said first pixel data frame and greater than said second pixel data frame, wherein a decoder constructs said pixel data frames including said first and second pixel data frame, said second pixel data frame constructed using said associated parameter together with pixel values of said first frame.

34. The method of claim 33 wherein said first and second pixel data frames are encoded as an MPLG-4 bitstream.

35. The method of claim 34 wherein said MPLG-4 bitstream includes at least one of video object frames and fields encoded using global motion compensation.

36. The method of claim 35 wherein said global motion is signaled by at least one bit flag.

37. The method of claim 33 wherein said first and second pixel data frames are received in said decoder.

38. The method of claim 37 wherein said first and second pixel data frames are transmitted to said decoder from said encoder.

39. The method of claim 38 wherein said encoder is remotely located from said decoder.

40. The method of claim 33 wherein said constructing said mosaic image includes a mosaic builder that merges said pixel data frames.

41. The method of claim 40 wherein said mosaic builder is activated by a mosaic controller in response to at least one of a user and a signal within the bitstream including said first and second pixel data frames.

42. The method of claim 40 wherein said constructing said mosaic image includes a blending factor to control contribution from different images.

43. The method of claim 40 wherein said constructing said mosaic image is synthesized with respect to an arbitrary but fixed reference viewpoint.

44. The method of claim 33 wherein said mosaic image is used as a representation of said sequence of pixel data frames.

46. The method of claim 40 wherein said first frame is selected as the initial mosaic image.