Method and apparatus for synthesizing images for video communication systems

• US 5,825,366 A
• Filed: 04/04/1997
• Issued: 10/20/1998
• Est. Priority Date: 09/01/1993
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Ishibashi, Satoshi, Yagi, Takashi, Oikawa, Shigeru, Arita, Kazuho
• Primary Examiner(s): BUCHEL JR, RUDOLPH JOHN
• Application Number:
  US08/832,851
• Time in Patent Office: 564 Days
• Field of Search: 345/418-9, 345/426, 345/431-32, 345/433, 345/435, 345/440-41, 345/113-15, 382/169, 382/172
• US Class Current: 345/634
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels

Abstract

According to the method of synthesizing a image from a plurality of images, each of the luminance I_p1xy -I_pNxy of each of the pixel p_1xy -p_Nxy positioned at the same coordinates (x,y) of N images P₁ -P_N are compared with each of the predetermined threshold luminance T₁ -T_N corresponding to images P₁ -P_N, wherein "N" is a natural number greater than 1. If the luminance I_p1xy -I_pNxy of the pixels p_1xy -p_Nxy are all greater than the corresponding threshold T₁ -T_N of corresponding image, the synthesized pixel p_xy at the coordinate (x,y) of a synthesized image is determined by synthesizing the pixels p_1xy -p_Nxy with predetermined synthesis ratio for each of the images P₁ -P_N. If the condition described above is not satisfied, the synthesized pixel p_xy at the coordinate (x,y) of the synthesized image is determined by synthesizing with arbitrary synthesis ratio only the pixels having luminance less than or equal to the corresponding threshold T₁ -T_N among the pixels p_1xy -p_Nxy. The procedure described above is performed with regard to all pixels in the region wherein images are to be synthesized.

Claims

1. A method of synthesizing an image on a display terminal of a video communication system from N images P₁ -P_N corresponding to writings and hand writings generated by multiple users at remote locations to generate a clear and coherent resultant image for group work, said images corresponding to a plurality of regions, the method comprising the steps of:

comparing a luminance I_p1xy -I_pNxy of each pixel P_1xy -P_Nxy positioned at same coordinates (x,y) of said N images P₁ -P_N with a predetermined threshold T₁ -T_N corresponding to each of said images P₁ -P_N, N being a natural number greater than 1;

determining a synthesized pixel P_xy at said coordinates (x,y) of a synthesized image by synthesizing said pixels P_1xy -P_Nxy using a predetermined synthesis ratio for each of said images P₁ -P_N when said luminance I_p1xy -I_pNxy of said pixels P_1xy -P_Nxy are all greater than the corresponding threshold T₁ -T_N ;

determining said synthesized pixel P_xy at said coordinates (x,y) of said synthesized image by synthesizing pixels having a luminance less than or equal to the corresponding threshold T₁ -T_N using an arbitrary synthesis ratio when luminance of at least one of said pixels is less than or equal to the corresponding threshold; and

repeating the above steps for each of the regions to be synthesized, to generating the clear and coherent resultant image.

2. A method of synthesizing an image on a display terminal of a video communication system from N images P₁ -P_N corresponding to writings and hand

movements generated by multiple users at remote locations to generate a clear and coherent resultant image for group work, said images corresponding to a plurality of regions, the method comprising the steps of:

comparing a luminance I_p1xy -I_pNxy of each pixel P_1XY -P_Nxy positioned at same coordinates (x,y) of said N images P₁ -P_N to each other, N being a natural number greater than 1;

taking a pixel having the lowest luminance among said pixels P_1xy -P_Nxy as a synthesized pixel P_xy at said coordinates (x,y) of a synthesized image; and

repeating the above steps for each of the regions to be synthesized, to generate the clear and coherent resultant image.

3. A method of synthesizing an image on a display terminal of a video communication system from N video images P₁ P_N and M still images S₁ -S_M corresponding to writings and hand movements generated by multiple users at remote locations to generate a clear and coherent resultant image for group work, said video and still images corresponding to a plurality of regions, N being a natural number greater than 1 and M being a natural number, the method comprising the steps of:

determining a value of M, and when M is 1, taking a still image S₁ as a synthesized still image S;

comparing, when M is greater than 1, a luminance I_s1xy -I_sMxy of each pixel S_1xy -S_Mxy positioned at same coordinates (x,y) of said M still images S₁ -S_M with a predetermined threshold T_S1 -T_SM corresponding to each of said still images S₁ -S_M ;

determining a synthesized still pixel s_xy at said coordinates (x,y) of a synthesized still image S by synthesizing said pixels s_1xy -s_Mxy using a predetermined synthesis ratio for each of said still images S₁ -S_M when said luminances I_s1xy -I_smxy of said pixels s_1xy -s_Mxy are all greater than the corresponding threshold T_s1 -T_sM ;

determining said synthesized still pixel s_xy at said coordinates (x,y) of said synthesized still image S by synthesizing pixels having a luminance less than or equal to the corresponding threshold T_s1 -T_sM using an arbitrary synthesis ratio when at least one of said pixels has a luminance less than or equal to the corresponding threshold;

repeating the above steps for each of the regions to be synthesized;

comparing a luminance I_v1xy -I_vNxy of each pixel v_1xy -v_Nxy positioned at same coordinates (x,y) of said N video images V₁ -V_N with a predetermined threshold T_v1 -T_vN corresponding to each of said video images V₁ -V_N ;

taking said synthesized still pixel s_xy at said coordinates (x,y) of said synthesized still image S as a synthesized pixel P_xy at said coordinates (x,y) of a synthesized image P when said luminances I_v1xy -I_vNxy of said pixels are all greater than the corresponding threshold T_v1 -T_vN ;

determining said synthesized pixel P_xy at said coordinates (x,y) of said synthesized image P by synthesizing pixels having a luminance less than or equal to the corresponding threshold T_v1 -T_vN among said pixels V_1xy -V_Nxy using an arbitrary ratio when at least one of said pixels has a luminance less than or equal to the corresponding threshold; and

repeating the above steps for each of the regions to be synthesized, to generate the clear and coherent resultant image.

4. A method of synthesizing an image on a display terminal of a video communication system from N video images and M still images generated by multiple users at remote locations. said video and still images corresponding to a plurality of regions, N being a natural number greater than 1 and M being a natural number, the method comprising the steps of:

determining a value of M, and when M is 1, taking a still image S₁ as a synthesized still image S;

comparing, when M is greater than 1, a luminance I_s1xy -I_sMxy of each pixel s_1xy -s_Mxy positioned at same coordinates (x,y) of said M still images S₁ -S_M to each other;

taking a pixel having a lowest luminance among said pixels s_1xy -s_Mxy as a synthesized still pixel s_xy at said coordinates (x,y) of a synthesized still image S;

repeating the above steps for each of the comparing a luminance I_v1xy -I_vNxy of each pixel v_1xy -v_Nxy positioned at same coordinates (x,y) of said N video images V₁ -V_N with a predetermined threshold T_v1 -T_vN corresponding to each of said video images V₁ -V_N ;

taking said synthesized still pixel s_xy at said coordinates (x,y) of said synthesized still image S as a synthesized pixel P_xy at said coordinates (x,y) of a synthesized image P when said luminance I_v1xy -I_vNxy of said pixels v_1xy -v_Nxy are all greater than the corresponding threshold T_v1 -T_vN ;

taking a pixel having a lowest luminance among the pixels having a luminance less than or equal to the corresponding threshold T_v1 -T_vN among said pixels v_1xy -v_Nxy as said synthesized pixel P_xy at said coordinates (x,y) of said synthesized image P when at least one of said pixels has a luminance less than or equal to the corresponding threshold; and

repeating the above steps for each of the regions to be synthesized, thereby generating the clear and coherent resultant image.

5. A method of synthesizing an image on a display terminal of a video communication system from N video images and M still images generated by multiple users at remote locations, said video and still images corresponding to a plurality of regions, N being a natural number greater than 1 and M being a natural number, the method comprising the steps of:

determining a value of M, and when M is 1, taking a still image S₁ as a synthesized still image S;

comparing, when M is greater than 1, a luminance I_s1xy -I_sMxy of each pixel s_1xy -s_Mxy positioned at same coordinates (x,y) of M still images S₁ -S_M to each other;

taking a pixel having a lowest luminance among said pixels s_1xy -s_Mxy as a synthesized still pixel s_xy at said coordinates (x,y) of a synthesized still image S;

repeating the above steps for each of the regions to be synthesized;

comparing a luminance I_v1xy -I_vNxy of each pixel V_1xy -V_Nxy positioned at same coordinates (x,y) of said N video images V₁ -V_N to each other;

selecting a pixel having a lowest luminance among said pixels v_1xy -v_Nxy ;

taking said synthesized still pixel s_xy at said coordinates (x,y) of said synthesized still image S as a synthesized pixel p_xy at said coordinates (x,y) of a synthesized image P when a luminance of the selected pixel is greater than a predetermined threshold;

taking said selected pixel as said synthesized pixel p_xy at said coordinates (x,y) of said synthesized image P when the luminance of the selected pixels is less than or equal to the predetermined threshold; and

repeating the above steps for each of the regions to be synthesized, to generate the clear and coherent resultant image.

6. An apparatus for synthesizing an image on a display terminal of a video communications system from N images P₁ -P_N generated by multiple users at remote locations, pixel by pixel for every pixel, N being a natural number greater than 1, the apparatus comprising:

comparing means for comparing a luminance signal I_p1xy -I_pNxy of each pixel P_1xy -P_Nxy positioned at same coordinates (x,y) of said N images P₁ -P_N, received from remote locations, with a predetermined threshold luminance signal S_T1 -S_TN corresponding to each of said images P₁ -P_N, and for outputting first information when said luminance signals I_p1xy -I_pNxy are all greater than the corresponding threshold luminance signal S_T1 -S_TN and outputting second information regarding the luminance signals less than or equal to the corresponding threshold luminance signal S_T1 -S_TN when the luminance of the selected pixels is less than or equal to the predetermined threshold; and

synthesizing means for synthesizing an image from said images P₁ -P_N with a predetermined ratio for each of said images P₁ -P_N, when said first information is ouputted from the comparing means, for synthesizing an image from images having the corresponding luminance signals to said second information among

said images P₁ -P_N with arbitrary ratio when said second information is ouputted from the comparing means.

7. An apparatus for synthesizing an image on a display terminal of a video communication system from N images P₁ -P_N generated by multiple users at remote locations. pixel by pixel for every pixel, N being a natural number greater than 1, the apparatus comprising:

comparing means for comparing a luminance signal I_p1 -I_pN of each of pixel P_1xy -P_Nxy positioned at same coordinates (x, y) of said N images P₁ -P_N received from the remote locations, and outputting information regarding a lowest luminance signal; and

selection means for selecting the pixel having the lowest luminance signal as a synthesized pixel P_xy at coordinates (x, y) of a synthesized image.

8. An apparatus for synthesizing an image on a display terminal of a video communication system from N video images and M still images generated by multiple users at remote locations, pixel by pixel for every pixel, N being a natural number greater than 1 and M being a natural number, the apparatus comprising:

first comparing means for comparing a luminance signal I_s1 -I_sM of each pixel positioned at same coordinates (x, y) of said M still images S₁ -S_M with a predetermined threshold luminance signal S_Ts1 -S_TsM corresponding to each of said still images S₁ -S_M, outputting first information when said luminance signals I_s1xy -I_smxy are all greater than the corresponding threshold signals S_Ts1 -S_TsM, and outputting second information regarding the luminance signals less than or equal to the corresponding threshold signals S_Ts1 -S_TsM when at least one of said pixels has a luminance less than or equal to the corresponding threshold;

first synthesizing means for synthesizing a synthesized still image from said still images S₁ -S_M with predetermined synthesis ratio for each of said still images S₁ -S_M when said first information is ouputted from the first comparing means, and synthesizing a synthesized still image from the corresponding images to said second information at an arbitrary ratio when said second information is ouputted from the first comparing means;

second comparing means for comparing a luminance signal I_v1 -I_vN of each pixel positioned at same coordinates (x, y) of said N video images V₁ -V_N with each of predetermined threshold luminance signal S_Tv1 -S_TvN corresponding to each of said video images V₁ -V_N, and when said luminance signals I_v1xy -I_vNxy are all greater than the corresponding threshold luminance signals S_Tv1 -S_TvN outputting third information indicating that at least one of said pixels has a luminance less than or equal to the corresponding threshold,

outputting fourth information regarding the luminance signals less than or equal to the corresponding threshold luminance signals S_Tv1 -S_TvN ;

second synthesizing means for outputting said synthesized still image as a synthesized image when said third information is ouputted from the second comparison means, and for synthesizing a synthesized image from an image having the corresponding luminance signals to said fourth information among said images V₁ -V_N with arbitrary ratio when at least one of said pixels has a luminance less than or equal to the corresponding threshold.

9. An apparatus for synthesizing an image on a display terminal of a video communication system from N video images and M still images generated by multiple users at remote locations, pixel by pixel for every pixel, N being a natural number greater than 1 and M being a natural number, the apparatus comprising:

first comparison means for comparing a luminance signal I_s1 ∝

I_sM of each pixel positioned at same coordinates (x,y) of M still images S₁ -S_M received from remote locations, and outputting first information regarding a lowest luminance signal;

selection means for selecting a still image having the luminance signal corresponding to said first information and outputting the selected still image as a synthesized still image;

second comparison means for comparing a luminance signal I_v1 -I_vN of each pixel of each of said N video images V₁ -V_N with a predetermined threshold luminance signal S_Tv1 -S_TvN corresponding to each of said video images V₁ -V_N and when said luminance signals I_v1xy -I_vNxy are greater than the corresponding threshold luminance signals S_Tv1 -S_TvN outputting second information or outputting the lowest luminance signal as third information when at least one of said pixels has a luminance less than or equal to the corresponding threshold; and

synthesizing means for outputting said synthesized still image as a synthesized image when said second information is ouputted from the second comparison means, and for outputting the video image having the luminance signal corresponding to said third information as said synthesized image when at least one of said pixels has a luminance less than or equal to the corresponding threshold.

10. An apparatus for synthesizing an image on a display terminal of a video communication system from N video images V₁ -V_N and M still images S₁ -S_M generated by multiple users at remote locations, pixel by pixel for every pixel, N being a natural number greater than 1 and M being a natural number, the apparatus comprising:

first comparison means for comparing a luminance signal I_s1 -I_sM of each pixel positioned at same coordinates of M still images S₁ -S_M, and outputting first information regarding the lowest luminance signal;

selection means for selecting the still image corresponding to said first information and outputting the selected still image as a synthesized still image;

second comparison means for comparing a luminance signal I_v1 -I_vN of each pixel of said N video images V₁ -V_N and outputting second information regarding a lowest luminance signal, comparing the lowest luminance signal with a predetermined threshold luminance signal, and when the lowest luminance signal is greater than said threshold luminance signal, outputting third information; and

synthesizing means for outputting said synthesized still image as a synthesized image when said third information is ouputted from the second comparison means, and for outputting the video image corresponding to said second information as said synthesized image when said lowest luminance signal has a luminance less than or equal to the corresponding threshold.