System and method for combining local and remote images such that images of participants appear overlaid on another in substanial alignment

• US 7,982,762 B2
• Filed: 08/27/2004
• Issued: 07/19/2011
• Est. Priority Date: 09/09/2003
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Thorne, Jeremy M, Chatting, David J, Morrison, David G
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US10/571,066
• Publication Number:
  US 20070064112A1
• Time in Patent Office: 2,517 Days
• Field of Search: 348 1401- 1416, 382263-269
• US Class Current: 348/14.07
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

The invention provides a method and system wherein first video images are directly overlaid with remote video images to produce a combined video image which is then displayed to the user and/or stored as appropriate. Preferably at least one of the first and/or remote images is subject to an image processing operation prior to the overlay operation being performed, the image processing operation being such that the scenes contained within the images to be overlaid when processed are separably distinguishable to a user within the combined video image when viewed by the user. The overlay operation is performed such that the scenes contained within the respective video images are substantially in alignment on top of each other. By overlaying the respective first and remote video images as described a single image is obtained within which the respective scenes of the respective first and remote images are still separably distinguishable, but which is still of an appropriate size for display on a screen of limited size without occlusion of one or other of the images.

Claims

1. A video calling system comprising:

a) a video camera arranged to produce local video images representative of a local video call participant;

b) a transmitter arranged to receive said local video images and send information relating to said local video images to a remote receiver and a receiver arranged to receive information relating to remote video images representative of a remote participant in the video call via a communications link;

c) a local video display arranged to display video images to said local video call participant; and

d) an image generator arranged to receive said local video images and information relating to said remote video images, and generate combined video images for display on said local video display by combining said local and remote images such that both the face of the local participant and the face of the remote participant are;

(i) displayed one on top of the other in substantial alignment; and

(ii) simultaneously displayed in the region in which each one of the faces overlies the other face.

2. A system according to claim 1, and further comprising:

an image processor arranged to process said local video images and/or said remote video images according to one or more respective image processing operations, and to output processed versions of the local and/or remote video images to the image generator as input thereto;

wherein said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the image generator.

3. A system according to claim 2, wherein one of the image processing operations comprises an image contrast enhancement operation.

4. A system according to claim 3 wherein the image contrast enhancement operation comprises detecting edges within the input image to produce an edge map, applying a threshold operation to the input image to produce a thresholded image, and combining the edge map with the thresholded image to produce the processed image.

5. A system according to claim 2, wherein one of the image processing operations comprises processing to render the processed image of increased opacity.

6. A system according to claim 2, wherein the remote video images are not processed by the image processor and the image generator operates to overlay the respective processed local video images onto the received remote video images.

7. A system according to claim 2, wherein the image processor is further operable to process the local video images twice to produce two processed versions of the local images;

wherein a first processed version of each image is input to the image generator as input thereto, and a second processed version of each image is input to the transmitter for transmission thereby.

8. A system according to claim 7, wherein different image processing operations are applied to the local video images to produce the first processed versions and the second processed versions respectively.

9. A system according to claim 1, wherein the local image includes the local user'"'"'s head, and/or the remote image includes a remote user'"'"'s head.

10. A system according to claim 2, the system further comprising a quality measurer for determining a measure of at least one characteristic indicative of image quality for the local video images, the image generator being responsive to an indication of the measured quality, such that at least one visible characteristic of the combined images of the local video call participant is dependent on the image quality of the local video images.

11. A system according to claim 10, wherein the degree to which the combined images relating to the local video call participant are opaque is dependent on the image quality of the local video images.

12. A video calling system comprising:

a) a virtual reality unit arranged to produce virtual reality style images of a local video call participant;

b) a transmitter arranged to receive said virtual reality style images and to send information relating to said virtual reality style images to a remote receiver, and a receiver to receive information relating to remote video images representative of a participant in the video call via a communications link;

c) a local video display arranged to display video images to said local video call participant; and

d) an image generator arranged to receive said local video images and said information relating to remote video images and to generate combined video images for display on said local video display by combining the virtual reality style image of the local participant with the image of the remote participant such that both the face of the local participant and the face of the remote participant are;

(i) displayed one on top of the other in substantial alignment; and

(ii) simultaneously displayed in the region in which each one of the faces overlies the other face.

13. A video calling method comprising:

a) producing local video images representative of a local video call participant;

b) sending information relating to said local video images and receiving information relating to remote video images representative of a participant in the video call via a communications link;

c) displaying video images to said local video call participant; and

d) generating combined video images for display by combining the video image of the local participant with the video image of the remote participant such that both the face of the local participant and the face of the remote participant are;

(i) displayed one on top of the other in substantial alignment; and

(ii) simultaneously displayed in the region in which each one of the faces overlies the other face.

14. A method according to claim 13, and further comprising:

processing said local video images and/or said remote video images according to one or more respective image processing operations, and using processed versions of the local and/or remote video images to the generating step as input thereto;

wherein said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the generating step.

15. A method according to claim 14, wherein one of the image processing operations comprises an image contrast enhancement operation.

16. A method according to claim 15 wherein the image contrast enhancement operation comprises detecting edges within the input image to produce an edge map, applying a threshold operation to the input image to produce a thresholded image, and combining the edge map with the thresholded image to produce the processed image.

17. A method according to claim 14, wherein one of the image processing operations comprises processing to render the processed image of increased opacity.

18. A method according to claim 14, wherein the remote video images are not processed by the image processing step, and the generating step operates to combine the respective processed local video images onto the received remote video images.

19. A method according to claim 14, wherein the image processing step further includes processing the local video images twice to produce two processed versions of the local images;

wherein a first processed version of each image is used by the generation step as input thereto, and a second processed version of each image is sent to a remote terminal via the communication link.

20. A method according to claim 19, wherein different image processing operations are applied to the local video images to produce the first processed versions and the second processed versions respectively.

21. A method according to claim 13, wherein the local image includes the local participant'"'"'s head, and/or the remote image includes a remote participant'"'"'s head.

22. A non-transitory computer readable storage medium storing a computer program or any one or more of a suite of computer programs such that when executed by a computer or collectively by a plurality of computers it/they cause the computer or computers to perform the method of claim 13.

23. A method according to 13, including the further step of evaluating the quality of the local video images, wherein a visible characteristic of the local participant in the generated combined video images is chosen in dependence on the evaluated quality of the local video image.

24. A method according to claim 23, wherein the visible characteristic is the degree of transparency or visibility of the local participant in the combined image.

25. A video calling method comprising:

a) producing virtual reality style images representative of a local video call participant;

b) sending information relating to said virtual reality style images and receiving information relating to remote video images representative of a participant in the video call via a communications link;

c) displaying video images to said local video call participant; and

d) generating combined video images for display by combining the virtual reality style image of the local participant with the image of the remote participant such that both face of the local participant and the face of the remote participant are;

(i) displayed one on top of the other in substantial alignment; and

(ii) simultaneously displayed in the region in which each one of the faces overlies the other face.

26. A video calling system comprising:

a) a video camera arranged to produce local video images representative of a local video call participant;

b) a transmitter arranged to receive said local video images and send information relating to said local video images to a remote receiver and a receiver arranged to receive information relating to remote video images representative of a remote participant in the video call via a communications link;

c) a local video display arranged to display video images to said local video call participant;

d) an image generator arranged to receive said local video images and information relating to said remote video images, and generate combined video images for display on said local video display by combining said local and remote images such that the images of the participants appear to be overlaid one on top of the other in substantial alignment; and

e) an image processor arranged to process said local video images and/or said remote video images according to one or more respective image processing operations, and to output processed versions of the local and/or remote video images to the image generator as input thereto;

wherein

said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the image generator; and

the local video images are processed according to an image contrast enhancement operation, and the remote video images are processed to render the processed image of increased opacity;

or

the remote video images are processed according to an image contrast enhancement operation, and the local video images are processed to render the processed image of increased opacity.

27. A video calling system comprising:

a) a video camera arranged to produce local video images representative of a local video call participant;

b) a transmitter arranged to receive said local video images and send information relating to said local video images to a remote receiver and a receiver arranged to receive information relating to remote video images representative of a remote participant in the video call via a communications link;

c) a local video display arranged to display video images to said local video call participant;

d) an image generator arranged to receive said local video images and information relating to said remote video images, and generate combined video images for display on said local video display by combining said local and remote images such that the images of the participants appear to be overlaid one on top of the other in substantial alignment; and

e) an image processor arranged to process said local video images and/or said remote video images according to one or more respective image processing operations, and to output processed versions of the local and/or remote video images to the image generator as input thereto;

wherein

said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the image generator;

the image processor is further operable to process the local video images twice to produce two processed versions of the local images;

wherein a first processed version of each image is input to the image generator as input thereto, and a second processed version of each image is input to the transmitter for transmission thereby;

different image processing operations are applied to the local video images to produce the first processed versions and the second processed versions respectively;

the first processed versions of the local video images are produced by processing the local video images according to an image contrast enhancement operation, and the second processed versions of the local video images are produced by processing the local video images according to render the processed image of increased opacity;

or

the second processed versions of the local video images are produced by processing the local video images according to an image contrast enhancement operation, and the first processed versions of the local video images are produced by processing the local video images to render the processed image of increased opacity.

28. A video calling method comprising:

a) producing local video images representative of a local video call participant;

b) sending information relating to said local video images and receiving information relating to remote video images representative of a participant in the video call via a communications link;

c) displaying video images to said local video call participant;

d) generating combined video images for display by combining the video image of the local participant with the video image of the remote participant such that the images of the participants appear to be overlaid one on top of the other in substantial alignment; and

e) processing said local video images and/or said remote video images according to one or more respective image processing operations, and using processed versions of the local and/or remote video images to the generating step as input thereto;

wherein

said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the generating step; and

the local video images are processed according to an image contrast enhancement operation, and the remote video images are processed to render the processed image of increased opacity;

or

the remote video images are processed according to an image contrast enhancement operation, and the local video images are processed to render the processed image of increased opacity.

29. A video calling method comprising:

a) producing local video images representative of a local video call participant;

b) sending information relating to said local video images and receiving information relating to remote video images representative of a participant in the video call via a communications link;

c) displaying video images to said local video call participant;

d) generating combined video images for display by combining the video image of the local participant with the video image of the remote participant such that the images of the participants appear to be overlaid one on top of the other in substantial alignment; and

e) processing said local video images and/or said remote video images according to one or more respective image processing operations, and using processed versions of the local and/or remote video images to the generating step as input thereto;

wherein

said image processing operations are operable to process said video images such that the respective participants of the local and remote video images are separably distinguishable in the combined image generated by the generating step;

the image processing step further includes processing the local video images twice to produce two processed versions of the local images;

wherein a first processed version of each image is used by the generation step as input thereto, and a second processed version of each image is sent to a remote terminal via the communication link;

different image processing operations are applied to the local video images to produce the first processed versions and the second processed versions respectively; and

the first processed versions of the local video images are produced by processing the local video images according to an image contrast enhancement operation, and the second processed versions of the local video images are produced by processing the local video images to render the processed image of increased opacity;

or

the second processed versions of the local video images are produced by processing the local video images according to an image contrast enhancement operation, and the first processed versions of the local video images are produced by processing the local video images to render the processed image of increased opacity.