Tiering and manipulation of peer's heads in a telepresence system

• US 9,756,288 B2
• Filed: 04/10/2013
• Issued: 09/05/2017
• Est. Priority Date: 04/10/2013
• Status: Active Grant

• Current Assignee: InterDigital Patent Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Huber, Mark J., Walker, Mark Leroy, Redmann, William Gibbens
• Primary Examiner(s): El-Zoobi, Maria
• Application Number:
  US14/767,336
• Publication Number:
  US 20160150186A1
• Time in Patent Office: 1,609 Days
• Field of Search: 348 1401- 1409, 709204
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

In a telepresence system, a local station receives images of participants at remote stations processed to isolate the remote participant in the image from image background. The local station merges the images of the participants at the remote stations to generate an output image in which a first remote participant appears to partially overlay another participant. The output image undergoes display at the local station.

Claims

1. A method of processing data at a local station to produce data representing images of participants at remote stations, comprising the steps of:

receiving at the local station data representing the images of the participants at the remote stations and data indicating facings of the participants at the remote stations with respect to cameras at the remote stations;

processing at the local station the data representing the images of the participants at the remote stations to isolate the participant in the image from image background;

merging, at the local station, data representing the processed images of the participants at the remote stations to generate data representing an output image in which a first processed participant image appears to partially overlay a second processed participant image and in which the first processed participant image appears in the output image horizontally translated relative to the second processed participant image by a distance determined in response to the data indicating the facings of the first and second participants; and

providing the data representing the output image at an output of the local station suitable for coupling to a display device.

2. The method according to claim 1 wherein the receiving step comprises:

accepting from the remote stations images of participants already processed to isolate the participant in the image from image background.

3. The method according to claim 1 further including the step of normalizing the first and second processed participant images in height.

4. The method according to claim 3 wherein the normalizing occurring in accordance with a distance from the top of a participant'"'"'s head to a centerline of the participant'"'"'s eyes in the image of the participant at the remote station.

5. The method according to claim 1 further including the step of scaling at least one of the processed participant images to cause the second processed participant image to appear smaller than the first processed participant image, whereby the second processed participant images appears further away due to perspective.

6. The method according to claim 1 wherein the distance the first processed participant image appears in the output image horizontally translated from the second processed participant image has a first value when at least one of the participants in the first and second processed participant images has a first facing and has a second value less than the first value otherwise.

7. The method according to claim 1 wherein at least one of the processed participant images is horizontally flipped with respect to the corresponding image of the participant at the remote station.

8. The method according to claim 1 further comprising the step of:

accepting an ordering of remote participants from a local participant,

whereby a local participant can select which of the remote participants corresponds to the first and second processed participant images.

17. The method of claim 1 wherein any of one of the first and second processed participant images serves as a placeholder image for at least a portion of time corresponding to the participant being out of view at the corresponding remote station.

18. The method of claim 17 wherein the placeholder image constitutes a freeze frame of the corresponding participant.

19. The method of claim 18 wherein the freeze frame undergoes processing to further indicate status of the corresponding participant.

9. Apparatus comprising:

an input buffer receiving and storing data representing images of a plurality of participants, each at one of a plurality of remote stations, and to receive and store data representing facings of the participants at the remote stations with respect to cameras at the remote stations;

a video controller coupled to the input buffer and processing the data representing the images to isolate the participant in the image from the image background and to generate data representing an output image in which a first processed participant image appears to partially overlay a second processed participant image and in which the first processed participant image appears in the output image horizontally translated relative to the second processed participant image by a distance determined in response to the data indicating the facings of the first and second participants; and

an output buffer coupled to the video controller and storing the data representing the output image and supplying the data representing the output image at an output suitable for coupling to a display device.

10. The apparatus according to claim 9 wherein the images stored in the input buffer have already undergone processing to isolate the participant in the image from image background.

11. The apparatus according to claim 9 wherein the video controller processes the data representing the first and second processed participant images to normalize the first and second processed participant images in height.

12. The apparatus according to claim 11 wherein the video controller processes the data representing the first and second processed participant images to normalize the first and second processed participant images in height in accordance with a distance from the top of a participant'"'"'s head to a centerline of the participant'"'"'s eyes in the image of the participant at the remote station.

13. The apparatus according to claim 9 wherein the video controller processes the data representing the first and second processed participant images to scale the second processed participant image to appear smaller than the first processed participant image, whereby the second processed participant images appears further away due to perspective.

14. The apparatus according to claim 9 wherein the distance the first processed participant image appears in the output image horizontally translated from the second processed participant image has a first value when at least one of the participants in the first and second processed participant images has a first facing, and has a second value less than the first value when the at least one participant has a second facing.

15. The apparatus according to claim 9 wherein the video controller processes the data representing at least one of the first and second processed participant images to horizontally flip the at least one of the processed participant images with respect to the corresponding image of the participant at remote station.

16. The apparatus according to the claim 9 wherein the video controller processes the data representing the first and second processed participant images to alter an ordering for the first and second processed participant images responsive to a command from a local participant to select which of the remote participants corresponds to the first and second processed participant images.

20. The apparatus according to claim 9 wherein one of the images of the plurality of participants constitutes a placeholder image for at least a portion of a time that the corresponding participant is out of view at the corresponding remote station.

21. The apparatus according to claim 20 wherein the placeholder image constitutes a freeze frame of the corresponding participant.

22. The apparatus according to claim 21 wherein the freeze frame undergoes processing to further indicate the status of the corresponding participant.