System and method for combining a plurality of video streams generated in a videoconference

• US 9,065,974 B2
• Filed: 07/01/2013
• Issued: 06/23/2015
• Est. Priority Date: 11/21/2008
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (Plantronics, Inc.)
• Inventors: Halavy, Avishay
• Primary Examiner(s): Woo, Stella
• Application Number:
  US13/932,145
• Publication Number:
  US 20130300819A1
• Time in Patent Office: 722 Days
• Field of Search: 348/14.08, 348/14.09, 348/14.01, 348/14.03, 348/14.05, 348/14.07, 348/36, 348/39, 348/218.1, 382/284, 382/294
• US Class Current: 1/1
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Disclosed herein is a method and apparatus for videoconferencing that allows video images from two or more cameras at the same site to be displayed as a single panoramic image. Accordingly, a conferencing endpoint having a single monitor can display the panoramic image of the two or more video images from an endpoint having multiple cameras, such as a telepresence endpoint. A sliding display area can be used to define manually a zoomed portion of the panoramic image to be displayed. Alternatively, the zoomed portion may be determined automatically. The zoomed portion may be changed during the course of the conference.

Claims

1. A method of forming a panoramic video image from video images received from two or more video cameras located at a single videoconferencing site, comprising:

receiving two or more video streams generated by the two or more video cameras;

identifying an overlay area that appears in the video image data related to at least two of the video streams;

forming a stream of panoramic video image data representative of the two or more video streams by stitching the video image data of the two or more video stream in real time,

wherein the panoramic video image data contains a single instance of the video data related to the identified overlay area.

2. The method of claim 1, further comprising decoding the two or more video streams to yield video image data for each of the video streams.

3. The method of claim 1, further comprising:

encoding the stream of panoramic video image data; and

sending the encoded stream of panoramic video image data toward a videoconferencing entity.

4. The method of claim 3, wherein the videoconferencing entity is a regular endpoint.

5. The method of claim 1, wherein the overlay area is identified based on similar image elements repeated in at least two of the video image data from the two or more video streams.

6. The method of claim 5, wherein the repeated similar image elements are identified manually by a user.

7. The method of claim 5, wherein the repeated similar image elements are identified automatically.

8. The method of claim 1, wherein the overlay area is identified from a snapshot video image from each one of the two or more video streams.

9. The method of claim 5, wherein the repeated similar image elements are alignment indicators projected in a field of view of the two or more cameras at the single conferencing site.

10. The method of claim 1, further comprising scaling the video image data of at least one of the video streams.

11. The method of claim 1, further comprising height adjusting the video image data for at least one of the video streams.

12. The method of claim 1, further comprising adjusting a rotation of the video image data for at least one of the video streams.

13. The method of claim 1, further comprising creating a stitching instruction file comprising data about the overlay area.

14. The method of claim 1, wherein stitching the video image data comprises loading a saved stitching instruction file comprising data about the overlay area.

15. The method of claim 1, wherein identifying the overlay area comprises calculating and storing a scale factor.

16. The method of claim 1, wherein identifying the overlay area comprises calculating and storing a rotation factor.

17. The method of claim 1, further comprising providing a sliding display area for selecting a portion of the panoramic image to be displayed on a videoconferencing display.

18. The method of claim 17, wherein the portion of the panoramic image selected is controllable by a user.

19. The method of claim 18, wherein the portion of the panoramic image selected is manually controllable via a remote control.

20. The method of claim 17, wherein the portion of the panoramic image is selected automatically.

21. The method of claim 20, wherein the portion of the panoramic image selected is determined based on the audio energy of audio signals corresponding to the compressed video streams.

22. The method of claim 3, wherein identifying, forming, and encoding are performed at a telepresence endpoint.

23. The method of claim 3, wherein identifying, forming, and encoding are performed at a multipoint control unit (MCU).

24. A method for displaying two or more video images received from a telepresence videoconferencing system as a panoramic video image on a single videoconferencing display unit, comprising:

receiving at least two compressed video streams from the telepresence videoconferencing endpoint;

decoding the compressed video streams to yield video image data for each of the two or more video streams;

identifying in real time an overlay area that appears in the video image data related to at least two of the decoded video streams; and

stitching the video image data of the video streams of the two or more video streams in real time to form panoramic video image data representative of the two or more video streams,

wherein the panoramic video image data contains a single instance of the video data related to the identified overlay area.

25. The method of claim 24, further comprising:

zooming a portion of the panoramic video image; and

displaying the zoomed portion of the panoramic video image on a videoconferencing display unit.

26. The method of claim 24, wherein the stitching comprises removing the overlay area in at least one of the two or more video images.

27. The method of claim 25, wherein the zoomed portion of the displayed panoramic video image is selected by a user of the videoconferencing display unit.

28. The method of claim 25, wherein the zoomed portion of the displayed panoramic video image is slid over the panoramic video image.

29. A video conferencing unit for forming a panoramic video image from compressed video streams from two or more cameras at a single videoconferencing terminal, comprising:

a video control unit adapted to;

decode the compressed video streams to yield video image data representative of video images for each of the video streams;

identify an overlay area that appears in the video image data related to at least two of the video images;

stitch the video image data of the video images to form panoramic video image data representative of the two or more video images wherein the panoramic video image data contains a single instance of the video data related to the identified overlay area,

encode the panoramic image data; and

send the encoded panoramic image data toward a videoconferencing terminal.

30. The video conferencing unit of claim 29, wherein the overlay area is identified based on repeated similar image elements occurring in at least two of the video images.

31. The video conferencing unit of claim 30, wherein the repeated similar image elements are identified manually by a user.

32. The video conferencing unit of claim 30, wherein the repeated similar image elements are identified automatically.

33. The video conferencing unit of claim 30, wherein the repeated similar image elements are alignment indicators projected in a field of view of the two or more cameras.

34. The video conferencing unit of claim 29, wherein the video control unit is further adapted to scale at least one of the video images.

35. The video conferencing unit of claim 29, wherein the video control unit is further adapted to align at least one of the video images.

36. The video conferencing unit of claim 29, wherein the video control unit is further adapted to create a stitching instruction file comprising data about the overlay area.

37. The video conferencing unit of claim 29, wherein the video control unit is further adapted to calculate and store a scale factor for at least one of the video images.

38. The video conferencing unit of claim 29, wherein the video control unit is further adapted to provide a sliding display area for selecting a portion of the panoramic image to be displayed on the videoconferencing display.

39. The video conferencing unit of claim 38, wherein placement of the sliding display area on the panoramic image is controllable by a user.

40. The video conferencing unit of claim 39, wherein placement of the sliding display area is controllable manually via a remote control.

41. The video conferencing unit of claim 38, wherein placement of the sliding display area is determined automatically.

42. The video conferencing unit of claim 41, wherein placement of the sliding display area is determined based on the audio energy of audio signals corresponding to the compressed video streams.