Presentation of enhanced communication between remote participants using augmented and virtual reality

• US 9,959,676 B2
• Filed: 01/21/2015
• Issued: 05/01/2018
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Incorporated
• Inventors: Barzuza, Tamar, Wiener, Yair, Modai, Ori
• Primary Examiner(s): Nguyen, Khai N
• Application Number:
  US14/601,535
• Publication Number:
  US 20150213650A1
• Time in Patent Office: 1,196 Days
• Field of Search: 348 1407
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Embodiments disclosed herein provide methods, systems, and computer readable storage media for presenting a meeting between remote participants. In a particular embodiment, a method provides, during a conference session, compiling view information used to determine a first view seen by the first participant. The method further provides receiving conference video comprising at least video captured of a second participant remote to the physical location of the first participant. Also, the method provides presenting the video captured of the second participant such that the second participant appears at a first position of a meeting location in the first view to the first participant.

Claims

1. A non-transitory computer readable storage medium having instructions stored thereon for presenting a conference session established by a conferencing system to facilitate a meeting between remote participants that each use Augmented Reality (AR) or Virtual Reality (VR) to participate in the meeting, the instructions, when executed by a head mounted display (HMD) worn by a first participant of the remote participants, direct the HMD to:

during the conference session, compile view information used to determine a first view seen by the first participant located at a meeting location, wherein the first view includes what the first participant sees of the meeting location through transparent lenses of the HMD;

receive, via a communication interface of the HMD, a conference video stream comprising at least video captured of a second participant, of the remote participants, remote to the meeting location; and

present the video captured of the second participant on the transparent lenses such that the second participant appears at a first position of the meeting location in the first view to the first participant to appear as though the second participant is physically at the meeting location with the first participant.

2. The non-transitory computer readable medium of claim 1, wherein the instructions further direct the HMD to:

transfer the view information to a conferencing server; and

wherein the conferencing server uses the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

3. The non-transitory computer readable medium of claim 1, wherein the instructions further direct the HMD to:

use the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

4. The non-transitory computer readable medium of claim 1, wherein the instructions further direct the HMD to:

determine where the first position is located within the meeting location.

5. The non-transitory computer readable medium of claim 1, wherein the conference video stream further comprises video captured of a third participant remote to the physical location of the first participant, and the instructions further direct the HMD to:

present the video captured of the third participant such that the third participant appears at a second position of the meeting location in the first view to the first participant.

6. The non-transitory computer readable medium of claim 1, wherein the video captured of the second participant includes audio captured of the second participant, and the instructions further direct the HMD to:

present the audio captured of the second participant such that the audio captured of the second participant appears to be coming from the first position of the meeting location.

7. The non-transitory computer readable medium of claim 1, wherein to compile the view information the instructions direct the HMD to:

capture video of a physical scene outward from the front of the HMD.

8. The non-transitory computer readable medium of claim 1, wherein the instructions further direct the HMD to:

receive information from a conferencing server that indicates the first position of the meeting location.

9. A head mounted display (HMD) for presenting a conference session established by a conferencing system to facilitate a meeting between remote participants that each use Augmented Reality (AR) or Virtual Reality (VR) to participate in the meeting, the HMD comprising:

a processing system configured to, during a conference session for the meeting, compile view information used to determine a first view seen by a first participant of the AR and VR participants wearing the HMD and located at a meeting location, wherein the first view includes what the first participant sees of the meeting location through transparent lenses of the HMD;

a communication interface configured to receive a conference video stream comprising at least video captured of a second participant, of the AR and VR participants, remote to the meeting location; and

a display system configured to present the video captured of the second participant on the transparent lenses such that the second participant appears at a first position of the meeting location in the first view to the first participant to appear as though the second participant is physically at the meeting location with the first participant.

10. The HMD of claim 9, further comprising:

the communication interface configured to transfer the view information to a conferencing server; and

wherein the conferencing server uses the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

11. The HMD of claim 9, further comprising:

the processing system configured to use the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

12. The HMD of claim 9, further comprising:

the processing system configured to determine where the first position is located within the meeting location.

13. The HMD of claim 9, wherein the conference video stream further comprises video captured of a third participant remote to the physical location of the first participant, and the HMD further comprising:

the display system configured to present the video captured of the third participant such that the third participant appears at a second position of the meeting location in the first view to the first participant.

14. The HMD of claim 9, wherein the video captured of the second participant includes audio captured of the second participant, and the HMD further comprising:

an audio system configured to present the audio captured of the second participant such that the audio captured of the second participant appears to be coming from the first position of the meeting location.

15. The HMD of claim 9, further comprising:

a video capture system configured to capture video of a physical scene outward from the front of the HMD; and

wherein the processing system configured to compile the view information comprises the processing system configured to receive the captured video of the physical scene outward from the front of the HMD.

16. The HMD of claim 9, further comprising:

the communication interface configured to receive information from a conferencing server that indicates the first position of the meeting location.