System and method for video conferencing

• US 10,341,611 B2
• Filed: 04/29/2014
• Issued: 07/02/2019
• Est. Priority Date: 04/30/2013
• Status: Active Grant

• Current Assignee: Inuitive Ltd.
• Original Assignee: Inuitive Ltd.
• Inventors: Ben-Bassat, David
• Primary Examiner(s): Nguyen, Phung-Hoang J
• Application Number:
  US14/763,840
• Publication Number:
  US 20150365628A1
• Time in Patent Office: 1,890 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00
• CPC Class Codes: H04N 13/376 for tracking left-right translational head movements, i.e. lateral movements
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A system for video conferencing is disclosed. The system comprises a data processor which receives from a remote location a stream of imagery data of a remote user, and displays an image of the remote user on a display device. The data processor also receives a stream of imagery data of an individual in a local scene in front of the display device, and extracts a gaze direction and/or a head orientation of the individual. The data processor varies a view of the image responsively to the gaze direction and/or the head orientation.

Claims

1. A system for video conferencing, comprising:

a data processor configured for receiving from a remote location a stream of imagery data of a remote user and displaying an image of said remote user on a display device, receiving a stream of imagery data of an individual in a local scene in front of said display device, extracting a head orientation of said individual, and varying a view of said image responsively to said head orientation;

wherein said variation of said view comprises varying a displayed orientation of said remote user such that a rate of change of said orientation matches a rate of change of said head orientation;

wherein said stream of imagery data comprises range data in a form of a depth map providing depth information at a lower resolution for a group of pixels of the image data.

2. The system according to claim 1, further comprising, said display device, and an imaging system constituted to receive a view of said local scene and transmit a stream of imagery data of said local scene to said data processor.

3. The system according to claim 2, wherein said imaging system is configured for capturing a video stream and range data.

4. The system according to claim 2, wherein said imaging system is configured for capturing stereoscopic image data.

5. The system according to claim 3, wherein said imaging system is configured for capturing stereoscopic image data.

6. The system according to claim 4, wherein said data processor is configured for calculating range data from said stereoscopic image data.

7. The system according to claim 5, wherein said data processor is configured for calculating range data from said stereoscopic image data.

8. The system according to claim 2, further comprising a light source configured for projecting a light beam onto said individual, wherein said data processor is configured for calculating range data by a time-of-flight technique.

9. The system according to claim 2, further comprising a light source configured for projecting a light pattern onto said individual, wherein said data processor system is configured for calculating range data by a technique selected from the group consisting of a triangulation technique and a structured light technique.

10. The system according to claim 1, wherein said stream of imagery data of said remote user comprises a video stream, and wherein said data processor is configured for reconstructing a three-dimensional image from said stream of imagery data and displaying said three-dimensional image on said display device.

11. The system according to claim 2, wherein said stream of imagery data of said remote user comprises a video stream, and wherein said data processor is configured for reconstructing a three-dimensional image from said stream of imagery data and displaying said three-dimensional image on said display device.

12. The system according to claim 1, wherein said variation of said view comprises varying a displayed orientation of said remote user oppositely to a change of said head orientation.

13. The system according to claim 2, wherein said variation of said view comprises varying a displayed orientation of said remote user oppositely to a change of said head orientation.

14. The system according to claim 1, wherein said variation of said view comprises maintaining a fixed orientation of said remote user relative to said head orientation.

15. The system according to claim 2, wherein said variation of said view comprises maintaining a fixed orientation of said remote user relative to said head orientation.

16. The system according to claim 1, further comprising at least one infrared light source constituted for illuminating at least a portion of said local scene, wherein said stream of imagery data of said individual comprises image data received from said individual in a visible range of wavelengths and in an infrared range of wavelengths.

17. The system according to claim 2, further comprising at least one infrared light source constituted for illuminating at least a portion of said local scene, wherein said stream of imagery data of said individual comprises image data received from said individual in a visible range of wavelengths and in an infrared range of wavelengths.

18. A system for at least two-way video conferencing between at least a first party at a first location and a second party at a second location, the system comprising, a first system at said first location and a second system at said second location, each of said first and said second systems being the system according to claim 1.

36. The system of claim 1, further comprising extracting a gaze direction of said individual, and varying a view of said image responsively to said gaze direction.

37. The system of claim 36, wherein said variation of said view comprises varying a displayed orientation of said remote user such that a rate of change of said orientation matches a rate of change of said gaze direction.

38. The system according to claim 36, wherein said variation of said view comprises varying a displayed orientation of said remote user oppositely to a change of said gaze direction.

19. A method of video conferencing, comprising:

receiving from a remote location a stream of imagery data of a remote user and displaying an image of said remote user on a display device;

using an imaging system for capturing a stream of imagery data of an individual in a local scene in front of said display device; and

using a data processor for extracting a head orientation of said individual, and varying a view of said image responsively to said head orientation;

wherein said varying said view comprises varying a displayed orientation of said remote user such that a rate of change of said orientation matches a rate of change of said head orientation

wherein said stream od imagery data comprises range data in a form of a depth map providing depth information at a lower resolution for a group of pixels of the image data.

20. The method according to claim 19, wherein said capturing said stream of imagery data comprises capturing a video stream and said range data.

21. The method according to claim 19, wherein said capturing said stream of imagery data comprises capturing stereoscopic image data.

22. The method according to claim 20, wherein said capturing said stream of imagery data comprises capturing stereoscopic image data.

23. The method according to claim 21, further comprising calculating range data from said stereoscopic image data.

24. The method according to claim 22, further comprising calculating range data from said stereoscopic image data.

25. The method according to claim 20, further comprising projecting a light beam onto said individual, and calculating said range data by a time-of-flight technique.

26. The method according to claim 20, further comprising projecting a light pattern onto said individual, and calculating said range data by a technique selected from the group consisting of a triangulation technique and a structured light technique.

27. The method according to claim 19, wherein said stream of imagery data of said remote user comprises a video stream, and the method comprises reconstructing a three-dimensional image from said stream of imagery data and displaying said three-dimensional image on said display device.

28. The method according to claim 20, wherein said stream of imagery data of said remote user comprises a video stream, and the method comprises reconstructing a three-dimensional image from said stream of imagery data and displaying said three-dimensional image on said display device.

29. The method according to claim 19, wherein said varying said view comprises varying a displayed orientation of said remote user oppositely to a change of said head orientation.

30. The method according to claim 20 wherein said varying said view comprises varying a displayed orientation of said remote user oppositely to a change of said head orientation.

31. The method according to claim 19, wherein said varying said view comprises maintaining a fixed orientation of said remote user relative to said head orientation.

32. The method according to claim 20, wherein said varying said view comprises maintaining a fixed orientation of said remote user relative to said head orientation.

33. The method according to claim 19, further comprising illuminating at least a portion of said local scene by infrared light, wherein said capturing said stream of imagery data of said individual comprises imaging said individual in a visible range of wavelengths and in an infrared range of wavelengths.

34. The method according to claim 20, further comprising illuminating at least a portion of said local scene by infrared light, wherein said capturing said stream of imagery data of said individual comprises imaging said individual in a visible range of wavelengths and in an infrared range of wavelengths.

35. A computer software product, comprising a non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a data processor, cause the data processor to receive a video stream of a remote user and a video stream of an individual in a local scene, and to execute the method according to claim 19.

39. The method of claim 19, further comprising extracting a gaze direction of said individual, and varying a view of said image responsively to said gaze direction.

40. The method of claim 39, wherein said variation of said view comprises varying a displayed orientation of said remote user such that a rate of change of said orientation matches a rate of change of said gaze direction.

41. The system according to claim 39, wherein said variation of said view comprises varying a displayed orientation of said remote user oppositely to a change of said gaze direction.