Scene Background Blurring Including Range Measurement

• US 20130194375A1
• Filed: 01/08/2013
• Published: 08/01/2013
• Est. Priority Date: 07/06/2010
• Status: Abandoned Application

• Current Assignee: DigitalOptics Corporation Europe Limited (Adeia, Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia, Inc.)
• Inventors: Michrowski, Guy, Shabtay, Gal, Cohen, Noy, Ben-Eliezer, Eyal, Goldenberg, Ephraim
• Application Number:
  US13/736,785
• Publication Number:
  US 20130194375A1
• Time in Patent Office: Days
• US Class Current: 348/14.07
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04N 23/815 for controlling the resolution by using a single image
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Different distances of two or more objects in a scene being captured in a video conference are determined by determining a sharpest of two or more color channels and calculating distances based on the determining of the sharpest of the two or more color channels. At least one of the objects is identified as a foreground object or a background object, or one or more of each, based on the determining of the different distances. The technique involves blurring or otherwise rendering unclear at least one background object or one or more portions of the scene other than the at least one foreground object, or combinations thereof, also based on the determining of distances.

Claims

1. (canceled)

2. A method of displaying a participant during a video conference against a blurred or otherwise unclear background, comprising:

using an imaging device including an optic, an image sensor and a processor;

determining different distances of two or more objects in a scene being captured in video, including;

performing an auto-focus sweep of the scene;

increasing a depth of field (DOF) including extending a delimit range of defocus distance over which a mean transfer function (MTF) is greater than 0.15 without decreasing aperture while maintaining a focus on said at least one foreground object at approximately said determined distance; and

generating a depth map of the scene based on the auto-focus sweep; and

identifying at least one of the objects as a foreground object or a background object, or one or more of each, based on the determining of the different distances; and

blurring or otherwise rendering unclear the background object.

3. The method of claim 2, further comprising detecting a face within the scene and designating the face as a foreground object.

4. The method of claim 3, further comprising enhancing an audio or visual parameter of the face, or both.

5. The method of claim 4, further comprising enhancing loudness, audio tone, or sound balance of words being spoken by a person associated with the face, or enhancing luminance, color, contrast, or size or location within the scene of the face, or combinations thereof.

6. The method of claim 3, further comprising recognizing and identifying the face as that of a specific person.

7. The method of claim 6, further comprising tagging the face with a stored identifier.

8. The method of claim 2, further comprising designating a nearest object as a foreground object.

9. The method of claim 2, further comprising designating one or more objects as background that are at a different distance than a foreground object.

10. The method of claim 9, further comprising blurring or otherwise rendering unclear at least one background object or one or more portions of the scene other than the at least one foreground object, or combinations thereof, also based on the determining of distances.

11. The method of claim 2, wherein the determining of the different distances comprises using a fixed focus lens.

12. The method of claim 2, wherein a portion of the scene other than a foreground object comprises a detected and recognized face or other object, and the method further comprises determining that the recognized face or other object is private.

13. The method of claim 2, wherein said distances comprises a distance between a video camera component and at least one of the two or more objects in the scene.

14. The method of claim 2, further comprising determining at least one distance based on applying a face model to a detected face within the scene.

15. The method of claim 2, wherein the determining distances comprises calculating distances based on the determining of the sharpest of the two or more color channels.

16. The method of claim 15, wherein the determining of the sharpest of two or more color channel comprises calculating the following for three color channels:

17. The method of claim 16, wherein the determining of the sharpest of two or more color channel further comprises calculating the following:

18. One or more non-transitory computer-readable storage media having code embedded therein for programming a processor to perform a method of displaying a participant during a video conference against a blurred or otherwise unclear background, wherein the method comprises:

determining different distances of two or more objects in a scene being captured in video, including;

performing an auto-focus sweep of the scene;

increasing a depth of field (DOF) including extending a delimit range of defocus distance over which a mean transfer function (MTF) is greater than 0.15 without decreasing aperture while maintaining a focus on said at least one foreground object at approximately said determined distance;

generating a depth map of the scene based on the auto-focus sweep;

identifying at least one of the objects as a foreground object or a background object, or one or more of each, based on the determining of the different distances; and

blurring or otherwise rendering unclear the background object.

19. The one or more computer-readable storage media of claim 18, wherein the method further comprises detecting a face within the scene and designating the face as a foreground object.

20. The one or more computer-readable storage media of claim 19, wherein the method further comprises enhancing an audio or visual parameter of the face, or both.

21. The one or more computer-readable storage media of claim 20, wherein the method further comprises enhancing loudness, audio tone, or sound balance of words being spoken by a person associated with the face, or enhancing luminance, color, contrast, or size or location within the scene of the face, or combinations thereof.

22. The one or more computer-readable storage media of claim 19, wherein the method further comprises recognizing and identifying the face as that of a specific person.

23. The one or more computer-readable storage media of claim 22, wherein the method further comprises tagging the face with a stored identifier.

24. The one or more computer-readable storage media of claim 18, wherein the method further comprises designating a nearest object as a foreground object.

25. The one or more computer-readable storage media of claim 18, wherein the method further comprises designating one or more objects as background that are at a different distance than a foreground object.

26. The one or more computer-readable storage media of claim 25, wherein the method further comprises blurring or otherwise rendering unclear at least one background object or one or more portions of the scene other than the at least one foreground object, or combinations thereof, also based on the determining of distances.

27. The one or more computer-readable storage media of claim 18, wherein the determining of the different distances comprises using a fixed focus lens.

28. The one or more computer-readable storage media of claim 18, wherein a portion of the scene other than a foreground object comprises a detected and recognized face or other object, and the method further comprises determining that the recognized face or other object is private.

29. The one or more computer-readable storage media of claim 18, wherein said distances comprises a distance between a video camera component and at least one of the two or more objects in the scene.

30. The one or more computer-readable storage media of claim 18, wherein the method further comprises determining at least one distance based on applying a face model to a detected face within the scene.

31. The one or more computer-readable storage media of claim 18, wherein the determining distances comprises calculating distances based on the determining of the sharpest of the two or more color channels.

32. The one or more computer-readable storage media of claim 31, wherein the determining of the sharpest of two or more color channel comprises calculating the following:

33. The one or more computer-readable storage media of claim 32, wherein the determining of the sharpest of two or more color channel further comprises calculating the following:

38. The apparatus of claim 33, wherein the method further comprises recognizing and identifying the face as that of a specific person.

39. The apparatus of claim 38, wherein the method further comprises tagging the face with a stored identifier.

34. A video conferencing apparatus, comprising:

a video camera including a lens, and an image sensor;

a microphone;

a display;

a processor;

one or more networking connectors; and

one or more computer-readable storage media having code embedded therein for programming a processor to perform a method of displaying a participant during a video conference against a blurred or otherwise unclear background, wherein the method comprises;

determining different distances of two or more objects in a scene being captured in video, including;

performing an auto-focus sweep of the scene;

increasing a depth of field (DOF) including extending a delimit range of defocus distance over which a mean transfer function (MTF) is greater than 0.15 without decreasing aperture while maintaining a focus on said at least one foreground object at approximately said determined distance;

generating a depth map of the scene based on the auto-focus sweep;

identifying at least one of the objects as a foreground object or a background object, or one or more of each, based on the determining of the different distances; and

blurring or otherwise rendering unclear the background object.

35. The apparatus of claim 34, wherein the method further comprises detecting a face within the scene and designating the face as a foreground object.

36. The apparatus of claim 35, wherein the method further comprises enhancing an audio or visual parameter of the face, or both.

37. The apparatus of claim 36, wherein the method further comprises enhancing loudness, audio tone, or sound balance of words being spoken by a person associated with the face, or enhancing luminance, color, contrast, or size or location within the scene of the face, or combinations thereof.

40. The apparatus of claim 34, wherein the method further comprises designating a nearest object as a foreground object.

41. The apparatus of claim 34, wherein the method further comprises designating one or more objects as background that are at a different distance than a foreground object.

42. The apparatus of claim 41, wherein the method further comprises blurring or otherwise rendering unclear at least one background object or one or more portions of the scene other than the at least one foreground object, or combinations thereof, also based on the determining of distances.

43. The apparatus of claim 34, wherein the determining of the different distances comprises using a fixed focus lens.

44. The apparatus of claim 34, wherein a portion of the scene other than a foreground object comprises a detected and recognized face or other object, and the method further comprises determining that the recognized face or other object is private.

45. The apparatus of claim 34, wherein said distances comprises a distance between a video camera component and at least one of the two or more objects in the scene.

46. The apparatus of claim 34, wherein the method further comprises determining at least one distance based on applying a face model to a detected face within the scene.

47. The apparatus of claim 34, wherein the determining distances comprises calculating distances based on the determining of the sharpest of the two or more color channels.

48. The apparatus of claim 47, wherein the determining of the sharpest of two or more color channel comprises calculating the following:

49. The apparatus of claim 48, wherein the determining of the sharpest of two or more color channel further comprises calculating the following: