PRIVACY CAMERA

• US 20140176663A1
• Filed: 12/20/2012
• Published: 06/26/2014
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross, Ramin, Mehran
• Granted Patent: US 9,124,762 B2
• Time in Patent Office: Days
• US Class Current: 348/14.07
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A privacy camera, such as a light field camera that includes an array of cameras or an RGBZ camera(s)) is used to capture images and display images according to a selected privacy mode. The privacy mode may include a blur background mode and a background replacement mode and can be automatically selected based on the meeting type, participants, location, and device type. A region of interest and/or an object(s) of interest (e.g. one or more persons in a foreground) is determined and the privacy camera is configured to clearly show the region/object of interest and obscure or replace the background according to the selected privacy mode. The displayed image includes the region/object(s) of interest clearly shown (e.g. in focus) and any objects in a background of the combined image shown having a limited depth of field (e.g. blurry/not in focus) and/or the background replaced with another image and/or fill.

Claims

1. A method for creating a privacy mode from a privacy camera, comprising:

determining objects of interest in a scene comprising a foreground and a background;

determining a privacy mode from privacy modes comprising a background blur and a background replacement;

when the determined privacy mode is the background blur, creating an image by setting a limited depth of field for the privacy camera such that the objects of interest in the scene in the foreground appears in focus and any objects within the background of the scene appear to not be in focus;

when the determined privacy mode is the background replacement, creating the image by setting a dynamically adjustable foreground depth range for the privacy camera used in determining the foreground to include the objects of interest in the scene and replacing the background with at least one of an image and a fill;

and

displaying the image.

2. The method of claim 1, wherein the privacy camera comprises an array of cameras having an array size of at least 2×

2.

3. The method of claim 1, wherein the privacy mode is determined using at least one of:

user privacy preferences and call information.

4. The method of claim 1, wherein determining the privacy mode further comprises using at least one of:

a call location, face detection and head pose, and device type.

5. The method of claim 3, wherein determining the privacy mode comprises determining when a meeting is at least one of:

an intra-company meeting, an inter-group meeting, and an inter-company meeting.

6. The method of claim 1, wherein the privacy mode is automatically determined using one or more of:

user privacy preferences, a call location, a device type, face detection, head pose, and call participants.

7. The method of claim 2, further comprising compensating for a failure in at least one of the cameras in the array of cameras using at least one of the other cameras within the array.

8. The method of claim 1, further comprising determining a depth estimate from the privacy camera.

9. The method of claim 3, wherein replacing the background with at least one of the image and the fill comprises determining the image to use based on one or more of:

user privacy preferences, a call location, a device type, face detection and head pose, a meeting type including an intra-company meeting, an inter-group meeting, and an inter-company meeting.

10. A computer-readable medium storing computer-executable instructions for using a camera including a privacy mode, comprising:

determining a region of interest in a foreground of a scene obtained by the camera;

determining a privacy mode from privacy modes comprising a background blur and a background replacement;

when the determined privacy mode is the background blur, creating an image by setting a limited depth of field for the camera such that the objects of interest in the scene in the foreground appears in focus and any objects within the background of the scene appear to not be in focus;

when the determined privacy mode is the background replacement, creating the image by automatically setting a foreground depth for the camera used in determining the foreground to include the region of interest in the scene based at least in part on a number of people in the scene and their respective depths and replacing the background with at least one of an image and a fill; and

displaying the image.

11. The computer-readable medium of claim 10, wherein the camera comprises and array of cameras having an array size of at least 2×

2.

12. The computer-readable medium of claim 10, wherein the privacy mode is automatically determined using at least one of:

user privacy preferences and call information.

13. The computer-readable medium of claim 12, further comprising using at least one of:

a call location and a device type in automatically determining the privacy mode.

14. The computer-readable medium of claim 10, further comprising determining a depth estimate.

15. The computer-readable medium of claim 10, wherein the privacy mode is automatically determined using one or more of:

user privacy preferences, a call location, a device type, and a meeting type including an intra-company meeting, an inter-group meeting, and an inter-company meeting.

16. A system, comprising:

a processor and memory;

an operating environment executing using the processor;

a privacy camera comprising;

one of;

an array of at least four cameras disposed on a base having at least a five millimeter spacing between a center of each of the at least four cameras;

or an RGBZ camera; and

a camera manager that is configured to perform actions comprising;

automatically determining a region of interest in a foreground of a scene obtained by the privacy camera;

determining a privacy mode from privacy modes comprising a background blur and a background replacement;

when the determined privacy mode is the background blur, creating an image by setting a limited depth of field for the privacy camera such that the objects of interest in the scene in the foreground appears in focus and any objects within the background of the scene appear to not be in focus;

when the determined privacy mode is the background replacement, creating the image by setting a foreground depth for the privacy camera used in determining the foreground to include the region of interest in the scene and replacing the background with at least one of an image and a fill; and

displaying the image.

17. The system of claim 16, wherein determining the privacy mode comprises using one or more of:

user privacy preferences, a call location, a device type, face detection and head pose, and a meeting type including an intra-company meeting, an inter-group meeting, and an inter-company meeting to automatically select the privacy mode.

18. The system of claim 16, wherein the array of cameras comprises an array of cameras having an array size of at least 2×

2.

19. The system of claim 16, further comprising compensating for a failure in at least one of the cameras in the array of cameras using at least one of the other cameras within the array.

20. The system of claim 16, wherein replacing the background with at least one of the image and the fill comprises determining the image to use based on one or more of:

user privacy preferences, a call location, face detection and head pose, a device type, a meeting type including an intra-company meeting, an inter-group meeting, and an inter-company meeting.