Binarization of a video stream

• US 10,051,253 B1
• Filed: 12/18/2015
• Issued: 08/14/2018
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Monastyrshyn, Yurii
• Primary Examiner(s): Liu, Li
• Application Number:
  US14/973,816
• Time in Patent Office: 970 Days
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: H04N 1/40012 Conversion of colour to monochrome
• CPC Class Codes: H04N 9/70 for colour killing

Abstract

Systems, devices, media, and methods are presented for receiving a set of images in a video stream, converting one or more images of the set of images to a set of single channel images, generating a set of approximation images from the set of single channel images, and generating a set of binarized images by thresholding the set of approximation images.

Claims

1. A computer implemented method, comprising:

receiving, by one or more processors of a client device, a set of images in a video stream, the set of images including at least a portion of a face, each pixel in the set of images having a first intensity value;

converting, by the one or more processors of the client device, one or more images of the set of images to a set of single channel images, the set of single channel images corresponding to a single channel associated with an aspect of the face;

generating a set of approximation images from the set of single channel images, the set of approximation images generated by, for each pixel, replacing a second intensity value with an approximation value, the second intensity value of a pixel being an intensity value of the pixel within a single channel image of the set of single channel images, the approximation value generated as a logarithm of the first intensity value and the second intensity value; and

generating a set of binarized images by thresholding the set of approximation images, wherein thresholding the set of approximation images is based on a threshold intensity value corresponding to a fractional luminance value of at least a portion of the set of images.

2. The method of claim 1, further comprising:

identifying an image capture mode from a set of image capture modes, each image capture mode of the set of image capture modes associated with an image processing task;

identifying a single channel associated with the image capture mode and the aspect of the face, the single channel selected from a set of single channels; and

in response to identifying the single channel, converting one or more images of the set of images to the set of single channel images by isolating one or more values for a set of pixels within the one or more images, the one or more values associated with the selected single channel.

3. The method of claim 1, wherein the set of single channels includes a red channel, a blue channel, a green channel, and a grayscale.

4. The method of claim 1 further comprising:

generating an integral representation for each single channel image of the set of single channel images by generating a summed area table for at least a portion of each single channel image of the set of single channel images.

5. The method of claim 4, wherein generating the set of approximation images further comprises:

identifying an image capture mode from a set of image capture modes;

identifying an original intensity value for a pixel within the set of images of the video stream;

selecting a pixel radius based on the image capture mode;

based on the summed area table and the selected pixel radius, generating the set of approximation images by replacing an intensity value of the pixel within one or more single channel images of the set of single channel images with an intensity average value for a set of pixels proximate to the pixel within the pixel radius; and

replacing the intensity average value for the pixel with an approximation value, the approximation value generated based on an original intensity value for the pixels within the set of images of the video stream and the intensity average value.

6. The method of claim 5, wherein selecting the pixel radius further comprises:

identifying an image capture mode from a set of image capture modes, each image capture mode of the set of image capture modes associated with an image processing task; and

based on the identified image capture mode, selecting the pixel radius from a set of pixel radii.

7. The method of claim 1, wherein generating the set of approximation images further comprises:

identifying an original intensity value for a pixel within the set of images of the video stream;

generating a blur intensity value for the pixel within one or more approximation images of the set of approximation images; and

replacing the blur intensity value for the pixel with an approximation value, the approximation value generated based on an original intensity value for the pixel and the blur intensity value for the pixel.

8. The method of claim 1, wherein generating the set of binarized images further comprises:

determining the threshold intensity value;

comparing an approximation value for one or more pixels within the set of approximation images with the threshold intensity value; and

modifying the approximation value for the one or more pixels to a first binary pixel value based on comparing the approximation value and the threshold intensity value.

9. The method of claim 1 further comprising:

determining an area of interest within one or more images of the set of images of the video stream;

converting the area of interest to a single channel area of interest;

generating an approximation area of interest from the single channel area of interest; and

generating a binarized area of interest images by thresholding the approximation area of interest.

10. A system, comprising:

one or more processor; and

a non-transitory machine-readable storage medium storing processor executable instructions that, when executed by the one or more processor, cause the one or more processor to perform operations comprising;

receiving, by the one or more processors of a client device, a set of images in a video stream, the set of images including at least a portion of a face;

converting, by the one or more processors of the client device, one or more images of the set of images to a set of single channel images, the set of single channel images corresponding to a single channel associated with an aspect of the face;

generating a set of approximation images from the set of single channel images, the set of approximation images generated by, for each pixel, relacing a second intensity value with an approximation value, the second intensity value of a pixel being an intensity value of the pixel within a single channel image of the set of single channel images, the approximation value generated as a logarithm of the first intensity value and the second intensity value; and

generating a set of binarized images by thresholding the set of approximation images, wherein thresholding the set of approximation images is based on a threshold intensity value corresponding to a fractional luminance value of at least a portion of the set of images.

11. The system of claim 10, wherein the operations further comprise:

identifying an image capture mode from a set of image capture modes, each image capture mode of the set of image capture modes associated with an image processing task;

identifying a single channel associated with the image capture mode and the aspect of the face, the single channel selected from a set of single channels; and

in response to identifying the single channel, converting one or more images of the set of images to the set of single channel images by isolating one or more values for a set of pixels within the one or more images, the one or more values associated with the selected single channel.

12. The system of claim 10, wherein the operations further comprise:

generating an integral representation for each single channel image of the set of single channel images by generating a summed area table for at least a portion of each single channel image of the set of single channel images.

13. The system of claim 12, wherein generating the set of approximation images further comprises:

identifying an image capture mode from a set of image capture modes;

identifying an original intensity value for a pixel within the set of images of the video stream;

selecting a pixel radius based on the image capture mode;

based on the summed area table and the selected pixel radius, generating the set of approximation images by replacing an intensity value of the pixel within one or more single channel images of the set of single channel images with an intensity average value for a set of pixels proximate to the pixel within the pixel radius; and

replacing the intensity average value for the pixel with an approximation value, the approximation value generated based on an original intensity value for the pixels within the set of images of the video stream and the intensity average value.

14. The system of claim 13, wherein selecting the pixel radius further comprises:

identifying an image capture mode from a set of image capture modes, each image capture mode of the set of image capture modes associated with an image processing task; and

based on the identified image capture mode, selecting the pixel radius from a set of pixel radii.

15. The system of claim 10, wherein generating the set of approximation images further comprises:

identifying an original intensity value for a pixel within the set of images of the video stream;

generating a blur intensity value for the pixel within one or more approximation images of the set of approximation images; and

replacing the blur intensity value for the pixel with an approximation value, the approximation value generated based on an original intensity value for the pixel and the blur intensity value for the pixel.

16. The system of claim 10, wherein generating the set of binarized images further comprises:

determining the threshold intensity value;

comparing an approximation value for one or more pixels within the set of approximation images with the threshold intensity value; and

modifying the approximation value for the one or more pixels to a first binary pixel value based on comparing the approximation value and the threshold intensity value.

17. The system of claim 10, wherein the operations further comprise:

determining an area of interest within one or more images of the set of images of the video stream;

converting the area of interest to a single channel area of interest;

generating an approximation area of interest from the single channel area of interest; and

generating a binarized area of interest image by thresholding the approximation area of interest.

18. A non-transitory machine-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:

receiving, by one or more processors of a client device, a set of images in a video stream, the set of images including at least a portion of a face;

converting, by the one or more processors of the client device, one or more images of the set of images to a set of single channel images, the set of single channel images corresponding to a single channel associated with an aspect of the face;

generating a set of approximation images from the set of single channel images, the set of approximation images generated by, for each pixel, relacing a second intensity value with an approximation value, the second intensity value of a pixel being an intensity value of the pixel within a single channel image of the set of single channel images, the approximation value generated as a logarithm of the first intensity value and the second intensity value; and

generating a set of binarized images by thresholding the set of approximation images, wherein thresholding the set of approximation images is based on a threshold intensity value corresponding to a fractional luminance value of at least a portion of the set of images.

19. The non-transitory machine-readable medium of claim 18, wherein the operations further comprise:

identifying an image capture mode from a set of image capture modes, each image capture mode of the set of image capture modes associated with an image processing task;

identifying a single channel associated with the image capture mode and the aspect of the face, the single channel selected from a set of single channels; and

in response to identifying the single channel, converting one or more images of the set of images to the set of single channel images by isolating one or more values for a set of pixels within the one or more images, the one or more values associated with the selected single channel.

20. The non-transitory machine-readable medium of claim 18, wherein the operations further comprise:

generating an integral representation for each single channel image of the set of single channel images by generating a summed area table for at least a portion of each single channel image of the set of single channel images;

identifying an image capture mode from a set of image capture modes;

identifying an original intensity value for a pixel within the set of images of the video stream;

selecting a pixel radius based on the image capture mode;

based on the summed area table and the selected pixel radius, generating the set of approximation images by replacing an intensity value of the pixel within one or more single channel images of the set of single channel images with an intensity average value for a set of pixels proximate to the pixel within the pixel radius; and

replacing the intensity average value for the pixel with an approximation value, the approximation value generated based on an original intensity value for the pixels within the set of images of the video stream and the intensity average value.