DETECTING MOTION REGIONS IN A SCENE USING AMBIENT-FLASH-AMBIENT IMAGES

• US 20160232672A1
• Filed: 08/13/2015
• Published: 08/11/2016
• Est. Priority Date: 02/06/2015
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Rezaiifar, Ramin
• Application Number:
  US14/825,932
• Publication Number:
  US 20160232672A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

Systems and methods described herein can compensate for aberrations produced by a moving object in an image captured using a flash. In some embodiments, a method includes capturing a first image at time t−Δt₁, where Δt₁ represents the time difference between capturing the first image and capturing the second image, capturing the second image at a time t, the second image captured using a flash. The method also includes capturing a third image at a time t+Δt₂, where Δt₂ represents the time difference between capturing the second image and capturing the third image, determining motion information of an object that is depicted in the first, second and third image, and modifying at least one portion of the second image using the motion information and a portion of the first image, a portion of the third image, or a portion of the first image and a portion of the third image.

Claims

1. A method for compensating for aberrations produced by a moving object in an image comprising:

generating a first image of a scene having a first exposure and a first external lighting;

generating a second image of the scene having a second exposure and a second external lighting, the second exposure and the second external lighting being different from the first exposure and the first external lighting, the second image captured at a time subsequent to the first image;

generating a third image of the scene having the first exposure and the first external lighting, the third image captured at a time subsequent to the second image; and

determining one or more motion regions using the first image and third image, the one or more motion regions indicating areas in one or more of the first image, second image, and third image that indicate a position of a moving object during the period of time over which the first image, second image, and third image are captured.

2. The method of claim 1, wherein the first image and the third image are generated using ambient light and the second image is generated using a flash to illuminate the scene.

3. The method of claim 1, further comprising adjusting auto white balance, auto exposure, and auto focusing parameters of the first and third image to the same values before determining the one or more motion regions.

4. The method of claim 1, wherein the determining one or more motion regions comprises:

modifying one or more pixels of the first image and the third image;

quantifying a difference value between each set of corresponding pixels, a set of corresponding pixels comprising a pixel in one of the first image or the third image and a pixel in the other of the first image or the third image corresponding to the same location in the image; and

thresholding the difference values between each set of corresponding pixels.

5. The method of claim 1, wherein the determining one or more motion regions is based at least in part on the location of each set of corresponding pixels having a difference value above a threshold value.

6. The method of claim 1, further comprising generating a fourth image using one or more portions of the second image corresponding to motion regions in one or more of the first image and the third image and one or more portions of one or more of the first image and the third image.

7. The method of claim 1, further comprising merging a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image.

8. The method of claim 7, wherein

merging a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image,

comprises layering one or more sections of one or more of the first image, the second image, or the third image, over a motion region detected in another one of the first image, second image, or third image, wherein the one or more sections comprise the same area of a scene as that concealed by a motion region from an image where the area was not concealed by a motion region.

9. A non-transitory computer readable storage medium storing instructions that, when executed, cause at least one physical computer processor to perform a method of adjusting the position of a touch input, the method comprising:

generating a first image of a scene having a first exposure and a first external lighting;

generating a second image of the scene having a second exposure and a second external lighting, the second exposure and the second external lighting being different from the first exposure and the first external lighting, the second image captured at a time subsequent to the first image;

generating a third image of the scene having the first exposure and the first external lighting, the third image captured at a time subsequent to the second image; and

determining one or more motion regions using the first image and third image, the one or more motion regions indicating areas in one or more of the first image, second image, and third image that indicate the position of a moving object during the period of time over which the first image, second image, and third image are captured.

10. The non-transitory computer readable storage medium of claim 9, wherein the first image and the third image are generated using ambient light and the second image is generated using a flash to illuminate the scene.

11. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises adjusting the auto white balance, auto exposure, and auto focusing parameters of the first and third image to the same values before determining the one or more motion regions.

12. The non-transitory computer readable storage medium of claim 9, wherein the method comprises:

modifying one or more pixels of the first image and the third image;

quantifying a difference value between each set of corresponding pixels, a set of corresponding pixels comprising a pixel in one of the first image or the third image and a pixel in the other of the first image or the third image corresponding to the same location in the image; and

thresholding the difference values between each set of corresponding pixels.

13. The non-transitory computer readable storage medium of claim 12, wherein the determining one or more motion regions is based at least in part on the location of each set of corresponding pixels having a difference value above a threshold value.

14. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises, generating a fourth image using one or more portions of the second image corresponding to motion regions in one or more of the first image and the third image and one or more portions of one or more of the first image and the third image.

15. The non-transitory computer readable storage medium of claim 9, wherein the method further comprises merging a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image.

16. The non-transitory computer readable storage medium of claim 15, wherein merging a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image.

comprises layering one or more sections of one or more of the first image, second image, or third image, over a motion region detected in another one of the first image, second image, or third image, wherein the one or more sections comprise the same area of a scene as that concealed by a motion region from an image where the area was not concealed by a motion region.

17. An apparatus configured to compensate for aberrations produced by a moving object in an image, comprising:

a flash system capable of producing illumination for imaging;

a camera coupled to the flash system, wherein the camera is configured to;

generate a first image of a scene having a first exposure and a first external lighting;

generate a second image of the scene having a second exposure and a second external lighting, the second exposure and the second external lighting being different from the first exposure and the first external lighting, the second image captured at a time subsequent to the first image;

generate a third image of the scene having the first exposure and the first external lighting, the third image captured at a time subsequent to the second image;

a memory component configured to store images captured by the camera; and

a processor configured to determine one or more motion regions using the first image and third image, the one or more motion regions indicating areas in one or more of the first image, second image, and third image that indicate the position of a moving object during the period of time over which the first image, second image, and third image are captured.

18. The apparatus of claim 17, wherein the first image and the third image are generated using ambient light and the second image is generated using a flash to illuminate the scene.

19. The apparatus of claim 17, wherein the processor is further configured to adjust auto white balance, auto exposure, and auto focusing parameters of the first image and the third image before determining the one or more motion regions.

20. The apparatus of claim 17, wherein the processor is configured to:

modify one or more pixels of the first image and the third image;

quantify a difference value between each set of corresponding pixels, a set of corresponding pixels comprising a pixel in one of the first image or the third image and a pixel in the other of the first image or the third image corresponding to the same location in the image; and

threshold the difference values between each set of corresponding pixels.

21. The apparatus of claim 20, wherein the processor is configured to determine one or more motion regions based at least in part on the location of each set of corresponding pixels having a difference value above a threshold value.

22. The apparatus of claim 17, wherein the processor is further configured to generate a fourth image using one or more portions of the second image corresponding to motion regions in one or more of the first image and the third image and one or more portions of one or more of the first image and the third image.

23. The apparatus of claim 17, wherein the processor is further configured to merge a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image.

24. The apparatus of claim 23, wherein the processor is configured to layer one or more sections of one or more of the first image, second image, or third image, over a motion region detected in another one of the first image, second image, or third image, wherein the one or more sections comprise the same area of a scene as that concealed by a motion region from an image where the area was not concealed by a motion region.

25. A method for compensating for aberrations produced by a moving object in an image that was captured using a flash illumination system, the method comprising:

capturing a first image at a time t−

Δ

t₁;

capturing a second image subsequent to the first image at a time t, said capturing the second image including activating the flash illumination system, wherein Δ

t₁ represents the time between capturing the first image and capturing the second image;

capturing a third image subsequent to the second image at a time t+Δ

t₂, wherein Δ

t₂ represents the time between capturing the second image and capturing the third image;

determining motion information of an object that is depicted in the first, second and third image; and

modifying at least one portion of the second image using the motion information and a portion of the first image, a portion of the third image, or a portion of the first image and a portion of the third image.

26. The method of claim 25, wherein the first image and the third image are captured using the same exposure and the same external lighting.

27. The method of claim 25, further comprising adjusting the auto white balance, auto exposure, and auto focusing parameters of the first and third image before determining the motion information.

28. The method of claim 25, wherein the determining motion information comprises:

modifying one or more pixels of the first image and the third image;

quantifying a difference value between each set of corresponding pixels, a set of corresponding pixels comprising a pixel in one of the first image or the third image and a pixel in the other of the first image or the third image corresponding to the same location in the image; and

thresholding the difference values between each set of corresponding pixels.

29. The method of claim 25, wherein modifying at least one portion of the second image comprises merging a portion of the second image with a portion of the first image,

a portion of the third image, or

a portion of the first image and a portion of the third image.

30. The method of claim 25, wherein merging a portion of the second image with a portion of the first image,

a portion of the third image,

or a portion of the first image and a portion of the third image,

comprises layering one or more sections of one or more of the first image, second image, or third image, over a motion region detected in another one of the first image, second image, or third image, wherein the one or more sections comprise the same area of a scene as that concealed by a motion region from an image where the area was not concealed by a motion region.