Methods for background subtraction using focus differences

• US 9,967,459 B2
• Filed: 12/08/2014
• Issued: 05/08/2018
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Chehade, Mohamed Nabil Hajj, Fateh, Sina, Itani, Sleiman, Yang, Allen Yang
• Primary Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US14/564,007
• Publication Number:
  US 20150093022A1
• Time in Patent Office: 1,247 Days
• Field of Search: 348 42- 60, 348E1307, 382181-231, 382154, 345 7- 9
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

First and second images are captured at first and second focal lengths, the second focal length being longer than the first focal length. Element sets are defined with a first element of the first image and a corresponding second element of the second image. Element sets are identified as background if the second element thereof is more in-focus than or as in-focus as the first element. Background elements are subtracted from further analysis. Comparisons are based on relative focus, e.g. whether image elements are more or less in-focus. Measurement of absolute focus is not necessary, nor is measurement of absolute focus change; images need not be in-focus. More than two images, multiple element sets, and/or multiple categories and relative focus relationships also may be used.

Claims

1. A method, comprising:

obtaining a first image, the first image having a first focal length and a first field of view;

obtaining a second image, the second image having a second focal length and a second field of view;

defining a set of elements comprising a first element of the first image and a second element of the second image, wherein the first element substantially corresponds to the second element;

determining a first rate of change of an image property of the first element;

determining a second rate of change of the image property of the second element; and

assigning the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.

2. The method of claim 1, wherein the set of elements comprises a foreground object that is not substantially fully in-focus in either the first image or the second image.

3. The method of claim 1, wherein the first rate of change is a relative spatial rate of change of an image property of the first image and the second rate of change is a relative spatial rate of change of an image property of the first image.

4. The method of claim 3, wherein the image property comprises a pixel gradient.

5. The method of claim 3, wherein the image property comprises brightness.

6. The method of claim 3, wherein the image property comprises color.

7. The method of claim 3, wherein the image property comprises color channel value.

8. The method of claim 3, wherein the image property comprises at least one of a red channel value, a green channel value, or a blue channel value.

10. The method of claim 1 further comprising determining a first relative degree of edge definition of the first image and a second relative degree of edge definition of the second image.

11. The method of claim 10 further comprising determining a relative spatial rate of change of the image property as indicating a relative degree of edge definition.

9. The method of 4, wherein the image property comprises at least one of a cyan channel value, a yellow channel value, a magenta channel value, or a black channel value.

12. A method, comprising steps of:

obtaining a first image, the first image having a first focal length and a first field of view;

obtaining a second image, the second image having a second focal length and a second field of view, wherein the first focal length and the second focal length are different;

defining a set of elements comprising a first element of the first image and a second element of the second image, wherein the first element substantially corresponds to the second element;

comparing a first rate of change of an image property of the first element with a second rate of change of the image property of the second element; and

assigning the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.

13. The method of claim 12, wherein the set of elements comprises a foreground object, wherein the first image and the second image are not substantially fully in-focus.

14. The method of claim 12, further comprising comparing a sign of a first relative degree of focus of first element and a second relative degree of focus of the second element.

15. The method of claim 14, wherein comparing the relative degree of the first element and the second element comprises comparing a relative spatial rate of change of the image property.

16. The method of claim 15, further comprising determining a greater relative spatial rate of change of the image property.

17. The method of claim 15, wherein the image property comprises pixel gradient.

18. The method of claim 15, wherein the image property comprises brightness.

19. The method of claim 15, wherein the image property comprises color.

20. The method of claim 15, wherein the image property comprises color channel value.

21. The method of claim 15, wherein the image property comprises at least one a red channel value, a green channel value, or blue channel value.

22. The method of claim 15, wherein the image property comprises at least one a cyan channel value, a yellow channel value, a magenta channel value, or a black channel value.

23. The method of claim 14, further comprising determining a greater relative degree the first element and the second element as indicating a greater relative degree of focus.

24. A method, comprising steps of:

obtaining a first image, the first image having a first focal length and a first field of view;

obtaining a second image, the second image having a second focal length and a second field of view;

determining the second focal length is greater than the first focal length;

defining a set of elements comprising a first element of the first image and a second element of the second image, wherein the first element substantially corresponds to the second element;

determining a first rate of change of an image property of the first element;

determining a second rate of change of the image property of the second element; and

assigning the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.