Methods for background subtraction using focus differences

• US 10,070,054 B2
• Filed: 12/08/2014
• Issued: 09/04/2018
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Chehade, Mohamed Nabil Hajj, Fateh, Sina, Itani, Sleiman, Yang, Allen Yang
• Primary Examiner(s): Chen, Chia-Wei A
• Application Number:
  US14/563,997
• Publication Number:
  US 20150093030A1
• Time in Patent Office: 1,366 Days
• Field of Search: 348 42- 60, 348E1307, 382181-231, 382154, 345 7- 9, 1 42- 60, 1E1307
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

First and second images are captured at first and second focal lengths, the second focal length being longer than the first focal length. Element sets are defined with a first element of the first image and a corresponding second element of the second image. Element sets are identified as background if the second element thereof is more in-focus than or as in-focus as the first element. Background elements are subtracted from further analysis. Comparisons are based on relative focus, e.g. whether image elements are more or less in-focus. Measurement of absolute focus is not necessary, nor is measurement of absolute focus change; images need not be in-focus. More than two images, multiple element sets, and/or multiple categories and relative focus relationships also may be used.

Claims

1. A method, comprising:

receiving a first image with a first focal length and a first field of view;

receiving a second image with a second focal length and a second field of view, wherein the second focal length is longer than the first focal length, and wherein the second field of view is similar to the first field of view;

defining a set of elements comprising a first element in the first image and a second element in the second image, wherein the first element substantially corresponds to the second element;

determining a first rate of change of an image property of the first element, the first rate of change is indicative of a first focus level of the first element;

determining a second rate of change of the image property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assigning the set of elements as background elements when the second focus level is greater than or equal to the first focus level.

2. The method of claim 1, further comprising:

determining that an element in the set of elements is not substantially in-focus in the first image or the second image; and

assigning the element as a foreground object in the first image and the second image.

3. The method of claim 1, wherein the first focal length is less than a distance to a foreground object in the first field of view.

4. The method of claim 1, wherein the first focal length is substantially equal to a distance to a foreground object in the first field of view.

5. The method of claim 1, wherein the first focal length is greater than a distance to a foreground object in the first field of view.

6. The method of claim 1, wherein the second focal length is less than a distance to a foreground object in the second field of view.

7. The method of claim 1, wherein the second focal length is substantially equal to a distance to a foreground object in the second field of view.

8. The method of claim 1, wherein the second focal length is greater than a distance to a foreground object in the second field of view.

9. The method of claim 1, wherein the first focal length is approximately at least a defined distance to a foreground object in the first field of view.

10. The method of claim 1, wherein the first focal length corresponds to a length of an of an individual operating a device to capture the first image.

11. The method of claim 1, wherein the first focal length corresponds to a maximum extension of a hand of an individual operating a device to capture the first image.

12. The method of claim 1, wherein the first focal length is at least 0.5 meters.

13. A method, comprising;

receiving a first image with a first focal length and a first field of view;

receiving a second image with a second focal length and a second field of view, the second focal length is longer than the first focal length, and wherein the first field of view is substantially similar to the second field of view;

defining a set of elements comprising a first element in the first image and a second element in the second image, wherein the first element substantially corresponds to the second element;

determining a first rate of change of an image property of the first element, the first rate of change is in of a first focus level of the first element;

determining a second rate of change of the image property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assigning the set of elements to a category when the second focus level is greater than or equal to the first focus level.

14. The method of claim 13, further comprising:

determining that an element in the set of elements is not substantially in-focus in the first image or the second image; and

assigning the element as a foreground object in the first image and the second image.

15. The method of claim 13, wherein the first focal length is less than a distance to a foreground object in the first field of view.

16. The method of claim 13, wherein the first focal length is substantially equal to a distance to a foreground object in the first field of view.

17. The method of claim 13, wherein the first focal length is greater than a distance to a foreground object in the first field of view.

18. The method of claim 13, wherein the second focal length is less than a distance to a foreground object in the second field of view.

19. The method of claim 13, wherein the second focal length is substantially equal to a distance to a foreground object in the second field of view.

20. The method of claim 13, wherein the second focal length is greater than a distance to a foreground object in the second field of view.

21. The method of claim 13, wherein the first focal length is approximately at least a defined distance to a foreground object in the second field of view.

22. The method of claim 13, wherein the first focal length corresponds to a length of an arm of an individual operating a device to capture the first image.

23. The method of claim 13, wherein the first focal length corresponds to a maximum extension of a hand of an individual operating a device to capture the first image.

24. The method of claim 13, wherein the first focal length is at least 0.5 meters.

25. A method, comprising:

receiving a first image with a first focal length and a first field of view;

receiving a second image with a second focal length and a second field of view, wherein the second focal length is longer than the first focal length, and wherein the first field of view is substantially similar to the second field of view;

defining a set of elements comprising a first element in the first image and a second element in the second image, wherein the first element substantially corresponds to the second element;

determining a first rate of change of an Image property of the first element, the first rate of change is indicative of a first focus level of the first element;

determining a second rate of change of the property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assigning the set of elements as background elements when;

none of the elements in the set of elements is substantially fully in-focus;

the first focal length or the second focal length is less than a distance to a foreground object in the first image or the second image;

or

the first focal length or the second focal length is greater than the distance to the foreground object in the first image or the second image.