Apparatus for background subtraction using focus differences

• US 9,924,091 B2
• Filed: 12/08/2014
• Issued: 03/20/2018
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Chehade, Mohamed Nabil Hajj, Fateh, Sina, Itani, Sleiman, Yang, Allen Yang
• Primary Examiner(s): Chen, Chia-Wei A
• Application Number:
  US14/564,010
• Publication Number:
  US 20150092021A1
• Time in Patent Office: 1,198 Days
• Field of Search: 348 42- 60, 345 7- 9
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

First and second images are captured at first and second focal lengths, the second focal length being longer than the first focal length. Element sets are defined with a first element of the first image and a corresponding second element of the second image. Element sets are identified as background if the second element thereof is more in-focus than or as in-focus as the first element. Background elements are subtracted from further analysis. Comparisons are based on relative focus, e.g. whether image elements are more or less in-focus. Measurement of absolute focus is not necessary, nor is measurement of absolute focus change; images need not be in-focus. More than two images, multiple element sets, and/or multiple categories and relative focus relationships also may be used.

Claims

1. An apparatus, comprising:

a housing configured to be wearable by a user;

a first image sensor coupled to the housing, the first image sensor having a first focal length and a first field of view;

a second image sensor coupled to the housing, the second image sensor having a second focal length and a second field of view, wherein;

the first field of view and the second field of view in a region approximate the user;

the first field of view and the second field of view are substantially similar;

the second focal length is greater in length than the first focal length;

the first image sensor is configured to capture a first image; and

the second image sensor is configured to capture a second image; and

a processor operably coupled to the first image sensor and the second image sensor, wherein the processor is configured to;

receive the first image from the first image sensor;

receive the second image from the second image sensor;

define a set of elements comprising a first element of the first image and a second element of said second image, wherein the first element substantially corresponds to the second element;

determine a first rate of change of an image property of the first element, wherein the first rate of change is indicative of a first focus level of the first element;

determine a second rate of change of the image property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assign the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.

2. The apparatus of claim 1, wherein neither the first element nor the second element is fully in-focus.

3. The apparatus of claim 1, wherein the first focal length and the second focal length are preset.

4. The apparatus of claim 1, wherein the first focal length and second focal length are fixed.

5. The apparatus of claim 1, wherein the first image sensor and the second image sensor are arranged to generate stereo image pairs.

6. The apparatus of claim 1, further comprising a beam splitter, wherein the beam splitter, the first image sensor, and the second image sensor are arranged such that the first field of view of the first image sensor is substantially identical to the second field of view of the second image sensor.

7. The apparatus of claim 1, wherein the first image sensor and the second image sensor are arranged proximate one another and substantially aligned with each other, such that the first field of view of the first image sensor is substantially identical to the second field of view of the second image sensor.

8. The apparatus of claim 1, wherein the first image sensor and the second image sensor are adapted to capture the first image and the second image, respectively, at substantially the same point in time.

9. The apparatus of claim 1, wherein the processor is further to exclude the set of elements from subsequent analysis.

10. An apparatus, comprising:

a first image sensor having a first field of view and a first focal length, the first image sensor to capture a first image;

a second image sensor having a second field of view and a second focal length, the second image sensor to capture a second image, wherein the first field of view and the second field of view are substantially similar; and

a processor operably coupled to the first image sensor and the second image sensor, wherein the processor is to;

receive the first image from the first image sensor;

receive the second image from the second image sensor;

define a set of elements comprising a first element of the first image and a second element of said second image, wherein the first element substantially corresponds to the second element;

determine a first rate of change of an image property of the first element, wherein the first rate of change is indicative of a first focus level of the first element;

determine a second rate of change of the image property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assign the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.

11. The apparatus of claim 10, wherein neither the first element nor the second element are fully in-focus.

12. The apparatus of claim 10, wherein the first image sensor and the second image sensor are disposed in a head mounted display.

13. The apparatus of claim 10, wherein the first image sensor is preset to the first focal length, and wherein the second image sensor is preset to the second focal length.

14. The apparatus of claim 10, wherein the first focal length is fixed and wherein the second focal length is fixed.

15. The apparatus of claim 10, wherein the first image and the second image are a stereo image pair.

16. The apparatus of claim 10, further comprising a beam splitter, wherein the beam splitter, the first image sensor, and the second image sensor are arranged such that the first field of view of the first image sensor is substantially identical to the second field of view of the second image sensor.

17. The apparatus of claim 10, wherein the first image sensor and the second image sensor are arranged proximate one another and substantially aligned with each other such that the first field of view is substantially similar to the second field of view.

18. The apparatus of claim 10, wherein the first image sensor and the second image sensor are adapted to capture the first and second images at substantially the same point in time.

19. The apparatus of claim 10, wherein the processor is further to exclude the set of elements from subsequent analysis.

20. An apparatus, comprising:

a housing configured to be wearable by a user;

a processor disposed on the housing;

a first image sensor connected to the housing, wherein the first image sensor is to capture a first image at a first focal length;

a second image sensor connected to the housing, wherein;

the second image sensor is to capture a second image at a second focal length at approximately the same time as the first image sensor captures the first image,

the second focal length longer than the first focal length;

the first image sensor and the second image sensor are in communication with the processor, and wherein the first image sensor and the second image sensor being configured to generate stereo image pairs;

a first display and a second display connected to the housing and in communication with the processor, such that when the housing is worn on the user the first display and the second display are disposed proximate to and substantially aligned with eyes of the user, wherein the processor is to;

define a set of elements, the set of elements comprising a first element of the first image and a second element of said second image, wherein the first element substantially corresponds to the second element;

determine a first rate of change of an image property of the first element, wherein the first rate of change is indicative of a first focus level of the first element;

determine a second rate of change of the image property of the second element, the second rate of change is indicative of a second focus level of the second element; and

assign the set of elements to at least one category in view of a relative difference between the first rate of change and the second rate of change.