Visual obstruction removal with image capture

• US 8,964,025 B2
• Filed: 04/12/2011
• Issued: 02/24/2015
• Est. Priority Date: 04/12/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cudak, Gary D., Hardee, Christopher J., Humes, Randall C., Miller, Heather C.
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): RAHAMAN, MOHAMMED S
• Application Number:
  US13/085,391
• Publication Number:
  US 20120262569A1
• Time in Patent Office: 1,414 Days
• Field of Search: 348/135
• US Class Current: 348/135
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

An object located within an image foreground relative to image content within a field of view of at least one camera lens is identified, using image processing module, as an image obstruction. Responsive to identifying the image obstruction, points of view of at least one camera lens are determined that provide image content portions of the image content that, in combination, yield a complete image of the image content without the image obstruction located within the image foreground. The image content portions are combined to yield the complete image of the image content without the image obstruction located within the foreground. The combined complete image is stored to a memory.

Claims

1. A system, comprising:

a memory;

a processor programmed to;

identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction;

determine, responsive to identifying the image obstruction, a plurality of points of view of the at least one camera lens that provide a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground, the processor being programmed to;

move, via a lens controller, the at least one camera lens within at least one of a horizontal and a vertical direction to a first lens line of sight representing a first of the plurality of points of view;

obtain a first of the plurality of image content portions of the image content;

move, via the lens controller, the at least one camera lens within at least one of the horizontal and the vertical direction to a second lens line of sight representing a second of the plurality of points of view;

obtain a second of the plurality of image content portions of the image content; and

determine that the obtained first and second of the plurality of image content portions of the image content in combination yield the complete image of the image content without the image obstruction located within the image foreground;

combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

store the combined complete image to the memory.

2. The system of claim 1, where, in being programmed to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction, the processor is programmed to:

perform distance calculation processing of pixels within the field of view of the at least one camera lens;

determine relative distances of objects within the field of view of the at least one camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one camera lens;

determine that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identify the object as the image obstruction based upon its location within the distance range of exclusion.

3. The system of claim 1, where, in being programmed to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction, the processor is programmed to:

perform image recognition of objects within the field of view of the at least one camera lens;

compare recognized objects within the field of view of the at least one camera lens with image items within a database of configured obstruction images;

determine that at least one image item within the database of obstruction images matches the object located within the image foreground; and

identify the object as the image obstruction.

4. The system of claim 3, where, in being programmed to compare recognized objects within the field of view of the at least one camera lens with image items within a database of obstruction images, the processor is programmed to:

retrieve a user image omission profile from a database comprising image item database identifiers of obstruction images configured for omission from captured images; and

compare the recognized objects within the field of view of the at least one camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

5. The system of claim 1, further comprising a display and where the processor is further programmed to:

display the image content within the field of view of one of the at least one camera lens on the display; and

where, in being programmed to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction, the processor is programmed to;

detect a touchscreen selection on an area of a display surface of the display;

identify a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identify an image area within the boundary of the object as the image obstruction.

6. The system of claim 5, where the image area within the boundary of the image comprises a selected object and where the processor is further programmed to:

configure a user image omission profile with the selected object; and

add the selected object to a database of obstruction images.

7. The system of claim 1, where, in being programmed to combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground, the processor is programmed to perform image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.

8. A computer program product comprising a computer readable storage device including computer readable program code, where the computer readable program code when executed on a computer causes the computer to:

identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction;

determine, responsive to identifying the image obstruction, a plurality of points of view of the at least one camera lens that provide a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground, the computer readable program code when executed on the computer causing the computer to;

move, via a lens controller, the at least one camera lens within at least one of a horizontal and a vertical direction to a first lens line of sight representing a first of the plurality of points of view;

obtain a first of the plurality of image content portions of the image content;

move, via the lens controller, the at least one camera lens within at least one of the horizontal and the vertical direction to a second lens line of sight representing a second of the plurality of points of view;

obtain a second of the plurality of image content portions of the image content; and

determine that the obtained first and second of the plurality of image content portions of the image content in combination yield the complete image of the image content without the image obstruction located within the image foreground;

combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

store the combined complete image to a memory.

9. The computer program product of claim 8, where, in causing the computer to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction, the computer readable program code when executed on the computer causes the computer to:

perform distance calculation processing of pixels within the field of view of the at least one camera lens;

determine relative distances of objects within the field of view of the at least one camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one camera lens;

determine that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identify the object as the image obstruction based upon its location within the distance range of exclusion.

10. The computer program product of claim 8, where, in causing the computer to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction the computer readable program code when executed on the computer causes the computer to:

perform image recognition of objects within the field of view of the at least one camera lens;

compare recognized objects within the field of view of the at least one camera lens with image items within a database of configured obstruction images;

determine that at least one image item within the database of obstruction images matches the object located within the image foreground; and

identify the object as the image obstruction.

11. The computer program product of claim 10, where, in causing the computer to compare recognized objects within the field of view of the at least one camera lens with image items within a database of obstruction images the computer readable program code when executed on the computer causes the computer to:

retrieve a user image omission profile from a database comprising image item database identifiers of obstruction images configured for omission from captured images; and

compare the recognized objects within the field of view of the at least one camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

12. The computer program product of claim 8, where the computer readable program code when executed on the computer further causes the computer to:

display the image content within the field of view of one of the at least one camera lens on a display; and

where, in causing the computer to identify an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction the computer readable program code when executed on the computer causes the computer to;

detect a touchscreen selection on an area of a display surface of the display;

identify a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identify an image area within the boundary of the object as the image obstruction.

13. The computer program product of claim 12, where the image area within the boundary of the image comprises a selected object and where the computer readable program code when executed on the computer further causes the computer to:

configure a user image omission profile with the selected object; and

add the selected object to a database of obstruction images.

14. The computer program product of claim 8, where, in causing the computer to combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground the computer readable program code when executed on the computer causes the computer to perform image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.

15. A method, comprising:

identifying, via an image processing module, an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction;

determining, responsive to identifying the image obstruction, a plurality of points of view of the at least one camera lens that provide a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground, comprising;

moving, via a lens controller, the at least one camera lens within at least one of a horizontal and a vertical direction to a first lens line of sight representing a first of the plurality of points of view;

obtaining a first of the plurality of image content portions of the image content;

moving, via the lens controller, the at least one camera lens within at least one of the horizontal and the vertical direction to a second lens line of sight representing a second of the plurality of points of view;

obtaining a second of the plurality of image content portions of the image content; and

determining that the obtained first and second of the plurality of image content portions of the image content in combination yield the complete image of the image content without the image obstruction located within the image foreground;

combining the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

storing the combined complete image to a memory.

16. The method of claim 15, where identifying, via an image processing module, an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction comprises:

performing distance calculation processing of pixels within the field of view of the at least one camera lens;

determining relative distances of objects within the field of view of the at least one camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one camera lens;

determining that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identifying the object as the image obstruction based upon its location within the distance range of exclusion.

17. The method of claim 15, where identifying, via an image processing module, an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction comprises:

performing image recognition of objects within the field of view of the at least one camera lens;

comparing recognized objects within the field of view of the at least one camera lens with image items within a database of configured obstruction images;

determining that at least one image item within the database of obstruction images matches the object located within the image foreground; and

identifying the object as the image obstruction.

18. The method of claim 17, where comparing recognized objects within the field of view of the at least one camera lens with image items within a database of obstruction images comprises:

retrieving a user image omission profile from a database comprising image item database identifiers of obstruction images configured for omission from captured images; and

comparing the recognized objects within the field of view of the at least one camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

19. The method of claim 15, further comprising:

displaying the image content within the field of view of one of the at least one camera lens on a display; and

where identifying, via an image processing module, an object located within an image foreground relative to image content within a field of view of at least one camera lens as an image obstruction comprises;

detecting a touchscreen selection on an area of a display surface of the display;

identifying a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identifying an image area within the boundary of the object as the image obstruction.

20. The method of claim 19, where the image area within the boundary of the image comprises a selected object and further comprising:

configuring a user image omission profile with the selected object; and

adding the selected object to a database of obstruction images.

21. The method of claim 15, where combining the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground comprises performing image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.