Visual obstruction removal with image capture

• US 9,191,642 B2
• Filed: 03/30/2012
• Issued: 11/17/2015
• Est. Priority Date: 04/12/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cudak, Gary D., Hardee, Christopher J., Humes, Randall C., Miller, Heather C.
• Primary Examiner(s): RAHAMAN, MOHAMMED S
• Application Number:
  US13/435,124
• Publication Number:
  US 20120262572A1
• Time in Patent Office: 1,327 Days
• Field of Search: 348/50, 348/135
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

An object located within an image foreground relative to image content within a field of view of at least one camera lens is identified, using image processing module, as an image obstruction. Responsive to identifying the image obstruction, points of view of at least one camera lens are determined that provide image content portions of the image content that, in combination, yield a complete image of the image content without the image obstruction located within the image foreground. The image content portions are combined to yield the complete image of the image content without the image obstruction located within the foreground. The combined complete image is stored to a memory.

Claims

1. A method, comprising:

by an image processing module of a camera;

identifying an object located within an image foreground relative to image content within a field of view of at least one movable camera lens of the camera as an image obstruction; and

responsive to identifying the image obstruction;

capturing, by real-time movement of the at least one movable camera lens using a lens position controller to a plurality of points of view of the at least one movable camera lens, a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground;

combining the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

storing the combined complete image to a memory.

2. The method of claim 1, where identifying the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens of the camera as the image obstruction comprises:

performing distance calculation processing of pixels within the field of view of the at least one movable camera lens;

determining relative distances of objects within the field of view of the at least one movable camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one movable camera lens;

determining that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identifying the object as the image obstruction based upon its location within the distance range of exclusion.

3. The method of claim 1, where identifying the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens of the camera as the image obstruction comprises:

performing image recognition of objects within the field of view of the at least one movable camera lens;

comparing recognized objects within the field of view of the at least one movable camera lens with image items within a database of configured obstruction images;

determining that at least one image item within the database of configured obstruction images matches the object located within the image foreground; and

identifying the object as the image obstruction.

4. The method of claim 3, where comparing the recognized objects within the field of view of the at least one movable camera lens with the image items within the database of configured obstruction images comprises:

retrieving a user image omission profile from the database comprising image item database identifiers of obstruction images configured for omission from captured images; and

comparing the recognized objects within the field of view of the at least one movable camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

5. The method of claim 1, further comprising:

displaying the image content within the field of view of one of the at least one movable camera lens on a display; and

where identifying the object located within the image foreground relative to the image content within the field of view the of at least one movable camera lens of the camera as the image obstruction comprises;

detecting a touchscreen selection on an area of a display surface of the display;

identifying a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identifying an image area within the boundary of the object as the image obstruction.

6. The method of claim 5, where the image area within the boundary of the image comprises a selected object and further comprising:

configuring a user image omission profile with the selected object; and

adding the selected object to a database of obstruction images.

7. The method of claim 1, where combining the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground comprises performing image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.

8. A system, comprising:

at least one movable camera lens of a camera;

a lens position controller;

a memory; and

a processor programmed to;

identify an object located within an image foreground relative to image content within a field of view of the at least one movable camera lens as an image obstruction; and

responsive to identifying the image obstruction;

capture, by real-time movement of the at least one movable camera lens using the lens position controller to a plurality of points of view of the at least one movable camera lens, a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground;

combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

store the combined complete image to the memory.

9. The system of claim 8, where, in being programmed to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens as the image obstruction, the processor is programmed to:

perform distance calculation processing of pixels within the field of view of the at least one movable camera lens;

determine relative distances of objects within the field of view of the at least one movable camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one movable camera lens;

determine that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identify the object as the image obstruction based upon its location within the distance range of exclusion.

10. The system of claim 8, where, in being programmed to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens as the image obstruction, the processor is programmed to:

perform image recognition of objects within the field of view of the at least one movable camera lens;

compare recognized objects within the field of view of the at least one movable camera lens with image items within a database of configured obstruction images;

determine that at least one image item within the database of configured obstruction images matches the object located within the image foreground; and

identify the object as the image obstruction.

11. The system of claim 10, where, in being programmed to compare the recognized objects within the field of view of the at least one movable camera lens with the image items within the database of configured obstruction images, the processor is programmed to:

retrieve a user image omission profile from the database comprising image item database identifiers of obstruction images configured for omission from captured images; and

compare the recognized objects within the field of view of the at least one movable camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

12. The system of claim 8, further comprising a display and where the processor is further programmed to:

display the image content within the field of view of one of the at least one movable camera lens on the display; and

where, in being programmed to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens as the image obstruction, the processor is programmed to;

detect a touchscreen selection on an area of a display surface of the display;

identify a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identify an image area within the boundary of the object as the image obstruction.

13. The system of claim 12, where the image area within the boundary of the image comprises a selected object and where the processor is further programmed to:

configure a user image omission profile with the selected object; and

add the selected object to a database of obstruction images.

14. The system of claim 8, where, in being programmed to combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground, the processor is programmed to perform image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.

15. A computer program product comprising a computer readable storage device including computer readable program code, where the computer readable program code when executed on a computer causes the computer to:

identify an object located within an image foreground relative to image content within a field of view of at least one movable camera lens of a camera as an image obstruction; and

responsive to identifying the image obstruction;

capture, by real-time movement of the at least one movable camera lens using a lens position controller to a plurality of points of view of the at least one movable camera lens, a plurality of image content portions of the image content that in combination yield a complete image of the image content without the image obstruction located within the image foreground;

combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground; and

store the combined complete image to a memory.

16. The computer program product of claim 15, where, in causing the computer to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens of the camera as the image obstruction, the computer readable program code when executed on the computer causes the computer to:

perform distance calculation processing of pixels within the field of view of the at least one movable camera lens;

determine relative distances of objects within the field of view of the at least one movable camera lens based upon the performed distance calculation processing of the pixels within the field of view of the at least one movable camera lens;

determine that the object is located at a distance within a configured distance range of exclusion relative to remaining image content; and

identify the object as the image obstruction based upon its location within the distance range of exclusion.

17. The computer program product of claim 15, where, in causing the computer to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens of the camera as the image obstruction, the computer readable program code when executed on the computer causes the computer to:

perform image recognition of objects within the field of view of the at least one movable camera lens;

compare recognized objects within the field of view of the at least one movable camera lens with image items within a database of configured obstruction images;

determine that at least one image item within the database of configured obstruction images matches the object located within the image foreground; and

identify the object as the image obstruction.

18. The computer program product of claim 17, where, in causing the computer to compare the recognized objects within the field of view of the at least one movable camera lens with the image items within the database of configured obstruction images, the computer readable program code when executed on the computer causes the computer to:

retrieve a user image omission profile from the database comprising image item database identifiers of obstruction images configured for omission from captured images; and

compare the recognized objects within the field of view of the at least one movable camera lens with a subset of the image items within the database of obstruction images configured within the user image omission profile for omission from captured images.

19. The computer program product of claim 15, where the computer readable program code when executed on the computer further causes the computer to:

display the image content within the field of view of one of the at least one movable camera lens on a display; and

where, in causing the computer to identify the object located within the image foreground relative to the image content within the field of view of the at least one movable camera lens of the camera as the image obstruction, the computer readable program code when executed on the computer causes the computer to;

detect a touchscreen selection on an area of a display surface of the display;

identify a boundary of the object, where the object is at least partially represented within the selected area of the display surface; and

identify an image area within the boundary of the object as the image obstruction.

20. The computer program product of claim 19, where the image area within the boundary of the image comprises a selected object, and where the computer readable program code when executed on the computer further causes the computer to:

configure a user image omission profile with the selected object; and

add the selected object to a database of obstruction images.

21. The computer program product of claim 15, where, in causing the computer to combine the plurality of image content portions to yield the complete image of the image content without the image obstruction located within the foreground, the computer readable program code when executed on the computer causes the computer to perform image registration of the plurality of image content portions to form the complete image of the image content without the image obstruction located within the foreground.