Image Processing Systems and Methods

• US 20180040111A1
• Filed: 02/12/2016
• Published: 02/08/2018
• Est. Priority Date: 02/13/2015
• Status: Active Grant

• Current Assignee: Plantronics, Inc.
• Original Assignee: Light Blue Optics Ltd (Lifesize Communications Incorporated)
• Inventors: BENET BALLESTER, Raul, KATZ, Eyal, MAOZ, Barak, CABLE, Adrian James, SMITH, Euan Christopher, MCCAUGHAN, Gareth John, Thornton, Shane
• Granted Patent: US 10,540,755 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

We describe a method of capturing writing or drawing on a whiteboard. The method comprises: inputting camera data for a succession of image frames, wherein the camera data is from a camera directed towards the whiteboard and the image frames comprise successive images of the whiteboard from the camera; and user filter processing data from said image frames to remove parts of said image frames corresponding to parts of a user or user pen writing or drawing on said whiteboard. The user-filter processing comprises filtering to distinguish between motion of the user/user pen parts in the image frames and writing/drawing image information in the image frames which appears or changes during said writing or drawing but which is thereafter substantially unchanging. The method outputs writing/drawing data from the user filter-processing, this defining captured writing or drawing from the whiteboard.

Claims

1. A method of capturing writing or drawing on a whiteboard, the method comprising:

inputting camera data for a succession of image frames, wherein said camera data is from a camera directed towards said whiteboard and said image frames comprise successive images of said whiteboard from said camera;

user filter processing data from said image frames to remove parts of said image frames corresponding to parts of a user or user pen writing or drawing on said whiteboard;

wherein said user filter processing comprises filtering to distinguish between motion of said user/user pen parts in said image frames and writing/drawing image information in said image frames which appears or changes during said writing or drawing but which is thereafter substantially unchanging; and

outputting writing/drawing data from said user filter processing, wherein said writing/drawing data defines captured writing or drawing on said whiteboard.

2. The method of claim 1, wherein said user filter processing comprises subdividing said image frames into blocks for blockwise processing, said blockwise processing comprising determining when the image content of a block has changed by less than a threshold over a user filter time interval;

and in response identifying that the block does not include user/pen parts.

3. The method of claim 2, wherein said determining comprises comparing corresponding blocks in frames at different times to determine a measure of a difference between pixel values of the blocks;

comparing said measure against said threshold;

determining a time period for which said measure is less than said threshold; and

identifying that the block does not include user/pen parts when said measure is less than said threshold for greater than said user filter time interval.

4. The method of claim 1, wherein said user filter processing comprises subdividing said image frames into blocks for blockwise processing, wherein the area of a block in square pixels is at least ten times the area of the tip of a pen writing or drawing on said whiteboard.

5. The method of claim 1, the method further comprising:

brightness or color normalization processing of said image frames for said user filter processing.

6. The method of claim 5, the method further comprising:

subdividing said image frames into blocks;

wherein said brightness or color normalization processing comprises normalizing the brightnesses or colors of pixels within said blocks.

7. The method of claim 6,wherein said normalizing the brightnesses or colors of pixels within said blocks comprises normalizing according to a brightness or color percentile greater than the 50^th percentile.

8. The method of claim 1, wherein said user filter processing comprises temporally filtering said image frames with a user filter time constant to distinguish said user/pen from said writing or drawing on said whiteboard.

9. The method of claim 8, the method further comprising spatially filtering said image frames to attenuate changes in illumination across said whiteboard as seen by said camera.

10. The method of claim 1, wherein said camera data is from a camera directed at an acute angle towards said whiteboard and said image frames comprise successive images of said whiteboard from said acute angle;

the method further comprising distortion correction processing to correct for distortion resulting from said acute angle direction of said camera.

11. The method of claim 10 wherein said distortion correction processing comprises applying a projective transform to said camera data to provide corrected image frame data for said image frames, and wherein said user filter processing operates on said corrected image frame data.

12. The method of claim 1, the method further comprising:

applying a fisheye distortion correction to said camera data.

14. The method of claim 1, the method further comprising:

cleaning said data from said user filter processing by blockwise thresholding.

15. The method of claim 1, the method further comprising:

successively updating said writing/drawing data to provide dynamic capture of said writing or drawing on said whiteboard.

20. The method of claim 1, the method further comprising:

using a camera to capture a succession of images of the whiteboard; and

transmitting said writing/drawing data to a remote location for sharing.

21. A non-transitory data carrier carrying processor control code to, when running, implement the method claim 1.

13. (canceled)

16-19. -19. (canceled)

22. Whiteboard sharing apparatus comprising:

a camera to be directed to capture an image from a whiteboard; and

signal processing apparatus, coupled to said camera and configured to;

input camera data for a succession of image frames, wherein said camera data is from a camera directed towards said whiteboard and said image frames comprise successive images of said whiteboard from said camera user filter process data from said image frames to remove parts of said image frames corresponding to parts of a user or user pen writing or drawing on said whiteboard;

wherein said user filter processing comprises filtering to distinguish between motion of said user/user pen parts in said image frames and writing/drawing image information in said image frames which appears or changes during said writing or drawing but which is thereafter substantially unchanging; and

output writing/drawing data from said user filter processing, wherein said writing/drawing data defines captured writing or drawing on said whiteboard.

23. The whiteboard sharing apparatus of claim 22, wherein said camera is directed at an acute angle towards said whiteboard and said image frames comprise successive images of said whiteboard from said acute angle;

and wherein said signal processing apparatus is further configured to process said image frames to correct for distortion resulting from said acute angle direction of said camera.

24. The whiteboard sharing apparatus of claim 22, wherein said user filter processing comprises subdividing said image frames into blocks for blockwise processing, said blockwise processing comprising determining when the image content of a block has changed by less than a threshold over a user filter time interval;

and in response identifying that the block does not include user/pen parts.

25. The whiteboard sharing apparatus of claim 22, wherein said signal processing apparatus is further configured to subdividing said image frames into blocks and normalize the brightnesses or colors colours of pixels within said blocks prior to said user filter processing.

26-45. -45. (cancelled)