Method and system for online presentations of writings and line drawings

• US 20010035976A1
• Filed: 02/13/2001
• Published: 11/01/2001
• Est. Priority Date: 02/15/2000
• Status: Abandoned Application

• Current Assignee: Poon, Andrew
• Original Assignee: Poon, Andrew
• Inventors: Poon, Andrew
• Application Number:
  US09/785,050
• Publication Number:
  US 20010035976A1
• Time in Patent Office: Days
• US Class Current: 358/1.15
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00286 with studio circuitry, devices or equipment, e.g. television cameras television studio circuitry, devices or equipment per se H04N5/222

Abstract

A method and system for enabling writings and/or drawings created during or in advance of a virtual meeting or the like to be electronically delivered to an online audience or stored for subsequent on-demand viewing such that the writings and/or drawings may be replicated on an audience member'"'"'s computer in a manner that makes them clearly readable. The invention is implemented via a software application that runs on a computer to which a video capture device is connected. The software application and/or computer peripheral components process captured video content to filter out data that do not pertain to the writings and/or drawings, based on the unique characteristics of writings and drawings as compared with other artifacts that may occupy the visual images. The remaining pertinent data is then transmitted to the on-line audience or saved for later on-demand viewing. In an additional implementation, a composite image comprising a writing area portion and an additional portion of the visual content of the presentation is replicated for online viewing.

Claims

1. A method for processing visual content corresponding to writings and/or line drawings presented during a presentation such that such visual content may be replicated for viewing by persons not attending the presentation, comprising:

directing a video capture device at a writing surface such that the writing surface occupies a substantial portion of a field of view of the video capture device;

capturing visual content with the video capture device pertaining to writings and/or line drawings created on the writing surface during the presentation or prepared on the writing surface in advance of the presentation, thereby producing a plurality of frames of pixilated data; and

cleaning up the visual content that is captured by processing the frames of pixilated data to remove data corresponding to artifacts in the visual content that do not pertain to the writings and/or line drawings through application of a set of image processing functions that remove such data based on unique characteristics of writings and/or line drawings that are used to distinguish pixilated data pertaining to the writings and/or line drawings from the pixilated data pertaining to the artifacts.

2. The method of

claim 1

, further comprising compressing the frames of pixilated data after the frames of pixilated data have been cleaned up.

3. The method of

claim 2

, further comprising:

transmitting the frames of the pixilated data that have been compressed over a network to an on-line audience member'"'"'s computer;

decoding the frames of pixilated data at the on-line audience member'"'"'s computer to produce a replication of the visual content of the presentation on the on-line audience member'"'"'s computer.

4. The method of

claim 3

, further comprising:

capturing audio content produced during the presentation;

converting the audio content into compressed audio data;

transmitting the compressed audio data over the network to the on-line audience member'"'"'s computer; and

decompressing the compressed audio data and applying further processing of the audio data on the on-line audience member'"'"'s computer so as to replicate the audio content of the presentation at the on-line audience member'"'"'s computer, in substantial synchrony with the visual content that is replicated.

5. The method of

claim 2

, further comprising storing the compressed frames of pixilated data into a file so as to enable on-demand viewing of the presentation at a later point in time.

6. The method of

claim 1

, wherein the video capture device produces data having color attributes, and wherein the set of processing functions includes converting the data with color attributes into grayscale data.

7. The method of

claim 1

, wherein the set of processing functions include performing a frame averaging function whereby the pixilated data values for a given frame are determined by averaging pixilated data values over a plurality of frames.

8. The method of

claim 1

, wherein the set of processing functions includes a flat field correction function that removes undesired artifacts including shadows, reflections, and lighting variations from the image data by performing a two-dimensional high-pass filter to remove low frequency pixel variations in the frames.

9. The method of

claim 1

, wherein the set of processing functions includes a thresholding function comprising converting the value of each pixel to either a binary one or zero based on whether an attribute of that pixel falls above or below a threshold value, said threshold value comprising a predetermined value based on one of characteristics corresponding to anticipated subject matter for the presentation, a user specified value, a calculated value based on a frame-by-frame analysis, or a calculated value based on analysis of data corresponding to various areas within the same frame.

10. The method of

claim 1

, wherein the set of processing functions includes performing a morphological filtering function comprising changing data values of individual pixels and/or small groups of pixels that have discontinuities with data values of adjacent pixels such that the discontinuities are removed.

11. The method of

claim 1

, wherein a color of the writing surface is defined as a background color, and wherein the set of processing functions includes grouping substantially adjacent pixels with a color other than the background color into blobs.

12. The method of

claim 11

, wherein the blobs are classified as (a) writings on the writing surface or (b) objects between the video capture device and the writing surface based on features of each blob, said features including at least one of:

a number of pixels in the blob;

a width of a bounding box encompassing the blob;

a height of a bounding box encompassing the blob;

a ratio of a number of pixels in the blob versus the number of pixels in a bounding box encompassing the blob; and

the color(s) of the pixels in the blob.

13. The method of

claim 12

, wherein the set of processing functions further includes discarding pixels belonging to blobs that are classified as objects between the video capture device and the writing surface.

14. The method of

claim 1

, wherein the set of processing functions includes classifying each pixel into one of N color categories, where 2<

=N<

=M band M<

=8, based on the color of that pixel and/or the color of the pixels in the vicinity of that pixel.

15. The method of

claim 14

, wherein 2<

=N<

=5 corresponding to pixels that are not the color of the writing surface being categorized as being black, red, green and blue.

16. The method of

claim 1

, wherein the set of processing functions includes performing an image registration function enabling data corresponding to frames that are captured while the video capture device may have been shifted relative to the writing surface to be aligned with frames captured prior to the video capture device being shifted relative to the writing surface.

17. The method of

claim 1

, wherein the set of processing functions includes:

performing a subtraction function, whereby data values for pixels corresponding to a previous frame are subtracted from data values for those pixels in a current frame; and

discarding data corresponding to pixel values that have not changed between the previous frame and the current frame.

18. The method of

claim 17

, further comprising:

determining if a frame comprises irrelevant data based on whether the data values after subtraction for selected pixels or for a number of pixels spread out over a substantial area of the frame exceed a threshold indicating that there is a substantial difference between the data values in the previous and current frames; and

discarding those frames that are determined to comprise irrelevant data.

19. The method of

claim 18

, wherein a count is maintained comprising a number of sequential frames that have been discarded, further comprising forcing a discarded frame to be retrained if the count reaches a threshold value.

20. The method of

claim 17

, wherein after subtraction function is performed, discarded data corresponding to pixel values that have not changed between the previous frame and the current frame are saved into a reference frame by combining the discarded data with data saved from previous frames.

21. The method of

claim 20

, wherein the saved data are merged with previously saved data by adding the data and then averaging the resultant sum over the number of frames for which data is contributed.

22. The method of

claim 20

, wherein a thresholding function is applied to the data saved in the previous frames in order to remove data that exist in less than a desired number of frames.

23. The method of

claim 20

, wherein the reference frame can be retrieved on demand and transmitted or otherwise saved into a permanent medium.

24. The method of

claim 1

, further comprising:

enabling a user to select an area within the field of view of the video capture device in which the drawings and/or line drawings of the presentation are to be located;

identifying pixilated data corresponding to the area selected by the user and portions of the field of view outside of the area selected by the user; and

performing image processing on the pixelated data to clean up the visual content only on pixelated data corresponding to the area selected by the user.

25. A method for processing visual content corresponding to writings and/or line drawings presented during a presentation such that such visual content may be replicated for viewing by persons not attending the presentation, comprising:

directing a video capture device at a writing surface such that the writing surface occupies a substantial portion of a field of view of the video capture device;

capturing visual content with the video capture device pertaining to writings and/or line drawings created on the writing surface during the presentation or prepared on the writing surface in advance of the presentation, thereby producing a plurality of frames of pixilated data;

performing a flat field correction function that removes undesired artifacts including shadows, reflections, and lighting variations from the image data by performing a two-dimensional high-pass filter to remove low frequency pixel variations in the frames;

performing a blob analysis function comprising;

grouping substantially adjacent pixels with a color other than a background color of the writing surface into blobs; and

classifying the blobs into (a) writing or drawing marks on the writing surface or (b) objects between the video capture device and the writing surface based on features of each blob; and

removing pixelated data corresponding blobs that are classified as objects between the video capture device and the writing surface; and

performing a frame averaging function whereby the pixilated data values for a given frame are determined by averaging pixilated data values over a plurality of frames.

26. The method of

claim 25

, further comprising performing a thresholding function comprising converting the value of each pixel to either a binary one or zero based on whether an attribute of that pixel falls above or below a threshold value, said threshold value comprising a predetermined value based on one of characteristics corresponding to anticipated subject matter for the presentation, a user specified value, a calculated value based on a frame-by-frame analysis, or a calculated value based on analysis of data corresponding to various areas within the same frame.

27. The method of

claim 25

, further comprising:

performing a subtraction function, whereby data values for pixels corresponding to a previous frame are subtracted from data values for those pixels in a current frame; and

discarding data corresponding to pixel values that have not changed between the previous frame and the current frame.

28. A method for processing visual content corresponding to writings and/or line drawings presented during a presentation such that such visual content may be replicated over the Internet to an online audience, comprising:

directing a video capture device at a writing surface such that the writing surface occupies a substantial portion of a field of view of the video capture device;

capturing visual content with the video capture device pertaining to writings and/or line drawings created on the writing surface during the presentation or prepared on the writing surface in advance of the presentation, thereby producing a plurality of frames of pixilated data;

cleaning up the visual content that is captured by processing the frames of pixilated data to remove data corresponding to artifacts in the visual content that do not pertain to the writings and/or line drawings through application of a set of image processing functions that remove such data based on unique characteristics of writings and/or line drawings that are used to distinguish pixilated data pertaining to the writings and/or line drawings from the pixilated data pertaining to the artifacts;

compressing the frames of pixilated data after the frames of pixilated data have been cleaned up to produce encoded data;

transmitting the encoded data over the Internet to an on-line audience member'"'"'s computer;

decoding the encoded data at the on-line audience member'"'"'s to produce a replication of the visual content of the presentation on the on-line audience member'"'"'s computer.

29. The method of

claim 28

, further comprising:

capturing audio content produced during the presentation;

converting the audio content into compressed audio data;

transmitting the compressed audio data over the Internet to the on-line audience member'"'"'s computer; and

decoding the compressed audio data on the on-line audience member'"'"'s computer so as to replicate the audio content of the presentation at the on-line audience member'"'"'s computer, in substantial synchrony with the visual content that is replicated.

30. A method for processing visual content including a first portion corresponding to writings and/or line drawings presented during a presentation and a second portion corresponding to additional visual content corresponding to the presentation such that the visual content is replicated on an online audience member'"'"'s computer, comprising:

directing a video capture device at a writing surface such that the writing surface occupies a portion of a field of view of the video capture device;

enabling a user to define a first portion of the field of view of the video capture device corresponding to a writings area in which the writings and/or line drawings will be displayed during the presentation;

enabling the user to define a second portion of the field of view of the video capture device corresponding to an additional area of the visual content that is to be replicated for viewing by persons not attending the presentation

capturing visual content with the video capture device to produce a plurality of frames of pixilated data;

separating portions of the pixilated data into data corresponding to the writings area and the additional area;

cleaning up the pixilated data corresponding to the writings area to produce a first portion of encoded data by removing data corresponding to artifacts in the visual content that do not pertain to the writings and/or line drawings through application of a set of image processing functions that remove such data based on unique characteristics of writings and/or line drawings that are used to distinguish pixilated data pertaining to the writings and/or line drawings from the pixilated data pertaining to the artifacts;

applying conventional image processing techniques to the pixilated data corresponding to the additional area to produce a second portion of encoded data, wherein the conventional image processing technique reduces an amount of data that describes each frame;

transmitting the first and second portions of encoded data over a communications network to an on-line audience member'"'"'s computer; and

decoding the first and second portions of encoded data on the online audience member'"'"'s computer to produce a composite image that comprises a replication of both the writings area portion and the additional area portion of the visual content of the presentation.

31. The method of

claim 30

, wherein the conventional image processing technique comprises MPEG compression.

32. The method of

claim 30

, wherein the first and second portions of the encoded data are transmitted in a single stream of data.

33. The method of

claim 30

, wherein the first and second portions of the encoded data are transmitted in separate streams of data.

34. An article of manufacture comprising a medium on which a plurality of machine-readable instructions are stored, said machine-readable instructions when executed performing functions including:

capturing visual content with a video capture device that is directed at a writing surface such that the writing surface occupies a substantial portion of a field of view of the video capture device, said visual content pertaining to writings and/or line drawings created on the writing surface during the presentation or prepared on the writing surface in advance of the presentation, thereby producing a plurality of frames of pixilated data; and

cleaning up the visual content that is captured by processing the frames of pixilated data to remove data corresponding to artifacts in the visual content that do not pertain to the writings and/or line drawings through application a set of processing functions that remove such data based on unique characteristics of writings and/or line drawings that are used to distinguish pixilated data pertaining to the writings and/or line drawings from the pixilated data pertaining to the artifacts.

35. The article of manufacture of

claim 34

, wherein execution of the machine-readable instructions cleans up the visual content by performing the functions of:

performing a flat field correction function that removes undesired artifacts including shadows, reflections, and lighting variations from the image data by performing a two-dimensional high-pass filter to remove low frequency pixel variations in the frames;

performing a blob analysis function comprising;

grouping substantially adjacent pixels with a color other than a background color of the writing surface into blobs; and

classifying the blobs into (a) writing or drawing marks on the writing surface or (b) objects between the video capture device and the writing surface based on features of each blob; and

removing pixelated data corresponding blobs that are classified as objects between the video capture device and the writing surface; and

performing a frame averaging function whereby the pixilated data values for a given frame are determined by averaging pixilated data values over a plurality of frames.

36. A system for capturing visual content corresponding to writings and/or line drawings presented during a presentation such that such visual content may be replicated for viewing by persons not attending the presentation, comprising:

a first computer including;

a memory in which a plurality of machine instructions are stored;

a processor, coupled to the memory; and

a display screen; and

a video capture device, linked in communication with the computer;

wherein execution of the machine instructions on said processor causes the first computer to perform the functions of;

capturing visual content with a video capture device that is directed at a writing surface such that the writing surface occupies a substantial portion of a field of view of the video capture device, said visual content pertaining to writings and/or line drawings created on the writing surface during the presentation or prepared on the writing surface in advance of the presentation, thereby producing a plurality of frames of pixilated data; and

cleaning up the visual content that is captured by processing the frames of pixilated data to remove data corresponding to artifacts in the visual content that do not pertain to the writings and/or line drawings through application a set of processing functions that remove such data based on unique characteristics of writings and/or line drawings that are used to distinguish pixilated data pertaining to the writings and/or line drawings from the pixilated data pertaining to the artifacts.

37. The system of

claim 36

, further comprising a video adapter coupled to the computer, said video adapter processing analog input from the video capture device to produce the plurality of frames of pixilated data.

38. The system of

claim 36

, further comprising:

a microphone; and

an audio adapter coupled to the computer and receiving audio input signals from the microphone, said audio adapter converting the audio input signals into a digital format.

39. The system of

claim 36

, further comprising:

a second computer linked to the first computer via a network connection, said second computer including;

a memory in which a plurality of machine instructions are stored;

a processor, coupled to the memory; and

a display screen,

wherein execution of the machine instructions by the processor in the first computer cause the first computer to further perform the functions of;

compressing the frames of pixilated data after the frames of pixilated data have been cleaned up to produce encoded data;

transmitting the encoded data over the network connection to the second computer,

and wherein execution of the machine instructions by the processor in the second computer causes the second computer to decode the encoded data were transmitted to the second computer to produce a replication of the visual content of the presentation on the display screen of the second computer.