Method and apparatus for processing, displaying and communicating images

• US 6,388,654 B1
• Filed: 10/02/1998
• Issued: 05/14/2002
• Est. Priority Date: 10/03/1997
• Status: Expired due to Term

• Current Assignee: McGraw-Hill Companies Incorporated
• Original Assignee: Tegrity, Inc. (McGraw-Hill Education, Inc.)
• Inventors: Halevy, Gilad, Platzker, Daniel M., Segal, Tsakhi, Biran, Liel
• Primary Examiner(s): WU, XIAO MIN
• Application Number:
  US09/166,211
• Time in Patent Office: 1,320 Days
• Field of Search: 345/156, 345/157, 345/158, 345/169, 345/7, 345/8, 345/9, 345/180, 345/179, 345/181, 345/182, 345/173, 345/1.1, 345/1.2, 345/2.1, 345/2.2
• US Class Current: 345/156
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 5/74 Projection arrangements for image reproduction, e.g. using eidophor

Abstract

The application discloses an apparatus and method of processing images, hand-drawn or written on a suitable Writing Surface, viewed by an Image Sensor such as a video camera, and captured by an image sensing circuit (i.e., frame grabber or similar) used for acquisition of image frames by a computer. More particularly, this invention discriminates among changes detected in these Viewed Images in order to identify and disregard non-informational, transient and/or redundant content. Removal of such content, a writer'"'"'s arm for example, from the captured image facilitates isolating meaningful changes, specifically intentional new Writings and Erasures appearing on the Writing Surface. Preserving only meaningful changes on the surface promotes optimized: “video image” storage and compressed transmission of a subset of the visual data, when used in conjunction with digital computers at remote locations optionally equipped with projected image and/or conventional computer display systems. The invention provides for displaying a composite of the remote sites'"'"' changes, transmitted over conventional communications channels, omitting Local Markings where the display device is a projector. The preferred embodiment of the present invention is applied to remote presentation aids, in particular “whiteboards” such as those typically used in educational lectures and commercial sales and training settings, and in particular to remote whiteboards. As will become clear, however, the application of the present invention is not restricted to whiteboards exclusively, but can work with any Writing Surface suitable for image acquisition (e.g., flipcharts or paper on a desk). Receive-only sites may participate in the remote presentation without contributing Local Markings to the composite. The present invention is compatible with conventional software products which will carry audio and other data streams on the same channel as the compressed image signals.

Claims

1. An apparatus for providing a portion of a composite image on a first surface to a second surface remote from said first surface, said apparatus including:

(a) an image sensor for providing a first signal indicative of images on said first surface;

(b) a first computer for receiving said first signal and responsive to a first stored signal to provide a differential signal;

(c) a second computer for receiving said differential signal and responsive to a second stored signal to provide a second signal; and

(d) a device for receiving said second signal to provide said composite image to said second surface.

2. The apparatus of claim 1, wherein said first computer includes an executable program in which said first signal is compared to said stored signal to provide said differential signal which is filtered to exclude signals indicative of interfering objects present in said first signal.

3. The apparatus of claim 2 in which said executable program further processes said first signal, said first signal having a succession of frames, examining said succession of frames for variations in said differential signal, thereby identifying said interfering objects present in said first signal.

4. The apparatus of claim 1, wherein said composite image comprises background information and markings and wherein the differential signal is filtered to exclude a background signal from the composite image.

5. The apparatus of claim 4, wherein said background signal is a projected signal, said apparatus further comprising a second device for providing said projected signal onto said first surface.

6. The apparatus of claim 5, wherein the projected signal changes periodically.

7. The apparatus of claim 5, wherein the projected signal changes frequently and is indicative of markings from additional remote surfaces.

8. The apparatus of claim 1, further comprising a storage medium for storing said composite image, said composite image comprising images from said first and second surfaces at a given time.

9. The apparatus of claim 1, further comprising a transmission interface for transmitting said composite image, said composite image comprising images from said first and second surfaces at a given time.

10. The apparatus of claim 7, further comprising a storage medium for storing said composite image, said composite image comprising images from said first, second and additional remote surfaces at a given time.

11. The apparatus of claim 7, further comprising a transmission interface for transmitting said composite image, said composite image comprising images from said first, second and additional remote surfaces at a given time.

12. An apparatus as in claim 1, wherein said second surface is a display screen.

13. An apparatus as in claim 1, wherein said first and second stored signals are the same signal.

14. A method of providing a signal indicative of markings made on a surface in a viewing field, which signal is not indicative of the presence of interfering objects should they be present in said viewing field, comprising the steps:

a. providing a signal indicative of said viewing field, said signal being composed of sequential frames of said viewing field;

b. detecting changes between successive ones of said frames of said viewing field to provide a series of changed viewing field signals;

c. filtering said series of said changed viewing field signals to exclude signals indicative of interfering objects to provide a series of filtered, changed viewing field signals;

d. detecting changes between successive ones of said series of filtered, changed viewing field signals to provide said signal indicative of markings made on said surface.

15. The method of claim 14, further providing a storage medium for storing at least one of said series of said filtered, changed viewing field signals.

16. The method of claim 14, further providing a transmission interface for transmitting at least one of said series of said filtered, changed viewing field signals.

17. The method of claim 14, wherein said viewing field comprises background information and wherein said method further comprises the step of filtering said signal indicative of said viewing field to exclude said background information.

18. An apparatus for providing a marked image onto a second surface, said marked image corresponding to markings on a first surface, said first surface having a complete image thereon comprising background information and said markings, said apparatus, comprising:

an image sensing circuit for capturing said complete image to provide a complete image signal;

a device responsive to a differential signal for providing said marked image onto said second surface; and

a unit for receiving said complete image signal and responsive to a background signal for providing said differential signal.

19. An apparatus as in claim 18 wherein said second surface is a display screen.

20. An apparatus as in claim 18 wherein said first and second stored signals are the same signal.