Manipulation of electronic media using off-line media

• US 20020193975A1
• Filed: 06/19/2001
• Published: 12/19/2002
• Est. Priority Date: 06/19/2001
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Zimmerman, Thomas Guthrie
• Granted Patent: US 7,085,693 B2
• Time in Patent Office: Days
• US Class Current: 703/6
• CPC Class Codes: G09B 19/24 Use of tools
• CPC Class Codes: G09B 25/02 of industrial processes; of machinery
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

The system generally comprises a probe device, a computing device, an electronic media, a presentation device, and a suitable medium as a specimen. The probe is manipulated over the specimen and information from the probe is transmitted to the computing device. The computing device uses the probe information to identify the image and probe position on the specimen. Based upon the specimen and position of the probe on the specimen, electronic media is presented simulating an output of a real tool used to inspect an actual specimen. The hand manipulation of the probe coordinated with the display of the electronic media provides the visceral experience normally encountered by a user of the real tool. In one embodiment, the probe is a camera and the specimen an image identified by examination of an image feature database. In an alternative embodiment, the probe is a finger.

Claims

1. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, said system comprising:

a probe device comprising at least a hand-held probe section, said probe device transmitting information of said selected portion to said computer;

a computer storage medium, said storage medium retaining real tool data, said real tool data representative of an output of a real tool examining a selected portion of an actual item corresponding to said selected portion of said image, and

wherein said computer retrieves real tool data synchronized to said position and displays said real tool data.

2. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said off-line medium includes a plurality of icons printed thereon, each of said icons indicating a different real tool, selection of an icon by said hand-held probe part changing said multimedia data synchronized with a portion of said image so as to be representative of an output of the real tool indicated by said icon.

3. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein a computer display includes a plurality of icons printed thereon, each of said icons indicating a different real tool.

4. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

5. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said hand-held probe includes a camera to capture said selected portion.

6. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 5, further including image retrieval means to match said sampled image to stored reference images.

7. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 5, further including position determination means to determine relative position of said sampled image to said reference image.

8. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said hand-held probe section is a bar code reader and said off-line medium ha s a plurality of bar codes printed thereon juxtaposed with said image, each of said bar codes designating position data such that said bar code reader reads one of said bar codes, transmits data representative of said bar code, and said computer retrieves and displays data synchronized to said bar code.

9. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 8, wherein said bar codes are printed on an overlay superimposed on said off-line media.

10. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 8, wherein said bar codes are printed utilizing infrared ink.

11. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said probe device transmits position information such that said displayed data continuously changes synchronously to said hand-held probe parts movement across said image.

12. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using magnetic fields to determine the hand-held probe position.

13. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using electric fields to determine the hand-held probe position.

14. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 13, wherein said probe is part of a human hand.

15. A system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using ultrasonic sound to determine the hand-held probe position.

17. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, wherein said presentation device is a visual display.

18. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, further including a multimedia database wherein multimedia data is synchronized to movement of said hand-held imager over said off-line media.

19. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, wherein synchronization simulates the functions of real tools.

20. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 19, wherein real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

21. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, further comprising:

a camera in said hand-held imager, said camera imaging said selected region of off-line media and outputting a sampled image;

a database, containing digital representations of reference images, said reference images including selected regions of off-line media;

an image retriever, receiving said sampled image and identifying said sampled image as a selected region of a reference image in said database,

a position detector receiving said sampled images and outputting position of said sample image in said identified reference image, and

wherein said presentation device presents said electronic media based on said position and said identified reference image.

22. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21, wherein said electronic media is an image.

23. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21, wherein said electronic media is sound, including any of spoken work, music, or sound effects.

24. A system in which electronic data to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21 wherein response simulates the function of a real tool selected from the list of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

25. A system in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, as per claim 16, wherein said image is divided into a plurality of regions, each of said regions having a bar code printed therein, and is said electronic media representative of an output of a real tool examining a region of an actual item corresponding to said region of said image detected by a hand-held bar code reader used as said hand-held imager.

26. A system in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, as per claim 25, wherein said off-line medium includes a plurality of icons printed thereon, each of said icons indicating a different real tool, selection of an icon by said hand-held bar code reader changing said multimedia data synchronized with each bar code printed in a region of said image so as to be representative of an output of the real tool indicated by said icon.

27. A system in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, said image divided into a plurality of regions, each of said regions having a bar code printed therein, as per claim 25, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

28. A system in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, said image divided into a plurality of regions, each of said regions having a bar code printed therein, as per claim 16, wherein said image is an image of a structure of a database and navigation of said database is synchronized to movements of said hand-held bar code reader over said image.

30. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, as per claim 29, said method further comprising:

receiving icon information representative of one of a plurality of icons pointed to by said hand-held probe, said icon representative of a real tool;

changing said data synchronized with said location of said image so as to be representative of an output of the real tool indicated by said icon.

31. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, as per claim 29, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

16. A system in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, said system comprising:

a hand-held imager, said imager detecting a region of said off-line media and transmitting an electronic representation of said region to said computer;

a presentation device, said presentation device presenting electronic media, synchronized to said region of off-line media.

29. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, said method comprising:

receiving position information representative of a location of said image printed on said off-line medium said hand-held probe part is pointing to;

determining data synchronized to said location;

retrieving said data, said data representative of an output of a real tool examining a location of an actual item corresponding to said location of said image pointed to by said hand-held probe part, and

displaying said data.