Manipulation of electronic media using off-line media

• US 7,085,693 B2
• Filed: 06/19/2001
• Issued: 08/01/2006
• Est. Priority Date: 06/19/2001
• Status: Expired due to Fees

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Zimmerman, Thomas Guthrie
• Primary Examiner(s): Rodriguez, Paul L.
• Assistant Examiner(s): STEVENS, THOMAS H
• Application Number:
  US09/883,376
• Publication Number:
  US 20020193975A1
• Time in Patent Office: 1,869 Days
• Field of Search: 703/6, 455/556.1, 345/179, 345/181, 434/262, 434/317
• US Class Current: 703/6
• CPC Class Codes: G09B 19/24 Use of tools
• CPC Class Codes: G09B 25/02 of industrial processes; of machinery
• CPC Class Codes: G09B 9/00 Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131

Abstract

The system generally comprises a probe device, a computing device, an electronic media, a presentation device, and a suitable medium as a specimen. The probe is manipulated over the specimen and information from the probe is transmitted to the computing device. The computing device uses the probe information to identify the image and probe position on the specimen. Based upon the specimen and position of the probe on the specimen, electronic media is presented simulating an output of a real tool used to inspect an actual specimen. The hand manipulation of the probe coordinated with the display of the electronic media provides the visceral experience normally encountered by a user of the real tool. In one embodiment, the probe is a camera and the specimen an image identified by examination of an image feature database. In an alternative embodiment, the probe is a finger.

Claims

1. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, said system comprising:

a probe device comprising at least a hand-held probe section, said probe device transmitting information of said selected portion to said computer;

a computer storage medium, said storage medium retaining multimedia data, said multimedia data representative of an output of a real tool examining a selected portion of an actual item corresponding to said selected portion of said image, and

wherein said computer retrieves said multimedia data synchronized to a position of said selected portion and displays said multimedia data to simulate a real tool.

2. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said off-line medium includes a plurality of icons printed thereon, each of said icons indicating a different real tool, selection of an icon by said hand-held probe part changing said multimedia data synchronized with a portion of said image so as to be representative of an output of the real tool indicated by said icon.

3. A computer-based systen for simulating a real tool in which a computer display data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein a computer display includes a plurality of icons printed thereon, each of said icons indicating a different real tool.

4. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

5. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said hand-held probe includes a camera to capture an image of said selected portion and outputs a sampled image.

6. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 5, further including image retrieval means to match said sampled image to stored reference images.

7. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 5, further including position determination means to determine relative position of said sampled image to said reference image.

8. A computer-based system for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said hand-held probe section is a bar code reader and said off-line medium ha a plurality of bar codes printed thereon juxtaposed with said image, each of said bar codes designating position data such that said bar code reader reads one of said bar codes, transmits data representative of said bar code, and said computer retrieves and displays data synchronized to said bar code.

9. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 8, wherein said bar codes are printed on an overlay superimposed on said off-line media.

10. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 8, wherein said bar code reader recognizes said bar codes printed with infrared ink.

11. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 1, wherein said probe device transmits position information such that said displayed data continuously changes synchronously to said hand-held probe parts movement across said image.

12. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using magnetic fields to determine the hand-held probe position.

13. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using electric fields to determine the hand-held probe position.

14. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein a user uses a part of his hand as said probe.

15. A computer-based systen for simulating a real tool in which a computer displays data based upon a selected portion of an image printed on an off-line medium, as per claim 11, wherein said probe device is a digitizer using ultrasonic sound to determine the hand-held probe position.

16. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, said system comprising:

a hand-held imager, said imager detecting a region of said off-line media and transmitting an electronic representation of said region to said computer;

a presentation device, said presentation device presenting electronic media, synchronized to said region of off-line media.

17. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, wherein said presentation device is a visual display.

18. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, further including a multimedia database storing said electronic media to be presented wherein said electronic media is synchronized to movement of said hand-held imager over said off-line media.

19. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, wherein said synchronization simulates the functions of real tools.

20. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 19, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

21. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 16, further comprising:

a camera in said hand-held imager, said camera imaging said selected region of off-line media and outputting a sampled image;

a database, containing digital representations of reference images, said reference images including selected regions of off-line media;

an image retriever, receiving said sampled image and identifying said sampled image as a selected region of a reference image in said database,

a position detector receiving said sampled images and outputting position of said sample image in said identified reference image, and

wherein said presentation device presents said electronic media based on said position and said identified reference image.

22. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21, wherein said electronic media is an image.

23. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21, wherein said electronic media is sound, including any of spoken work, music, or sound effects.

24. A computer-based systen in which electronic media to be presented by a computing device is synchronized to a selected region of off-line medium, as per claim 21 wherein said presented electronic media simulates the function of a real tool selected from the list of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

25. A computer-based systen in which electronic media to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, as per claim 16, wherein said image is divided into a plurality of regions, each of said regions having a bar code printed therein, and said electronic media representative of an output of a real tool examining a region of an actual item corresponding to said region of said image detected by a hand-held bar code reader used as said hand-held imager.

26. A computer-based systen in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, as per claim 25, wherein said off-line medium includes a plurality of icons printed thereon, each of said icons indicating a different real tool, selection of an icon by said hand-held bar code reader changing said electronic media synchronized with each bar code printed in a region of said image so as to be representative of an output of the real tool indicated by said icon.

27. A computer-based systen in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, said image divided into a plurality of regions, each of said regions having a bar code printed therein, as per claim 25, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.

28. A computer-based systen in which data to be displayed by a computer is synchronized to a selected region of an image printed on a off-line medium, said image divided into a plurality of regions, each of said regions having a bar code printed therein, as per claim 25, wherein said image is an image of a structure of a database and navigation of said database is synchronized to movements of said hand-held bar code reader over said image.

29. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, said method comprising:

receiving position information representative of a location of said image printed on said off-line medium said hand-held probe part is pointing to;

determining data synchronized to said location;

retrieving said data, said data representative of an output of a real tool examining a location of an actual item corresponding to said location of said image pointed to by said hand-held probe part, and

displaying said data.

30. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, as per claim 29, said method further comprising:

receiving icon information representative of one of a plurality of icons pointed to by said hand-held probe, said icon representative of a real tool;

changing said data synchronized with said location of said image so as to be representative of an output of the real tool indicated by said icon.

31. A method of simulating a real tool in which a computer displays data based upon a selection of a location of an image printed on a off-line medium by a hand-held probe, said off-line medium including a plurality of icons printed thereon, each of said icons indicating a different real tool, as per claim 29, wherein said real tool is any one of a telescope, spectrum analyzer, radio telescope, magnetometer, scale, seismometer, ground penetrating radar, x-ray, pH device, thermometer, stethoscope, electrophoretic device, Geiger counter, chemical assay device, book reader, word pronouncer, book translator, or dictionary.