Method and apparatus for operating the overlay of computer-generated effects onto a live image

• US 6,175,343 B1
• Filed: 02/24/1998
• Issued: 01/16/2001
• Est. Priority Date: 02/24/1998
• Status: Expired due to Fees

• Current Assignee: Anivision, Inc.
• Original Assignee: Anivision, Inc.
• Inventors: Mitchell, Brian K., Wells, Robert V.
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): MARC-COLEMAN, MARTHE Y
• Application Number:
  US09/028,565
• Time in Patent Office: 1,057 Days
• Field of Search: 348/39, 348/115, 358/209, 358/87, 358/335, 358/629, 345/7, 345/8
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/16 Constructional details or arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A method and apparatus for observing exhibit sites which provides the ability to overlay computer generated images and sound on a live exhibit to create and present these overlays on demand in varying levels of detail for use by the general public, in indoor and outdoor settings, without requiring instruction, operators, fitting, training, or supervision. The viewing device may comprise a viewing portion, a processing portion, a displaying portion, and an audio portion. Each of these components combine to provide the user with the ability to view an actual exhibit site overlaid with informational or historic re-creations to enhance the exhibit experience, as opposed to a separate display booth or other means which takes the exhibit viewer'"'"'s attention away from the actual exhibit site.

Claims

1. An exhibit viewing device comprising:

an image head for housing an integrated electronics system, wherein the integrated electronics system is reconfigurable for the dynamic and correlated generation and overlay of visual effects onto a natural light source exhibit viewed through the exhibit viewing device; and

a head mount for user-controlled navigation of the image head.

2. The exhibit viewing device of claim 1 wherein the integrated electronic system is further operative to generate audio signals and wherein the image head further comprises a speaker system for broadcasting the audio signals.

3. The exhibit viewing device of claim 1 further comprising a position tracking system for determining the viewing elevation and viewing azimuth of the exhibit viewing device so that the visual effects may be coordinated, generated and overlaid onto the proper natural light source exhibit viewed through the exhibit viewing device.

4. The exhibit viewing device of claim 3 wherein the position tracking system is an electromagnetic measurement device.

5. The exhibit viewing device of claim 3 wherein the position tracking system comprises shaft encoders.

6. The exhibit viewing device of claim 1 wherein the integrated electronics system comprises an optical unit for overlaying the visual effects onto the natural light source exhibit.

7. The exhibit viewing device of claim 1 wherein the image head further comprises a viewer through which the user views the natural light source exhibit and overlaid visual effects.

8. The exhibit viewing device of claim 1 wherein the device is weatherized for outdoor use to enable outdoor exhibits to be overlaid.

9. The exhibit viewing device of claim 1 wherein the image head is secured from an overhead structure.

10. The exhibit viewing device of claim 1 wherein the image head is received in a weatherized kiosk frame, similar to ATM machines.

11. The exhibit viewing device of claim 1 further comprising a stand connected to the head mount for stability of the exhibit viewing device and user access to the exhibit viewing device.

12. The exhibit viewing device of claim 1 further comprising a booth, wherein the booth encloses the image head and the head mount.

13. The exhibit viewing device of claim 1, wherein the electronic system is connected via a network to electronic systems in other similar viewing devices in order to exchange visual effects among the viewing devices.

14. The viewing system as recited in claim 1, wherein said integrated electronics system includes means for recognizing live, viewed objects contained within said exhibit and locking said over-laid visual effects onto selected said viewed objects such that movement of said image head does not alter the position of said over-laid visual effects with respect to said viewed objects.

15. For an electronic system for viewing an exhibit site incorporated into a viewing device, a method for overlaying a computer-generated image onto a viewed image of the exhibit site, comprising the steps of:

determining a horizontal and vertical position of the viewing device;

generating computer animation regarding the particular location of the exhibit site; and

overlaying the computer animation onto a lens view of the viewing device.

16. The method recited in claim 15 further comprising the step of determining whether a token has been inserted into the viewing device before determining the position of the viewing device.

17. The method recited in claim 15 further comprising the step of providing audio effects and the lens view to the user before guiding the user to move the viewing device.

18. The method recited in claim 15 further comprising the step of guiding a user of the viewing device to move the viewing device to a particular location of the exhibit site.

19. The method recited in claim 18 wherein the step of guiding the user of the viewing device to move the viewing device to a particular location of the exhibit site comprises displaying arrows on the lens view of the viewing device, wherein the arrows correspond to the direction which the user should move the viewing device to reach the particular location.

20. The method recited in claim 15 further comprising the step of recognizing a user'"'"'s request for more information regarding the particular location before generating computer animation regarding the particular location of the exhibit site.

21. The method recited in claim 15 wherein the step of determining a position of the viewing device comprises determining the direction the viewing device is positioned relative to an initial orientation of the viewing device.

22. A computer-readable medium having computer-executable instructions for performing the steps recited in claim 15.

23. The method recited in claim 15 further comprising the step of determining what the viewer is looking at and automatically adjusting a camera focus setting and the computer'"'"'s presentation of scaled information overlays to allow support for multiple exhibits of varying distances from the device.

24. The method recited in claim 15, wherein the method is an intuitive interaction augmented reality method, wherein users may easily navigate the unit, receive overlaid information, or request additional data without special instructions, fitting, or training.

25. The method of claim 15, wherein the step of generating computer animation further includes exchanging computer animation via a network with other electronic systems in other similar viewing devices.

26. A virtual-reality panoramic viewing device comprising:

an image head for housing at least one monitor, said image head capable of movement in a pan direction and a tilt direction;

at least one sensor for monitoring movement of the image head in the pan direction and tilt direction and generating pan direction data and tilt direction data; and

a computer for receiving the pan direction data and the tilt direction data and generating image data for transmittal to and display on the monitor.

27. The viewing device of claim 26 further comprising a head mount for user-controlled movement of the image head.

28. The viewing device of claim 26 wherein the at least one monitor is a pair of miniature cathode ray tubes (CRTs).

29. The viewing device of claim 28 further comprising an eyepiece in association with each of the miniature cathode ray tubes for viewing the image data on the miniature cathode ray tubes.

30. The viewing device of claim 26 wherein the computer is further capable of generating sound data for transmittal to and output via at least one speaker in the monitor.

31. The viewing device of claim 26, wherein the computer is connected via a network to computers in other similar viewing devices in order to exchange image data among the viewing devices.

32. The viewing device as recited in claim 26, wherein said image data generated by said computer includes image data for recreating an exhibit located in front of said viewing device and said computer includes means for recognizing live, viewed objects contained within said exhibit and locking over-laid visual effects onto selected said viewed objects such that movement of said image head does not alter the position of said over-laid visual effects with respect to said viewed objects.

33. The viewing device as recited in claim 26, wherein said computer includes means for acquiring live, viewed visual data from an exhibit located in front of said viewing device and means for recognizing selected objects contained within said exhibit.

34. A virtual-reality panoramic viewing system comprising:

an input subsystem for generating viewing direction data based on the directional view of a rotating head of the viewing system;

a control subsystem for receiving the viewing direction data and generating image data and sound data for transmittal to the output subsystem based on the viewing direction data; and

an output subsystem comprising at least one eyepiece and at least one electronic display screen for displaying the image data, and further comprising at least one speaker for broadcasting the sound data.

35. The viewing system of claim 34 wherein the input subsystem further comprises a control for zooming the image data.

36. The viewing system of claim 34, wherein neither the control subsystem nor the output subsystem are located in a single housing.

37. The viewing system of claim 34, wherein the control subsystem comprises a personal computer in electronic communication with said output subsystem.

38. The viewing system of claim 34, wherein the control subsystem is connected via a network to control subsystems in other similar viewing systems in order to exchange image data and sound data among the viewing systems.

39. The viewing system as recited in claim 34, wherein the control subsystem includes a position monitor, a position database operatively connected to the position monitor for identifying a selected viewing location from the viewing direction data, and a point of interest player for passing points of interest to a content manager.

40. The viewing system as recited in claim 34, wherein the image data generated by said control subsystem includes image data for recreating an exhibit located in front of said viewing system and wherein said control subsystem further includes means for recognizing live, viewed objects contained within said exhibit and locking over-laid visual effects onto selected said viewed objects such that movement of said output subsystem does not alter the position of said over-laid visual effects with respect to said viewed objects.

41. The viewing system as recited in claim 34, wherein said output subsystem includes means for acquiring live, viewed visual data contained within an exhibit located in front of said viewing system and said control subsystem includes means for recognizing selected objects contained within said exhibit.

42. A virtual-reality panoramic viewing system comprising:

means for displaying image data to a user of said viewing system, said displaying means including at least one eyepiece and an electronic display screen for converting stored and processed data into visual images;

means for housing said displaying means in an appropriate position to accommodate said user;

means for sensing and generating viewing direction data based upon the pan and tilt movements of said housing means;

means for receiving said viewing direction data and generating image data to said displaying means in response to said direction data.

43. The viewing system as recited in claim 42, wherein said image data includes pre-recorded image data representative of or associated with a live image, and wherein said image data is correlated to be over-laid onto said live image to alter a visual representation of said live image.

44. The viewing system of claim 42, wherein the means for generating image data is connected via a network to means for generating image data in other similar viewing systems in order to exchange image data among the viewing systems.

45. The viewing system as recited in claim 42, wherein said electronic display screen comprises a liquid crystal display and said displaying means includes means for passing image data to said liquid crystal display comprising a live exhibit image positioned in front of said housing means.

46. The viewing system as recited in claim 45, wherein said means for generating image data includes means for associating coordinate information with said viewing direction data and means for correlating said coordinate information with stored content for correlated overlay of images on data viewed by said displaying means.

47. The viewing system as recited in claim 42, wherein said means for generating image data includes means for associating coordinate information with said viewing direction data and means for correlating said coordinate information with stored content for correlated overlay of images on data viewed by said displaying means.