Method and apparatus for producing and storing, on a resultant non-transitory storage medium, computer generated (CG) video in correspondence with images acquired by an image acquisition device tracked in motion with respect to a 3D reference frame

• US 8,330,812 B2
• Filed: 05/22/2010
• Issued: 12/11/2012
• Est. Priority Date: 05/30/1995
• Status: Expired due to Fees

• Current Assignee: Simulated Percepts, LLC
• Original Assignee: Simulated Percepts, LLC
• Inventors: Maguire, Jr., Francis J.
• Primary Examiner(s): Lee, Michael
• Application Number:
  US12/785,439
• Publication Number:
  US 20100231706A1
• Time in Patent Office: 934 Days
• Field of Search: 348 42- 60, 348113-124, 345 7- 9, 345/630, 345/632, 345/633, 345/672, 345/679, 345/681, 434/40, 434 42- 44, 434/34, 434/35
• US Class Current: 348/121
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

Apparatus and method of (1) navigating an image acquisition device with both translatory and attitudinal movements while acquiring successive images in an object space and at the same time sensing the translatory and attitudinal movements of the device with respect to a three-dimensional reference frame of the object space, (2) providing successive computer generated (CG) images produced by a computer workstation from the successive images captured by the device in synchronization with the sensed translatory and attitudinal movements of the device with respect to the three-dimensional reference frame, and (3) storing the successive CG images on a non-transitory storage medium for later retrieval by a playback device for presentation by the playback device of said successive CG images to at least one eye of a viewer in an image space for perception of the successive CG images. A non-transitory storage medium storing the successive CG images is also provided.

Claims

1. A method comprising:

navigating a camera in motion with both translatory movements and attitudinal movements while capturing successive images,

wherein said translatory movements comprise three-dimensional translations in position of said camera as said camera is navigated in motion with respect to a three-dimensional reference frame,

tracking said translatory movements and said attitudinal movements of said camera as said camera is navigated in motion with respect to said three-dimensional reference frame,

providing successive computer generated images produced by a computer workstation in correspondence with said successive images captured by said camera for passive viewing in correspondence with said tracked translatory movements and said tracked attitudinal movements of said camera navigated in motion with respect to said three-dimensional reference frame, and

storing said successive computer generated images on a non-transitory storage medium for later retrieval from said non-transitory storage medium by a playback device for presentation by the playback device of said successive computer generated images to at least one eye of a viewer for perception of said successive computer generated images in a passive viewing role.

2. The method of claim 1, wherein said successive computer generated images include successive nonuniform computer generated images, each successive nonuniform computer generated image having a highly detailed component and a lesser detailed component, wherein a position of the highly detailed component is changed in selected successive nonuniform computer generated images.

3. The method of claim 1, further comprising providing an encoded motion control signal based on at least one of said tracked translatory movements and said tracked attitudinal movements and storing said encoded motion control signal along with said computer generated images for playback by said playback device for controlling a motion-controlled apparatus in changing in at least one of translatory movements and attitudinal movements of said motion-controlled apparatus in synchronization with said at least one of said tracked translatory movements and said tracked attitudinal movements, wherein said motion control signal based on said tracked attitudinal movements includes one or more of attitudinal roll, pitch, and yaw motion control signal components and wherein said motion control signal based on tracked translatory movements includes one or more linear motion control signal components.

4. The method of claim 1, wherein said camera is a stereoscopic camera for simulating two eyes of a person and said images captured by said camera are stereoscopic images taken from said stereoscopic camera, wherein said stored successive images are successive stereoscopic images produced as computer generated stereoscopic images by said computer workstation in correspondence with said stereoscopic images taken from said stereoscopic camera simulating two eyes of the person intersecting at various depths, said computer generated stereoscopic images for viewing by two eyes of the viewer correspondingly intersecting at various depths.

5. The method of claim 1, wherein said stored successive images comprise a field of view with informational content concentrated or highlighted in a portion thereof that changes position within said field of view between said successive images, wherein among said successive images for presentation to said at least one eye of said viewer are distinct visual fixation images presented for attention of said viewer, each distinct visual fixation image comprising a distinct point of view with distinct resolution wherein said distinct resolution of said each distinct visual fixation image comprises a distinct position of said region of concentrated or highlighted informational content within said field of view for said attention of said viewer.

6. A non-transitory storage medium having successive computer generated images stored thereon according to the method of claim 1 as an encoded video signal configured for decoding by a decoder and presentation of said successive computer generated images.

7. A non-transitory storage medium having successive computer generated images stored thereon according to the method of claim 2 as an encoded video signal configured for decoding by a decoder and presentation of said successive computer generated images.

9. A non-transitory storage medium having successive computer generated images stored thereon according to the method of claim 4 as an encoded video signal configured for decoding by a decoder and presentation of said successive computer generated images.

10. The method of claim 1, wherein said camera is a head mounted camera.

21. The method of claim 1, wherein said camera is navigated by a person free to walk about with said camera in navigating said camera in motion while acting as a cameraman and said tracking is carried out by one or more sensors comprising at least one of gyro, accelerometer, radio, magnetic, or light beam sensors.

22. The non-transitory storage medium of claim 6, wherein said camera is navigated by a person free to walk about with said camera in navigating said camera in motion while acting as a cameraman and said tracking is carried out by one or more sensors comprising at least one of gyro, accelerometer, radio, magnetic, or light beam sensors.

8. A non-transitory storage medium having successive computer generated images stored thereon as an encoded video signal configured for decoding by a decoder and presentation of said successive computer generated images to a viewer in a passive viewing role, said successive computer generated images generated by a computer in correspondence with navigation of a device in motion with both translatory movements and attitudinal movements while acquiring a point of view, wherein said translatory movements comprise three-dimensional translations in position of said device, and wherein said translatory movements and said attitudinal movements of said device are tracked as said device is navigated in motion with respect to a three-dimensional reference frame, said successive computer generated images generated by said computer in correspondence with said point of view acquired by said device for passive viewing in correspondence with said tracked translatory movements and said tracked attitudinal movements of said device navigated in motion with respect to said three-dimensional reference frame, said successive computer generated images stored on said non-transitory storage medium for later retrieval by a playback device for presentation by said playback device of said successive computer generated images to at least one eye of a viewer for presentation of said successive computer generated images in a passive viewing role.

25. The non-transitory storage medium of claim 8, wherein said device is navigated by a person free to walk about with said device in navigating said device in motion and said device is tracked by one or more sensors comprising at least one of gyro, accelerometer, radio, magnetic, or light beam sensors.

11. An apparatus, comprising:

a processor; and

a memory including software, said memory and the software configured to, with the processor, cause the apparatus at least to;

receive tracked signals indicative of translatory movements and attitudinal movements of an image acquisition device tracked in motion with respect to a three-dimensional reference frame as said image acquisition device is navigated with respect to said three-dimensional reference frame while at the same time receiving at least one image signal containing successive images acquired by said image acquisition device navigated with respect to said three-dimensional reference frame, wherein said translatory movements comprise three-dimensional translations in position of said image acquisition device navigated with respect to said three-dimensional reference frame.

provide at least one computer generated image signal containing successive computer generated images in correspondence with said successive images acquired by said image acquisition device made for passive viewing in correspondence with said tracked translatory movements and said tracked attitudinal movements of said image acquisition device with respect to said three-dimensional reference frame during tracked navigation of said image acquisition device with said translatory movements and attitudinal movements,

encode said at least one computer generated image signal as an encoded video signal, and

store said encoded video signal for decoding and playback of said successive computer generated images to a viewer in a passive viewing role.

12. The apparatus of claim 11, wherein said successive computer generated images include successive nonuniform computer generated images, each successive nonuniform computer generated image having a highly detailed component and a lesser detailed component, wherein a position of the highly detailed component is changed in selected successive nonuniform computer generated images.

13. The apparatus of claim 11, wherein said memory and the software are further configured to, with the processor, cause the apparatus to provide an encoded motion control signal based at least one of said tracked translatory movements and said tracked attitudinal movements and storing said encoded motion control signal along with said encoded video signal for decoding by said playback device for controlling a motion-controlled apparatus in changing in at least one of translatory movements and attitudinal movements of said motion-controlled apparatus in synchronization with at least one of said tracked translatory movements and said tracked attitudinal movements, wherein said motion control signal based on tracked attitudinal movements includes one or more of attitudinal roll, pitch, and yaw motion control signal components and wherein said motion control signal based on tracked translatory movements includes one or more linear motion control signal components.

14. The apparatus of claim 11, wherein said image acquisition device is a stereoscopic camera.

15. The apparatus of claim 11, wherein said image acquisition device comprises at least one head mounted camera.

23. The apparatus of claim 11, wherein said image acquisition device is navigated by a person free to walk about with said device in navigating said device in motion and said tracked signals are sensed by one or more sensors comprising at least one of gyro, accelerometer, radio, magnetic, or light beam sensors.

16. An apparatus, comprising:

an image acquisition device for navigation while acquiring successive images as said image acquisition device is navigated in motion with navigational movements;

at least one translatory position and orientation tracking device responsive to said navigational movements of said image acquisition device with respect to a three-dimensional reference frame for providing at least one tracked movement signal indicative of translatory movements and orientation movements as said image acquisition device is navigated in motion with said navigational movements, wherein said translatory movements comprise three-dimensional translations in position of said camera with respect to three-dimensional reference frame; and

a computer workstation, responsive to said at least one sensed movement signal and to said successive images acquired by said image acquisition device, for producing successive computer generated images in correspondence with said successive images acquired by said image acquisition device navigated in motion with respect to said three-dimensional reference frame and for storing said successive computer generated images as an encoded video signal, wherein said successive computer generated images are produced for passive viewing in correspondence with said tracked navigational movements of said image acquisition device with respect to said three-dimensional reference frame, said stored encoded video signal for decoding and playback of said successive computer generated images to a viewer in a passive viewing role.

17. The apparatus of claim 16, wherein said successive computer generated images include successive nonuniform computer generated images, each successive nonuniform computer generated image having a highly detailed component and a lesser detailed component, wherein a position of the highly detailed component is changed in selected successive nonuniform computer generated images with respect to a field of view of the selected successive nonuniform computer generated images.

18. The apparatus of claim 16, wherein said computer workstation is also for providing an encoded motion control signal based on said at least one tracked movement signal comprising at least one of tracked translatory movements and tracked orientation movements of said image acquisition device and storing said encoded motion control signal along with said encoded video signal for decoding and playback for controlling a motion-controlled apparatus in changing in at least one of translatory position and orientation of said motion-controlled apparatus in synchronization with at least one of said tracked translatory movements and said tracked orientation movements, wherein said motion control signal based on tracked orientation movements includes one or more of roll, pitch, and yaw motion control signal components and wherein said motion control signal based on tracked translatory movements includes one or more linear motion control components.

19. The apparatus of claim 16, wherein said image acquisition device is a stereoscopic camera for simulating two eyes and said successive images acquired by said stereoscopic camera are stereoscopic images, wherein said stored successive computer generated images are successive computer generated stereoscopic images produced by said computer workstation in correspondence with said stereoscopic images acquired by said stereoscopic camera simulating two eyes intersecting at various depths.

20. The apparatus of claim 16, wherein said image acquisition device comprises at least one head mounted camera.

24. The apparatus of claim 16, wherein said image acquisition device is navigated by a person free to walk about with said device in navigating said device in motion and said tracking device comprises one or more sensors including at least one of gyro, accelerometer, radio, magnetic, or light beam sensors.