APPARATUS AND METHOD FOR PLAYBACK OF AUDIO-VISUAL RECORDINGS

• US 20160277786A1
• Filed: 05/27/2016
• Published: 09/22/2016
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: 360ai Solutions LLC
• Original Assignee: 360Fly, Inc.
• Inventors: Rondinelli, Michael, Stentz, Anthony, Singh, Sanjiv, Herman, Herman
• Granted Patent: US 9,918,118 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/89 Time-base error compensation

Abstract

An imaging system comprising a panoramic visual image display, an associated directional sound playback device, and an associated motion reproduction device is disclosed. The imaging system conveys visual, sound and motion information related to a particular viewing direction to provide a realistic experience for the viewer. The imaging system can also comprise a panoramic visual image recording device capable of recording panoramic images, an associated directional sound capturing device capable of recording sound, and an associated directional motion capturing device capable of recording motion. Recorded panoramic images, sound and motion can be synchronously recorded to a common time code for simultaneous playback.

Claims

1. An audio-visual playback system comprising:

memory in which data representing recorded visual images and recorded audio is digitally stored to produce stored visual image data and stored audio data, wherein each recorded visual image is viewable from a plurality of viewing directions, and wherein the stored audio data is time-synchronized with the stored visual image data;

a user interface that facilitates user selection of one or more viewing directions of a recorded visual image; and

a processing unit operably coupled to the memory and the user interface, the processing unit operable to;

retrieve stored visual image data from the memory, the retrieved visual image data representing at least one recorded visual image;

communicate the retrieved visual image data to facilitate display of the at least one recorded visual image on a display device;

receive, through the user interface, user selection of a viewing direction of the plurality of viewing directions of a displayed visual image to produce a selected viewing direction;

responsive to the user selection, retrieve from the memory stored audio data, wherein the retrieved audio data is time-synchronized with the displayed visual image; and

communicate the retrieved audio data for play through an audio output such that the played audio corresponds to the selected viewing direction while the selected viewing direction is displayed on the display device.

2. The audio-visual playback system of claim 1, further comprising:

a motion reproduction device for reproducing motions associated with the recorded visual images, wherein at least one reproduced motion is time-synchronized with the displayed visual image and corresponds to the selected viewing direction.

3. The audio-visual playback system of claim 1, wherein the recorded visual images include panoramic visual images.

4. The audio-visual playback system of claim 1, wherein the processing unit forms part of a computer system, the display device is a computer monitor, and the audio output includes one or more speakers.

5. A processor-controlled method for playing back audio-visual content, the method comprising:

retrieving from memory digitally stored visual image data representing at least one recorded visual image, wherein each recorded visual image is viewable from a plurality of viewing directions;

communicating the retrieved visual image data to facilitate display of the at least one recorded visual image on a display device;

receiving user selection of a viewing direction of the plurality of viewing directions for a displayed visual image to produce a selected viewing direction;

responsive to the user selection, retrieving from memory digitally stored audio data representing recorded audio, wherein the retrieved audio data is time synchronized with the displayed visual image; and

communicating the retrieved audio data for play through an audio output such that the played audio corresponds to the selected viewing direction while the selected viewing direction is displayed on the display device.

6. The method of claim 5, further comprising:

automatically changing a viewing direction for the displayed visual image to produce a changed viewing direction; and

communicating the retrieved audio data for play through the audio output such that the played audio corresponds to the changed viewing direction while the changed viewing direction is displayed on the display device.

7. The method of claim 5, further comprising:

receiving user selection of a new viewing direction for the displayed visual image to produce a changed viewing direction; and

communicating the retrieved audio data for play through the audio output such that the played audio corresponds to the changed viewing direction while the changed viewing direction is displayed on the display device.

8. The method of claim 5, wherein communicating the retrieved audio data comprises:

modulating the retrieved audio data according to an alignment of an audio source with respect to the selected viewing direction.

9. The method of claim 5, wherein communicating the retrieved audio data comprises:

modulating a phase and a loudness of the retrieved audio data to emulate a position of an audio source with respect to the selected viewing direction.

10. The method of claim 5, wherein the recorded visual image data includes panoramic visual image data.

11. The method of claim 5, further comprising:

reproducing motions associated with the recorded visual images, wherein at least one reproduced motion is time-synchronized with the displayed visual image and corresponds to the selected viewing direction.

12. A processor-controlled method for playing back audio-visual content, the method comprising:

retrieving from memory digitally stored visual image data and audio data that was simultaneously recorded, the stored visual image data including a plurality of visual image samples, the stored audio data including a plurality of audio samples, each stored visual image sample and each stored audio sample including a respective time stamp, the stored visual image data representing at least one recorded visual image, wherein each recorded visual image is viewable from a plurality of viewing directions;

assembling the plurality of visual image samples and the plurality of audio samples based on respective time stamps thereof such that the plurality of visual image samples and the plurality of audio samples are time-synchronized to produce an assembled data stream;

communicating the assembled data stream to one or more playback devices such that, at any instant in time during playback, audio played through an audio output corresponds to a displayed visual image for which the played audio was recorded;

selecting a viewing direction of the plurality of viewing directions for the displayed visual image to produce a selected viewing direction; and

responsive to selection of the viewing direction, communicating the assembled data stream such that the played audio corresponds to the selected viewing direction while the selected viewing direction is displayed on the one or more playback devices.

13. The method of claim 12, wherein the stored visual image data represents at least one recorded panoramic visual image.

14. The method of claim 12, wherein selecting a viewing direction comprises:

receiving user selection of the viewing direction.

15. The method of claim 12, wherein selecting a viewing direction comprises:

automatically selecting the viewing direction.

16. The method of claim 12, wherein communicating the assembled audio stream such that the played audio corresponds to the selected viewing direction comprises:

modulating the plurality of audio samples according to an alignment of an audio source with respect to the selected viewing direction.

17. The method of claim 12, wherein communicating the assembled audio stream such that the played audio corresponds to the selected viewing direction comprises:

modulating a phase and a loudness of the plurality of audio samples to emulate a position of an audio source with respect to the selected viewing direction.