Apparatus and method for transmitting synchronized the five senses with a/v data

• US 20070126927A1
• Filed: 12/30/2003
• Published: 06/07/2007
• Est. Priority Date: 11/12/2003
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Lee, Soo-In, Yun, Kug-Jin, Ahn, Chung-Hyun, Cho, Suk-Hee, Hur, Nam-Ho, Kang, Hoon-Jong
• Application Number:
  US10/579,349
• Publication Number:
  US 20070126927A1
• Time in Patent Office: Days
• US Class Current: 348/473
• CPC Class Codes: A63J 17/00 Apparatus for performing colour-music
• CPC Class Codes: A63J 2005/001 enhancing the performance by involving senses complementary to sight or hearing
• CPC Class Codes: A63J 2005/003 Tactile sense
• CPC Class Codes: A63J 2005/008 Smell sense
• CPC Class Codes: A63J 25/00 Equipment specially adapted for cinemas cinematographic projection means G03B
• CPC Class Codes: A63J 5/00 Auxiliaries for producing special effects on stages, or in circuses or arenas
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4112 having fewer capabilities than the client, e.g. thin client having less processing power or no tuning capabilities
• CPC Class Codes: H04N 21/43079 of additional data with content streams on multiple devices
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams

Abstract

Provided is a five sensory data synchronizing and transmitting apparatus and method, and an actual-feeling multimedia data providing system and method using the same. The five sensory data synchronizing and transmitting apparatus and method forms packets by describing vibration, an odor and a taste expressed in video/audio based on touch, odor and taste data descriptors and synchronizes the touch/odor/taste packets with video/audio packets on a frame basis; and then, the actual-feeling multimedia data providing system and method demultiplexes the received packets transmitted from the five sensory data synchronizing and transmitting apparatus into video data, audio data, touch data, odor data and taste data and transmits them to corresponding devices to thereby provide a user with an actual-feeling multimedia service.

Claims

1. An apparatus for synchronizing and transmitting five sensory data, comprising:

a video/audio data generating means for generating video/audio data by receiving multimedia data from an external device;

a touch data describing means for describing vibration expressed in the multimedia data received from the external device based on a predefined touch data descriptor;

an odor data describing means for describing an odor expressed in the multimedia data transmitted from the external device based on a predefined odor data descriptor;

a taste data describing means for describing a taste expressed in the multimedia data transmitted from the external device based on a predefined taste data descriptor;

a video/audio packet forming means for forming video/audio packets out of the video/audio data generated in the video/audio generating means;

a touch/odor/taste packet forming means for forming a touch packet, an odor packet, and a taste packet out of the touch, odor and taste data which are described in the touch data describing means, the odor data describing means, and the taste data describing means, respectively;

a multiplexing means for multiplexing the video/audio packets generated in the video/audio packet generating means with the touch packet, the odor packet and the taste packet formed in the touch/odor/taste packet forming means to thereby synchronize the video/audio packets with the touch/odor/taste packets; and

a transmitting means for transmitting a multiplexed packet multiplexed in the multiplexing means.

2. The apparatus as recited in claim 1, wherein the touch data describing means describes vibration expressed in the multimedia data transmitted from the external device based on a descriptor describing whether touch data are described;

a descriptor describing whether right/left movement is described;

a descriptor describing whether up/down movement is described;

a descriptor describing whether back/forth movement is described;

a descriptor describing a distance of movement;

a descriptor describing a speed of movement;

a descriptor describing an acceleration of movement;

a descriptor describing whether right/left rotation is described;

a descriptor describing an angle of right/left rotation;

a descriptor describing a speed of right/left rotation; and

a descriptor describing an acceleration of right/left rotation.

3. The apparatus as recited in claim 2, wherein the odor data describing means describes an odor expressed in the multimedia data transmitted from the external device based on a descriptor describing whether the odor data are described;

a descriptor describing a kind of the odor; and

a descriptor describing an intensity of the odor.

4. The apparatus as recited in claim 3, wherein the taste data describing means describes a taste expressed in the multimedia data transmitted from the external device based on a descriptor describing whether the taste data are described;

a descriptor describing a kind of the taste; and

a descriptor describing an intensity of the taste.

5. The apparatus as recited in claim 1, wherein the touch/odor/taste packet forming means forms a touch packet including information on whether the touch data are described, information on a packet length, and information on the touch data descriptors described in the touch data describing means;

an odor packet including information on whether odor data are described, information on an odor packet length, and information on the odor data descriptors described in the odor data describing means; and

a taste packet including information on whether taste data are described, information on a taste packet length, and information on the taste data descriptors described in the taste data describing means.

6. The apparatus as recited in claim 1, wherein the multiplexing means adds the touch/odor/taste packets formed in the touch/odor/taste packet forming means to the end of a plurality of video/audio packets generated in the video/audio generating means on a basis of the multimedia data frame to thereby multiplex and synchronize the video/audio packets with the touch/odor/taste packets.

7. A method for synchronizing and transmitting five sensory data, comprising the steps of:

a) generating video/audio data by receiving multimedia data from an external device;

b) describing vibration, an odor and a taste expressed in the multimedia data transmitted from the external device to generate touch data, odor data and taste data based on predefined touch, odor and taste data descriptors, respectively;

c) forming video/audio packets out of the video/audio data; and

forming a touch packet, an odor packet and a taste packet out of the touch data, the odor data and the taste data, respectively;

d) performing synchronization by multiplexing the video/audio packets, the touch packet, the odor packet and the taste packet; and

e) transmitting a multiplexed packet to a receiving part.

8. The method as recited in claim 7, wherein in the step b), the vibration expressed in the multimedia data transmitted from the external device is described based on a descriptor describing whether touch data are described;

a descriptor describing whether right/left movement is described;

a descriptor describing whether up/down movement is described;

a descriptor describing whether back/forth movement is described;

a descriptor describing a distance of movement;

a descriptor describing a speed of movement;

a descriptor describing an acceleration of movement;

a descriptor describing whether right/left rotation is described;

a descriptor describing an angle of right/left rotation;

a descriptor describing a speed of right/left rotation;

a descriptor describing an acceleration of right/left rotation;

the odor expressed in the multimedia data received from the external device is described based on a descriptor describing whether the odor data are described;

a descriptor describing a kind of the odor; and

a descriptor describing an intensity of the odor; and

,

the taste expressed in the multimedia data received from the external device is described based on a descriptor describing whether the taste data are described;

a descriptor describing a kind of the taste; and

a descriptor describing an intensity of the taste.

9. The method as recited in claim 7, wherein in the step d), the touch packet, the odor packet and the taste packet are added to the end of a plurality of video/audio packets on a basis of a multimedia data frame to thereby multiplex and synchronize the video/audio packets with the touch packet, the odor packet, and the taste packet.

10. A system for providing actual-feeling multimedia data, comprising:

a video/audio data generating means for generating video/audio data by receiving multimedia data from an external device;

a touch data describing means for describing vibration expressed in the multimedia data transmitted from the external device based on a predefined touch data descriptor;

an odor data describing means for describing an odor expressed in the multimedia data received from the external device based on a predefined odor data descriptor;

a taste data describing means for describing a taste expressed in the multimedia data received from the external device based on a predefined taste data descriptor;

a video/audio packet forming means for forming video/audio packets out of the video/audio data generated in the video/audio generating means;

a touch/odor/taste packet forming means for forming a touch packet, an odor packet, and a taste packet out of the touch, odor and taste data described in the touch data describing means, the odor data describing means, and the taste data describing means, respectively;

a multiplexing means for multiplexing the video/audio packets generated in the video/audio packet generating means and the touch packet, the odor packet and the taste packet formed in the touch/odor/taste packet forming means to thereby synchronize the video/audio packets with the touch/odor/taste packets;

a transmitting means for transmitting a multiplexed packet obtained in the multiplexing means;

a receiving means for receiving the multiplexed packet;

a demultiplexing means for demultiplexing the multiplexed packet received by the receiving means into the video data, the audio data, the touch data, the odor data and the taste data;

a video device for decoding and outputting the video data demultiplexed by the demultiplexing means;

an audio device for decoding and outputting the audio data demultiplexed by the demultiplexing means;

a vibration device for providing vibration to a user by interpreting the touch data demultiplexed by the demultiplexing means;

an odor device for spraying chemical aromatics to a user by interpreting the odor data demultiplexed by the demultiplexing means; and

a taste device for releasing a taste forming material to a user by interpreting the taste data demultiplexed by the demultiplexing means.

11. The system as recited in claim 10, wherein the demultiplexing means deletes network-related information from the received packet in form of a compressed stream by depacketizing, divides the depacketized packet into the video data, the audio data, the touch data, the odor data and the taste data on a basis of a multimedia data frame, and transmits the video data, the audio data, the touch data, the odor data and the taste data to corresponding devices based on header information.

12. The system as recited in claim 10, wherein the vibration device moves to right and left, back and forth, and up and down or rotates by interpreting the touch data, which are demultiplexed in the demultiplexing means, based on a predefined touch data descriptor;

and a starting time and a duration time of movement or rotation operation are synchronized with a moving picture and a sound outputted from the video device and the audio device, respectively.

13. The system as recited in claim 12, wherein the odor device sprays the chemical aromatics by interpreting the odor data, which are demultiplexed in the demultiplexing means, based on a predetermined odor data descriptor;

and a starting time and a duration time of spraying operation are synchronized with a moving picture and a sound outputted from the video device and the audio device, respectively.

14. The system as recited in claim 13, wherein the taste device releases taste forming materials by interpreting the taste data, which are demultiplexed in the demultiplexing means, based on a predetermined taste data descriptor;

and a starting time and a duration time of releasing operation are synchronized with a moving picture and a sound outputted from the video device and the audio device, respectively.

15. A method for providing actual-feeling multimedia data in an actual-feeling multimedia data providing system, comprising the steps of:

a) generating video/audio data by receiving multimedia data from an external device;

b) describing vibration, an odor and a taste expressed in the multimedia data transmitted from the external device to thereby generate touch data, odor data and taste data based on predefined touch, odor and taste data descriptors, respectively;

c) forming video/audio packets out of the video/audio data; and

forming a touch packet, an odor packet and a taste packet out of the touch data, the odor data and the taste data, respectively;

d) performing synchronization by multiplexing the video/audio packets with the touch packet, the odor packet and the taste packet;

e) transmitting a multiplexed packet to a receiving part;

f) receiving the multiplexed packet and demultiplexing the multiplexed packet received by the receiving means into the video data, the audio data, the touch data, the odor data and the taste data;

g) decoding and outputting the demultiplexed video data and the demultiplexed audio data;

h) providing a user with vibration by interpreting the demultiplexed touch data;

i) spraying chemical aromatics to the user by interpreting the demultiplexed odor data; and

j) a taste device for releasing taste forming materials to a user by interpreting the demultiplexed taste data.