METHOD AND APPARATUS FOR GENERATING IMMERSIVE-MEDIA, MOBILE TERMINAL USING THE SAME

• US 20140082465A1
• Filed: 05/30/2013
• Published: 03/20/2014
• Est. Priority Date: 09/14/2012
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: JEE, Deock Gu, OH, Hyun Woo, YUN, Jae Kwan, HAN, Mi Kyong, KIM, Ji Yeon, JANG, Jong Hyun, PARK, Kwang Roh, PARK, Mi Ryong
• Application Number:
  US13/905,974
• Publication Number:
  US 20140082465A1
• Time in Patent Office: Days
• US Class Current: 715/202
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: H04N 1/00323 with a measuring, monitoring or signaling apparatus, e.g. for transmitting measured information to a central location
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3261 of multimedia information, e.g. a sound signal
• CPC Class Codes: H04N 2201/3264 of sound signals
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information

Abstract

A method and apparatus for generating immersive media and a mobile terminal using the method and apparatus is disclosed. An apparatus for generating immersive media comprises an image generation unit generating image data based on image signals, a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data, and an immersive media generation unit generating immersive media based on the sensory effect data.

Claims

1. An apparatus for generating immersive media, comprising:

an image generation unit generating image data based on image signals;

a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and

an immersive media generation unit generating immersive media based on the sensory effect data,

where the sensory effect data generation unit generates the sensory effect data including at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.

2. The apparatus of claim 1, wherein the sensory effect data generation unit comprises an image sensory effect extraction unit generating sensory effect data related to the image compliant with the image context based on the image signals;

a sound sensory effect extraction unit extracting a sound characteristic pattern based on the sound signals and generating sensory effect data related to the sound corresponding to the detected sound characteristic pattern;

an annotation generation unit generating sensory effect data based on the annotation information based on the user'"'"'s voice command;

a sensor sensory effect extraction unit obtaining sensing signals detected by the sensor module and based on the obtained sensing signals, generating the sensor-related sensory effect data;

an environment information generation unit obtaining environment information from the web through wireless communication and based on the obtained environment information, generating the environment information based sensory effect data; and

a metadata conversion unit converting the sensory effect data including at least one of the image-related sensory effect data, sound-related sensory effect data, annotation-related sensory effect data, sensor-related sensory effect data, and environment information-related sensory effect data into metadata.

3. The apparatus of claim 2, wherein the annotation generation unit comprises a voice recognition unit obtaining the voice command from the user and recognizing the voice command;

and an annotation information extraction unit determining whether the voice command is an annotation command existing in a predefined annotation table and if the voice command corresponds to the annotation command, generating the annotation information-based sensory effect data based on the voice command.

4. The apparatus of claim 2, wherein the environment information generation unit obtains the environment information including at least one piece of weather information from among temperature, humidity, wind, and the amount of rainfall corresponding to a predetermined time and location from the web.

5. The apparatus of claim 2, wherein the sensor module includes at least one of an orientation detection sensor, gravity sensor, accelerometer, illumination sensor, GPS sensor, ultraviolet sensor, and temperature sensor.

6. The apparatus of claim 1, wherein the image generation unit generates the image data by encoding the image signals obtained by a camera module during image capture and the sound signals obtained by a microphone module.

7. A mobile terminal, comprising:

a camera module obtaining image signals by capturing images;

an image generation unit generating image data based on image signals;

a sensory effect data generation unit generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and

an immersive media generation unit generating immersive media based on the sensory effect data,

where the sensory effect data generation unit generates the sensory effect data including at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.

8. A method for generating immersive media, comprising:

generating image data based on image signals;

generating sensory effect data by obtaining information related to a sensory effect for providing a sensory effect in conjunction with the image data; and

generating immersive media based on the sensory effect data,

where the sensory effect data includes at least one of sensory effect data related to an image obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.

9. The method of claim 8, wherein the annotation information-based sensory effect data is generated based on the voice command if the voice command is obtained from the user and recognized;

the recognized voice command is determined to correspond to an annotation command existing in a predefined annotation table; and

the voice command is the annotation command.

10. The method of claim 8, wherein the environment information-based sensory effect data is generated based on environment information obtained from the web through wireless communication.

11. The method of claim 10, wherein the environment information includes at least one piece of weather information from among temperature, humidity, wind, and the amount of rainfall corresponding to a predetermined time and location.

12. The method of claim 8, wherein the sensor module includes at least one of an orientation detection sensor, gravity sensor, accelerometer, illumination sensor, GPS sensor, ultraviolet sensor, and temperature sensor.

13. The method of claim 8, wherein the image-related sensory effect data includes a sensory effect compliant with image context based on the image signals.

14. The method of claim 8, wherein the sound-related sensory effect data includes a sound sensory effect data corresponding to a sound characteristic pattern detected from the sound signals.

15. The method of claim 8, wherein the image data is generated by encoding the image signals obtained by a camera module during image capture and the sound signals obtained by a microphone module.

16. A method for generating immersive media in a mobile terminal, comprising:

capturing images by using a camera module;

generating image data based on image signals obtained by the camera module;

generating sensory effect data by obtaining sensor effect-related information for providing a sensory effect in conjunction with the image data; and

generating immersive media based on the image data and the sensor effect data,

where the sensor effect data includes at least one of sensory effect data related to images obtained from the image signals, sensory effect data based on annotation information obtained from a user, sensor-related sensory effect data obtained from sensing signals detected by a sensor module, sensory effect data based on environment information obtained from the web, and sensory effect data related to sound obtained from sound signals.