METHOD AND APPARATUS FOR GENERATING METADATA OF IMMERSIVE MEDIA

• US 20140148220A1
• Filed: 11/21/2013
• Published: 05/29/2014
• Est. Priority Date: 11/23/2012
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: PARK, Mi Ryong, OH, Hyun Woo, YUN, Jae Kwan, HAN, Mi Kyong, KIM, Ji Yeon, JEE, Deock Gu, PARK, Kwang Roh
• Granted Patent: US 9,258,392 B2
• Time in Patent Office: Days
• US Class Current: 455/556.1
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04M 1/0202 Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets constructional features of telephone transmitters or receivers, e.g. of speakers or microphones H04M1/03
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

Disclosed are a method and an apparatus for generating metadata of immersive media and disclosed also are an apparatus and a method for transmitting metadata related information. The apparatus includes: at least one of a camera module photographing or capturing the image; a gyro module sensing horizontality; a global positioning sensor (GPS) module calculating a position by receiving a satellite signal; and an audio module recording audio; and a network module receiving sensor effect information from a sensor aggregator through a wireless communication network; and an application generating metadata by performing timer-synchronization of an image photographed based on the camera module, a sensor effect collected by using the gyro module or the GPS module, or audio collected based on the audio module.

Claims

1. A terminal device generating metadata of an image, comprising:

at least one of a camera module photographing or capturing the image;

a gyro module sensing horizontality;

a global positioning sensor (GPS) module calculating a position by receiving a satellite signal; and

an audio module recording audio, and

a network module receiving sensor effect information from a sensor aggregator through a wireless communication network,

an application generating metadata by performing timer-synchronization of an image photographed based on the camera module, a sensor effect collected by using the gyro module or the GPS module, or audio collected based on the audio module.

2. The terminal device of claim 1, wherein the application records the metadata in a file and closes the media file in which the metadata is being recorded when receiving an end event.

3. The terminal device of claim 2, wherein if the media file is a file recorded based on Moving Picture Experts Group4 (MPEG4), when the media file is closed, the metadata is used as a track of the media file.

4. The terminal device of claim 1, wherein when the application receives an AV event meaning recording of audio video (AV) data, the photographed image and the collected audio are combined to be recorded in an AV media file.

5. The terminal device of claim 4, wherein the AV media file is recorded as a name of a date-time-sequence number format, and if there is not AV media file which was prepared previously, the AV media file is newly generated or if the AV media file which is opened in advance, a content is recorded to be added.

6. The terminal device of claim 1, wherein the application allows the metadata to further include the sensor effect information received through the network module when receiving a sensor event meaning receiving of sensor data.

7. The terminal device of claim 1, wherein the application allows the metadata to further include the sensor effect collected from the gyro module or the GPS module when receiving a local sensor event meaning collecting of the sensor effect in the terminal device.

8. The terminal device of claim 1, wherein the application performs connection setting with the sensor aggregator based on an Internet protocol (IP) address of the sensor aggregator.

9. The terminal device of claim 8, wherein the application performs the timer synchronization by transmitting a timer synchronization message to the sensor aggregator through the network module.

10. A method for generating metadata of an image in a terminal device, comprising the steps of:

receiving sensor effect information from a sensor aggregator through a wireless communication network; and

generating the metadata by timer-synchronizing an image photographed based on a camera module, a sensor effect collected by using a gyro module or a GPS module, or audio recorded based on an audio module,

wherein when an end event is received, the metadata is recorded as a media file and the recording media file is closed,

when an AV event meaning recording of audio video (AV) data is received, the photographed image and the recorded audio are combined to be recorded in the media file,

when a sensor event meaning receiving of sensor data is received, the metadata further includes the sensor effect information received through the network module, and

when a local sensor event meaning collecting of a sensor effect is received by the terminal device, the metadata further includes the sensor effect collected by the gyro module or the GPS module.

11. A sensor aggregator transmitting information for generating metadata of an image, comprising:

at least one of a CO₂ sensor, a light sensor generating an illumination effect through the brightness of light, a humidity sensor generating a waterjet effect by detecting humidity, a temperature sensor generating a thermal effect by measuring a temperature, and a wind sensor generating a wind effect by measuring a wind strength;

a sensor aggregating unit aggregating or analyzing information collected through at least one of the CO₂ sensor, the light sensor, the humidity sensor, the temperature sensor, and the wind sensor; and

a network module transmitting the sensor effect information to a terminal device through a wireless communication network.

12. The sensor aggregator of claim 11, wherein the sensor aggregating unit closes the CO₂ sensor, the light sensor, the humidity sensor, the temperature sensor, and the wind sensor, closes the network module, and ends an application program when receiving an end event.

13. The sensor aggregator of claim 11, wherein the sensor aggregating unit performs connection setting with the terminal device, registers an Internet protocol (IP) address of the terminal device in a connection setting list, and performs timer synchronization with the terminal device, when receiving a connection request event of the terminal device.

14. The sensor aggregator of claim 11, wherein the sensor aggregating unit releases the connection setting with the terminal device and deletes the IP address of the terminal device written in the connection setting list, when receiving a connection release event of the terminal device.

15. The sensor aggregator of claim 11, wherein the sensor aggregating unit collects the sensor effect information from the CO₂ sensor, the light sensor, the humidity sensor, the temperature sensor, or the wind sensor, retrieves IP addresses from the connection setting list, and transmits the sensor effect information to the terminal device corresponding to the IP address, when receiving a sensor event meaning transmitting of the sensor effect information.

16. A method for transmitting information related to metadata in a sensor aggregator, comprising the steps of:

collecting a sensor effect from a CO₂ sensor detecting CO₂ gas, a light sensor generating an illumination effect by adjusting the brightness of light, a humidity sensor generating a waterjet effect by detecting humidity, a temperature sensor generating a thermal effect by measuring a temperature, or a wind sensor generating an wind effect by measuring a wind strength;

generating sensor effect information by aggregating or analyzing the collected sensor effect; and

transmitting the sensor effect information to a terminal device through a wireless communication network.

17. The method of claim 16, further comprising the step of closing the CO2 sensor, the light sensor, the humidity sensor, the temperature sensor, and the wind sensor, closing the network module, and ending an application program when the end event is input.

18. The method of claim 16, further comprising the step of performing connection setting with the terminal device, registering an Internet protocol (IP) address of the terminal device in a connection setting list, and performing timer synchronization with the terminal device, when a connection request event of the terminal device is input.

19. The method of claim 16, further comprising the step of releasing the connection setting with the terminal device and deleting the IP address of the terminal device written in the connection setting list, when a connection release event of the terminal device is input.

20. The method of claim 16, further comprising the step of collecting the sensor effect information from the CO₂ sensor, the light sensor, the humidity sensor, the temperature sensor, or the wind sensor, retrieving IP addresses from the connection setting list, and transmitting the sensor effect information to a terminal device corresponding to the IP address, when a sensor event meaning transmitting of the sensor effect information is input.