System and method for providing immersive surround environment for enhanced content experience

• US 9,473,813 B2
• Filed: 03/04/2010
• Issued: 10/18/2016
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Infosys Limited
• Original Assignee: Infosys Limited
• Inventors: Darbari, Akshay, Gupta, Puneet
• Primary Examiner(s): Mei, Xu
• Application Number:
  US12/717,168
• Publication Number:
  US 20110160882A1
• Time in Patent Office: 2,420 Days
• Field of Search: 700/94, 375/240.01, 472/65, 472/74, 472/64, 381/22, 381/23, 381/57, 381/307
• US Class Current: 1/1
• CPC Class Codes: G06F 16/60 of audio data
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: H03G 3/32 the control being dependent upon ambient noise level or sound level
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/436 Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59

Abstract

A method and system for providing an immersive surround environment for enhanced content experience is provided. The method includes receiving an encoded media stream and processing the stream to obtain decoded media data. The received media stream can also be captured real-time media. The method further includes extracting one or more environmental parameters from the stream. The environmental parameters are then embedded or encoded. Thereafter, the method includes providing the one or more environmental parameters to one or more devices that control environmental conditions to produce environmental effects.

Claims

1. A system for providing an immersive surround environment for enhanced content experience, the system comprising:

a decoder configured to receive an encoded media stream, the encoded media stream comprising different types of media data, environmental parameters associated with the different types of media data, frame synchronization information and stream elements information, and further configured to decompress and decode the media stream;

a content analyzer configured to analyze the decoded media stream and further configured to extract the environmental parameters corresponding to the different types of media data in the decoded media stream;

a stream and environmental parameter synchronization module configured to identify one or more additional environmental parameters based on the extracted environmental parameters and further configured to simulate the identified additional environmental parameters for the different types of media data;

an environmental effect encoder configured to encode the extracted environmental parameters and the simulated environmental parameters corresponding to the different types of media data and further configured to synchronize the environmental parameters with the frame synchronization information and the stream elements information corresponding to the encoded media stream to generate encoded environmental parameters; and

an encoded stream playback system configured to decode the encoded media stream and the encoded environmental parameters, wherein the encoded stream playback system is further configured to provide media output to an audio/visual equipment for playback and to provide the decoded environmental parameters to one or more output devices for simulating environmental effects in conjunction with the media output.

2. The system of claim 1, wherein the encoded stream playback system comprises:

a standard player/decoder configured to decode the encoded stream and provide the media output to the audio/visual equipment; and

an environmental effect playback system configured to provide the decoded environmental parameters to the one or more output devices for simulating the environmental effects.

3. The system of claim 1, wherein the encoded media stream is encoded in accordance with standard file formats, the standard file formats comprise:

MPEG-2, MPEG-4, AVS, VC-1, Dirac, H.264, AAC, AC3, G.711, G.723, MP3, JPEG, JPEG 2000, BMP, GIF, TIFF, WAV, MPEG, AC3, AAC, WMA, AVI, 3GP, WMV, DivX, AVCHD and PNG.

4. The system of claim 1, wherein the environmental parameters are at least one of temperature, ambient light, humidity and wind velocity.

5. The system of claim 4, wherein the environmental parameters are extracted from color layout, structure, shape, region, texture, location, face, motion of image/video stream and/or pitch, harmony, tempo, rhythm, loudness, lyrics analyzer for audio speech stream and all for audio-video stream.

6. The system of claim 1, wherein the different types of media data comprise an image stream, an audio stream, a speech stream, a video stream and an audio-video stream.

7. The system of claim 1, wherein the encoded media stream includes media stream captured at real-time.

8. A method for providing an immersive surround environment for enhanced content experience, the method comprising:

receiving an encoded media stream, the encoded media stream comprising different types of media data, environmental parameters associated with the different types of media data, frame synchronization information, stream elements information, and processing the encoded media stream to obtain the decoded media data;

extracting the one or more environmental parameters corresponding to the different types of media data from the decoded media data;

identifying one or more additional environmental parameters based on the extracted environmental parameters;

simulating the identified additional environmental parameters for the different types of media data;

encoding the extracted one or more environmental parameters and the simulated at least one of the environmental parameters;

synchronizing the encoded one or more environmental parameters with the frame synchronization information and the stream elements information corresponding to the encoded media stream to output an encoded environmental parameter stream;

decoding the encoded media stream to obtain media output;

decoding the encoded environmental parameter stream to obtain one or more environmental parameters; and

providing the one or more environmental parameters to one or more devices that control environmental conditions to simulate environmental effects in conjunction with the media output.

9. The method of claim 8, wherein the step of encoding the one or more environmental parameters comprises encoding the one or more environmental parameters at various levels within the encoded stream.

10. The method of claim 9, wherein the various levels comprise at least one of:

stream, group of pictures, frame, slice, macroblock and block.

11. The method of claim 8, wherein the step of encoding the one or more environmental parameters to output the encoded environmental parameter stream comprises the steps of:

replicating encoded media stream structure by retaining header information of the encoded media stream; and

mapping the one or more environmental parameters to one or more specific identifiers of the encoded media stream.

12. The method of claim 8, further comprising:

receiving an environmental parameter stream comprising the one or more environmental parameters;

replicating an encoded media stream by retaining header information of the encoded media stream;

incorporating placeholders corresponding to at least one of image streams, audio/speech streams, audio streams and audio-video file formats;

mapping the one or more environmental parameters to one or more specific identifiers of the replicated encoded media stream.

13. The method of claim 8, wherein the encoded media stream includes media stream captured at real-time.