Interactive multimedia applications device

• US 8,196,132 B2
• Filed: 04/26/2005
• Issued: 06/05/2012
• Est. Priority Date: 04/26/2005
• Status: Active Grant

• Current Assignee: Alterface S.A.
• Original Assignee: Alterface S.A.
• Inventors: Ergo, David, Marichal, Xavier, Wielemans, Xavier
• Primary Examiner(s): KENDALL, CHUCK O
• Application Number:
  US11/919,380
• Publication Number:
  US 20090113389A1
• Time in Patent Office: 2,597 Days
• Field of Search: None
• US Class Current: 717/172
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

An interactive multimedia applications device and method for an interactive multimedia application comprises one or more live media capture devices providing a media stream, an engine comprising a real time media processing module for processing said media stream, and rendering means connected to multimedia output devices. In addition, said device comprises (i) a virtual scenario description repository adapted for storing a plurality of scenarios expressed in a scenario programming language; (ii) a memory module adapted for storing an internal representation of one of said scenarios, and an internal representation of a virtual scene and (iii) a parser/loader for parsing a selected one of said plurality of scenarios, and loading it in said memory module.

Claims

1. An interactive multimedia applications device for an interactive multimedia application, comprising

one or more live media capture devices providing a media stream;

a virtual scenario description repository adapted for storing a plurality of scenarios expressed in a scenario programming language;

an engine comprising a real-time media processing module for processing said media stream, said real-time media processing module comprising means for extracting a descriptor stream from said media stream and for attaching said descriptor stream to said media stream, said engine further comprising triggers evaluation means adapted for receiving said descriptor stream and an internal representation of triggering rules, and adapted for updating said virtual scene in response to said descriptor stream and said triggering rules;

a memory module adapted for storing an internal representation of one of said scenarios, and an internal representation of a virtual scene;

a parser/loader for parsing a selected one of said plurality of scenarios, and loading it in said memory module, said parser/loader being adapted for parsing triggering rules from said virtual scenario repository and for loading said internal representation of said triggering rules into said memory module;

rendering means connected to multimedia output devices and adapted for updating said internal representation of a virtual scene in memory module according to said media stream as processed in real-time by said processing module, and adapted for using said updated internal representation for determining the appearance to be sent to said connected multimedia output devices.

2. A device according to claim 1, wherein said real-time media processing module is comprised of a plurality of plug-in modules, a plug-in module in the plurality performing at least one of

a specific processing of said media stream;

and

extraction of a descriptor stream from at least one of said media stream and said descriptor stream.

3. A device according to claim 2, wherein said plurality of plug-in modules comprises one or more plug-in modules for processing a descriptor provided by a predecessor plug-in module.

4. A device according to claim 2, wherein said plurality of plug-in modules comprises one or more splitter plug-in modules having:

at least one of a single input media stream and descriptor stream; and

a plurality of output media and descriptor streams.

5. A device according to claim 2, wherein said plurality of plug-in modules comprises one or more merger plug-in modules having:

at least one of a plurality of input media and descriptor streams; and

a single output media and descriptor stream.

6. A device according to claim 2, wherein said plurality of plug-in modules comprises:

parallel chains of plug-in modules working in dependence of settings; and

means for linking the settings of corresponding plug-ins in said parallel chains.

7. A device according to claim 2, wherein said plurality of plug-in modules comprises:

parallel chains of plug-in modules; and

means for synchronizing at least one of the media and descriptor streams in said parallel chains.

8. A device according to claim 2, wherein said trigger evaluation means is adapted for at least one of:

dynamically loading a plug-in module into said chain of plug-in modules; and

dynamically creating a new chain of plug-in modules in response to said descriptor stream and said triggering rules.

9. A device according to claim 2, wherein:

said scenario programming language is adapted for defining a descriptor grammar; and

said plurality of plug-in modules comprises plug-in modules adapted for extracting descriptors pursuant to said descriptor grammar, the engine being independent of said descriptor grammar.

10. A device according to claim 9, characterized in that said engine comprises means adapted for verifying that said plug-in modules adapted for extracting said descriptors produce descriptors according to said descriptor grammar.

11. A device according to claim 2, wherein at least one of the descriptors and the descriptor grammar are processed in binary form by the engine and plug-in-modules.

12. A method for creating an interactive multimedia application for an interactive multimedia applications device comprising one or more live media capture devices providing a media stream, an engine comprising a real-time media processing module for processing said media stream, and rendering means connected to multimedia output devices, the method comprising steps of

providing a virtual scenario description repository adapted for storing a plurality of scenarios expressed in a scenario programming language;

providing a memory module adapted for storing an internal representation of one of said scenarios, and an internal representation of a virtual scene;

providing a parser/loader for parsing a selected one of said plurality of scenarios, and loading it in said memory module;

updating said internal representation of a virtual scene in memory module according to said media stream as processed in real-time by said processing module;

using said updated internal representation for determining the appearance to be sent to said connected multimedia output devices;

parsing triggering rules from said virtual scenario repository and loading an internal representation of said triggering rules into said memory module;

extracting a descriptor stream from said media stream, and attaching said descriptor stream to said media stream; and

receiving said descriptor stream, said internal representation of triggering rules, and updating said virtual scene in response to said descriptor stream and said triggering rules.

13. A method according to claim 12, further comprising a step of providing a plurality of plug-in modules, a plug-in module in the plurality performing at least one of

a specific processing of said media stream;

and

the extraction of a descriptor stream from at least one of said media stream and the descriptor stream.

14. A method according to claim 13, wherein a plug-in module in said plurality of plug-in modules processes a descriptor provided by a predecessor plug-in module.

15. A method according to claim 13, wherein a plug-in module in said plurality of plug-in modules performs a step of splitting at least one of a single input media and the descriptor stream into at least one of a plurality of output media and descriptor streams.

16. A method according to claim 13, wherein a plug-in module in said plurality of plug-in modules performs a step of merging at least one of a plurality of input media and descriptor streams into a single output media and descriptor stream.

17. A method according to claim 13, further comprising steps of

providing parallel chains of plug-in modules working in dependence of settings, and

linking the settings of corresponding plug-ins in said parallel chains.

18. A method according to claim 13, further comprising steps of

providing parallel chains of plug-in modules, and

synchronizing the media and descriptor streams in said parallel chains.

19. A method according to claim 13, further comprising at least one of the steps of

dynamically loading a plug-in module into said chain of plug-in modules;

and

dynamically creating a new chain of plug-in modules in response to at least one of said descriptor stream and said triggering rules.

20. A method according to claim 13, further comprising steps of:

defining a descriptor grammar, and

providing one or more plug-in modules adapted for extracting descriptors pursuant to said descriptor grammar, the engine being independent of said descriptor grammar.

22. A method according to claim 13, wherein the engine and plug-in-modules perform the step of processing descriptors and descriptor grammar in, binary.

23. At least one computer-readable medium having stored thereon a computer program able to control a processor to perform the method of claim 12.

21. A method according to claim to 20, wherein said engine performs the step of verifying that said plug-in modules adapted for extracting said descriptors produce descriptors according to said descriptor grammar.