Method, system and module for mult-modal data fusion
First Claim
1. A method for multi-modal data fusion in a multi-modal system comprising a plurality of input modalities, the method comprising the steps of:
- (i) Receiving one or more segments of multi-modal data, each of said segments being associated respectively with said modalities;
(ii) Initiating a dynamically variable wait period after one of said segments is received, said dynamically variable wait period having a duration determined from data fusion timing statistics of the system;
(iii) Waiting for reception of any further said segments during said dynamically variable wait period; and
(v) Fusing said segments received during said steps (i) to (iii) to provide fused data.
4 Assignments
0 Petitions
Accused Products
Abstract
A method for multi-modal data fusion (100), a multi-modal system data fusion system (10) and module (24) that in use operates by receiving segments (125) of multi-modal data associated respectively with a modalitiy. Initiating (130) a dynamically variable wait period after one of the segments is received is then performed. The dynamically variable wait period has a duration determined from data fusion timing statistics of the system (10). A waiting (140) for reception of any further segments during the dynamically variable wait period is then effected and thereafter a fusing (145) of the segments received provides fused data that is sent (160) to a dialog manager (25).
-
Citations
21 Claims
-
1. A method for multi-modal data fusion in a multi-modal system comprising a plurality of input modalities, the method comprising the steps of:
-
(i) Receiving one or more segments of multi-modal data, each of said segments being associated respectively with said modalities;
(ii) Initiating a dynamically variable wait period after one of said segments is received, said dynamically variable wait period having a duration determined from data fusion timing statistics of the system;
(iii) Waiting for reception of any further said segments during said dynamically variable wait period; and
(v) Fusing said segments received during said steps (i) to (iii) to provide fused data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A multi-modal data fusion system comprising:
-
a plurality of modality processing modules;
a plurality of parsers coupled to a respective one of said modality processing modules;
a multi-modal fusion module having inputs coupled to outputs of said parsers, wherein in use said fusion module receives one or more segments of multi-modal data from at least one of said parsers, and initiates a dynamically variable wait period after one of said segments is received, said dynamically variable wait period having a duration determined from data fusion timing statistics of the system, the fusion module then waits for reception of any further said segments during said dynamically variable wait period and fuses said segments received to provide fused data. - View Dependent Claims (11, 12, 13, 14, 15, 16)
-
- 17. A multi-modal fusion module having inputs for coupling to outputs of parsers, wherein in use said fusion module receives one or more segments of multi-modal data from at least one of said parsers, and initiates a dynamically variable wait period after one of said segments is received, said dynamically variable wait period having a duration determined from data fusion timing statistics of the system, the fusion module then waits for reception of any further said segments during said dynamically variable wait period and fuses said segments received to provide fused data.
Specification