Apparatus and methods for user generated content indexing

• US 9,633,015 B2
• Filed: 07/26/2012
• Issued: 04/25/2017
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Telefonaktiebolaget LM Ericsson
• Original Assignee: Telefonaktiebolaget LM Ericsson
• Inventors: Arngren, Tommy, Lindegren, David, Sderberg, Joakim, Stlnacke, Marika
• Primary Examiner(s): Alam, Shahid
• Assistant Examiner(s): Sultana, Nargis
• Application Number:
  US13/559,258
• Publication Number:
  US 20140032562A1
• Time in Patent Office: 1,734 Days
• Field of Search: 707741, 707913
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures

Abstract

A method and client device is disclosed for indexing content of a multimedia file. The method comprises using a client device to segment the content of the multimedia file into a plurality of segments and to determine structure-searchable data for each segment. Determining structure searchable data for a segment comprises (1) identifying one or more features of respective multimedia types in the segment; (2) correlating each of the identified features to one or more respective keywords; and (3) calculating one or more respective relevance factors for each of the keywords, where at least one of the relevance factors is based on one or more characteristics of the client device. The method also comprises the client device transmitting the structure-searchable data (including the keywords, relevance factors, and respective media types of the identified features) to an indexing server.

Claims

1. A method for indexing content of a multimedia file, the method comprising:

segmenting the content of the multimedia file, by a client device, into a plurality of segments;

determining, by the client device, structure-searchable data for each segment, wherein the determining comprises;

identifying one or more features of respective multimedia types in the segment;

correlating each of the identified features to one or more respective keywords; and

calculating one or more respective relevance factors for each of the keywords, wherein at least one of the relevance factors is based on one or more characteristics of the client device; and

transmitting, by the client device, the structure-searchable data to an indexing server, wherein the structure-searchable data includes the keywords, and further includes, for each included keyword, a corresponding relevance value and information identifying at least one media type of a feature with which the keyword is correlated, wherein

for at least one keyword included in the structure-searchable data, the structure-searchable data comprises first media type information indicating that the keyword is associated with a first media type and second media type information indicating that the keyword is also associated with a second media type that is different than the first media type.

2. The method of claim 1, wherein the calculating of the at least one of the relevance factors is based on (a) content of one or more other multimedia files stored on the client device;

(b) a type of the client device; and

(c) a user-specified relevance of the client device.

3. The method of claim 1, wherein the one or more relevance factors comprise a degree of confidence that a given keyword describes the one or more features that were correlated with that keyword.

4. The method of claim 1, wherein transmitting the structure-searchable data to the indexing server further comprises transmitting data usable by the indexing server to identify a group of devices to which the client device has been registered.

5. The method of claim 1, further comprising:

transmitting the multimedia file in response to receiving a request from the indexing server.

21. The method of claim 1, wherein the one or more characteristics of the client device comprise a type of the client device.

22. The method of claim 1, wherein the one or more characteristics of the client device comprise a user-specified relevance of the client device.

23. The method of claim 1, wherein the one or more relevance factors include a prominence score indicating how prominently one of the one or more features is presented in the segment.

24. The method of claim 1, wherein the one or more relevance factors include an indication of whether the keyword is associated with an image, text, voice, or sound.

25. The method of claim 1, wherein calculating the one or more respective relevance factors for each of the keywords comprises using a first weight if the device type is a first type of client device and using a second weight that is higher than the first weight if the device type is a second type of client device.

26. The method of claim 2, wherein calculating the one or more respective relevance factors for each of the keywords comprises determining a weight to apply to the client device based on how frequently the keyword is associated with other content on the client device.

32. The method of claim 1, wherein transmitting the structure-searchable data (SSD) comprises transmitting a set of records comprising the SSD, wherein at least one of the records included in the set of records comprises:

a file identifier identifying the multimedia file, a device identifier identifying the client device, a segment identifier identifying one of the plurality of segments, a keyword, media type information identifying each media type with which the keyword is associated, and a keyword score.

33. The method of claim 1, wherein

transmitting the structure-searchable data (SSD) comprises transmitting a set of records comprising the SSD,

at least one of the records included in the set of records comprises:

1) a segment identifier identifying one of the plurality of segments,

     2) a keyword, media type information identifying each media type with which the keyword is associated, and

     3) a plurality of relevance factors for use in determining a final relevance score for the keyword included in the record, the plurality of relevance factors including a first relevance factor, a second relevance factor, and a third relevance factor, and

the method further comprising using said first, second and third relevance factors to determine the final relevance score.

34. The method of claim 33, wherein the first relevance factor is a prominence score, the second relevance factor is a confidence interval that indicates a confidence with which the keyword in the record is associated with one or more features included in the segment identified in the record, and a device-specific weight.

6. A client device for indexing content of a multimedia file, the client device comprising:

a computer processor;

a memory coupled to the processor and storing program instructions executable by the processor to implement;

segmenting the content of the multimedia file into a plurality of segments;

determining structure-searchable data for each segment, wherein the determining comprises;

identifying one or more features of respective multimedia types in the segment;

correlating each of the identified features to one or more respective keywords; and

calculating one or more respective relevance factors for each of the keywords, wherein at least one of the relevance factors is based on one or more characteristics of the client device; and

transmitting the structure-searchable data to an indexing server, wherein the structure-searchable data includes the keywords, and further includes, for each included keyword, a corresponding relevance value and information identifying at least one media type of a feature with which the keyword is correlated, wherein

for at least one keyword included in the structure-searchable data, the structure-searchable data comprises first media type information indicating that the keyword is associated with a first media type and second media type information indicating that the keyword is also associated with a second media type that is different than the first media type.

7. The client device of claim 6, wherein the calculating of the at least one of the relevance factors is based on:

(a) content of one or more other multimedia files stored on the client device;

(b) a type of the client device; and

(c) a user-specified relevance of the client device.

8. The client device of claim 6, wherein the one or more relevance factors comprise a degree of confidence that a given keyword describes the one or more features that were correlated with that keyword.

9. The client device of claim 6, wherein transmitting the structure-searchable data to the indexing server further comprises transmitting data usable by the indexing server to identify a group of devices to which the client device has been registered.

10. The client device of claim 6, wherein the program instructions are further executable to implement:

transmitting the multimedia file in response to receiving a request from the indexing server.

11. A method for creating a content index of multimedia content, the method comprising:

receiving, by an indexing server, structure-searchable data from a plurality of remote client devices, wherein the structure-searchable data describes content of multimedia files stored on the remote client devices; and

for each of the client devices, the indexing server performing;

identifying a respective client device group to which the client device has been registered, wherein the identified client device group is associated with a content index for the client device group, and

updating the content index for the client device group to which the client device has been registered using the structure-searchable data received from the client device, wherein the updating is dependent on a weighting scheme specific to the client device group.

12. The method of claim 11, wherein at least one of a plurality of entries of the content index identifies a file having different multimedia-types, the method further comprising calculating a relevance factor the at least one entry based on respective different weights associated with the different multimedia-types.

13. The method of claim 11, wherein the updating is further dependent on one or more relevance factors received in the structure-searchable data, wherein at least one of the relevance factors is based on:

(1) one or more characteristics of the client device, or (2) a measure of confidence that a given keyword describes one or more features correlated with that keyword in the structure searchable data.

14. The method of claim 13, wherein the one or more characteristics of the client device that the updating of the content index is dependent upon corresponds to at least one of:

(a) content of one or more other multimedia files stored on the client device;

(b) a type of the client device; and

(c) a user-specified weight for the client device.

15. The method of claim 11, further comprising:

determining, by the indexing server in response to receiving a content query from one of the remote client devices, that the remote client device is a member of a client device group;

determining, by the indexing server, a content index corresponding to the client device group; and

using, by the indexing server, the determined content index to identify one or more portions of the content responsive to the content query.

27. The method of claim 12, wherein updating the content index comprises calculating a relevance score for a multimedia segment stored on one of the remote client devices as a sum of two or more weights applied to the multimedia segment.

28. The method of claim 12, wherein the structure-searchable data identifies a keyword associated with a segment of the multimedia file and identifies the segment as having a first media type and a second media type, the method further comprising calculating a relevance factor for the segment by adding a first weight assigned to the first media type and a second weight assigned to the second media type.

29. The method of claim 28, wherein the determined relevance factor is an initial relevance factor, and wherein updating the index comprises determining an updated relevance factor based on the formula (initialScore+Prom)*WRel*WFact*WQual, wherein the initialScore is the initial relevance score, Prom is a prominence score indicating a degree of prominence in which features associated with the keyword is present in the segment, WRel is a confidence interval that indicates a confidence with which the keyword is associated with features of the segment, WFact is a device-specific weight that indicates how much the keyword is represented in the segment, and WQual is a quality factor indicating media quality of the segment.

30. The method of claim 11, wherein at least one of the updated content indices comprises:

a first record linking a first key word with a first segment of a multimedia file and with a first score; and

a second record that links said first key word with a second segment of the multimedia file and with second score, wherein

the second segment of the media file is different than the first segment.

31. The method of claim 11, wherein the weighting scheme specific to the client device group identifies a first weight for a first media type and a second weight for a second media type that is different than the first media type.

16. An indexing server for indexing content of a multimedia file, the indexing server comprising:

a computer processor; and

a memory coupled to the processor and storing program instructions executable by the processor to implement;

receiving structure-searchable data from a plurality of client devices, wherein the structure-searchable data describes content of multimedia files stored on the client devices; and

for each of the client devices;

identifying a respective client device group to which the client device has been registered, and

updating a content index of the client device group using the structure-searchable data received from the client device, wherein the updating is dependent on a weighting scheme specific to the client device group.

17. The indexing server of claim 16, wherein at least one of a plurality of entries of the index identifies a file having different multimedia types, the memory and the processor further configured to calculate a relevance factor for the at least one entry based on respective different weights associated with the different multimedia-types.

18. The indexing server of claim 16, wherein the updating is further dependent on one or more relevance factors received in the structure-searchable data, wherein at least one of the relevance factors is based on:

(1) one or more characteristics of the client device, or (2) a measure of confidence that a given keyword describes one or more features correlated with that keyword in the structure searchable data.

19. The indexing server of claim 18, wherein the one or more characteristics of the client device that the updating of the client index is dependent upon corresponds to at least one of:

(a) content of one or more other multimedia files stored on the client device;

(b) a type of the client device; and

(c) a user-specified weight for the client device.

20. The indexing server of claim 16, wherein the program instructions are further executable to implement:

determining, in response to receiving a content query from one of the client devices, that the client device is a member of a client device group;

determining a content index corresponding to the client device group; and

using the determined content index to identify one or more portions of the content responsive to the query.