Method and device for organizing user provided information with meta-information

• US 7,797,331 B2
• Filed: 12/20/2002
• Issued: 09/14/2010
• Est. Priority Date: 12/20/2002
• Status: Expired due to Fees

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Weingart, Peter, Luck, Matthias, Haberland-Schlosser, Knut, Theimer, Wolfgang, Serafat, Reza, Myka, Andreas, Rateitschek, Klaus, Makela, Jakke, Gortz, Udo
• Primary Examiner(s): Jalil; Neveen Abel
• Assistant Examiner(s): Hoang; Son T
• Application Number:
  US10/540,440
• Publication Number:
  US 20060217990A1
• Time in Patent Office: 2,825 Days
• Field of Search: None
• US Class Current: 707/769
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04M 1/2746 Sorting, e.g. according to history or frequency of use
• CPC Class Codes: H04M 1/56 Arrangements for indicating or recording the called number at the calling subscriber's set
• CPC Class Codes: H04M 1/57 Arrangements for indicating or recording the number of the calling subscriber at the called subscriber's set at the operator set in a manual exchange H04M5/20
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72433 for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 2250/60 logging of communication history, e.g. outgoing or incoming calls, missed calls, messages or URLs logging of telephone numbers H04M1/56, H04M1/57

Abstract

The present invention is directed to a method for organizing user provided information in a mobile terminal device with meta-information to allow retrieval of the user provided information, and a device configured to perform the method. The user provided information is obtained by any user operation against the mobile terminal device and context information is obtained associated with the User provided information. The meta-information is obtained from the context information and assigned to the user provided information. The meta-information is employed for retrieval of the User provided information by matching request information with the meta-information Further, the method for organizing user provided information employs calendar information for obtaining meta-information. Further, the method for organizing user provided information is adapted to organize user provided audio information. Further, the method for organizing user provided information is adapted to organize User provided audio information by embedding meta-information into the User provided audio information.

Claims

1. A method, comprising:

obtaining user provided information in consequence to any user operation on a mobile terminal device;

obtaining context information associated with said user provided information, wherein said context information is related to at least one current condition of the mobile terminal device at the time of said any user operation and includes calendar information from an electronic calendar implemented at said mobile terminal device;

obtaining a first time information in accordance with said user provided information;

obtaining one or more calendar entries included in said calendar information, wherein each calendar entry comprises a second time information with a start time and an end time;

comparing said first time information with each of said second time information to identify matching calendar entries by assigning a membership function to each of said second time information and deriving a membership grade value from each of said membership functions in accordance with said first period of time, wherein each membership function comprises a function in time, which rises from zero value at a predetermined moment in time before said start time of a respective calendar entry and becomes zero value at a predefined moment in time after said end time of said respective calendar entry, wherein each membership function defines an extended timeframe for a respective calendar entry;

obtaining meta-information from each matching calendar entry of said one or more obtained calendar entries;

assigning meta-information obtained from said context information and said membership grade value to said user provided information, said membership grade value defining a measure which allows estimation of a reliability of a timely relatedness for retrieval; and

directing storage of said user provided information and said meta-information in a history storage in order to establish an information history functionality;

wherein said meta-information is employable for retrieval of said user provided information by matching request information of a retrieval request with said meta-information for selecting a user provided information assigned to said meta-information matched to said request information.

2. Method according to claim 1, wherein said assigning meta-information comprises:

extracting said meta-information from said context information.

3. Method according to claim 1, wherein said obtaining of said user provided information comprises:

receiving user input information being generated by user operation of any input means of said mobile terminal device and/or

receiving transaction information and/or communication information being received via any communication interface of said mobile terminal device,

wherein said user input information and/or said transaction information and/or said communication information represent said user provided information.

4. Method according to claim 1, wherein said assigning of said meta-information to said user provided information comprises:

extracting labeling information and/or indexing information from said context information;

assigning said labeling information and/or indexing information to said user provided information; and

directing storage of said labeling information and/or indexing information and said user provided information;

wherein said labeling information and/or indexing information is employed as said meta-information for establishing retrieval functionality of said user input history.

5. Method according to claim 1, wherein associating said context information with said user provided information comprises:

generating referencing information for at least a part of said user provided information;

directing storage of said referencing information in a first storage area; and

directing storage of said part of said user provided information in a second storage area;

wherein said referencing information comprises address information which allow to retrieve said part of user provided information from said second storage area.

6. Method according to claim 1, comprising:

partitioning said user provided information into a plurality of information parts, each information part relating to a maximum pre-defined period of time, each information part being matched separately.

7. Method according to claim 1, in case said first period of time exceeds said second period of time:

sectioning said user provided information into at least two information sections, one of said at least two information sections fitting with said second period of time, said one fitting information section being matched.

8. Method according to claim 1, wherein each of said plurality of calendar entries being included in said calendar information is semantically structured and said obtaining of meta-information from each matching calendar entry comprises:

obtaining of meta-information from each of said matching semantically structured calendar entry.

9. Method according to claim 1, further comprising:

obtaining said user provided information including user provided audio information and in parallel additional user provided information, said user provided audio information being recorded and stored;

obtaining said context information in parallel to said user provided audio information, said context information comprising user input information generated in consequence to user action against said mobile terminal device; and

generating meta-information comprising information about said additional user provided information and said context information.

10. Method according to claim 9, wherein said user input information comprises control signals related to said recording of said user provided audio information, and said method further comprises:

on receiving a start signal;

initiating said recording and directing storage of said user provided audio information;

on receiving a keyword signal;

initiating a recording of a user provided audio keyword information including keywords relating to said user provided audio information;

on receiving an attach signal;

associating additional information with said user provided audio information;

on receiving a pause signal;

pausing said recording and storing of said user provided audio information;

on receiving a continue signal;

resuming said recording and storing of said user provided audio information subsequently to said pausing of said recording and storing; and

on receiving a stop signal;

stopping said recording and storage of said user provided audio information and generating said recording context information.

11. Method according to claim 9, wherein said recording context information comprises:

information and time information about said user provided audio information;

time information about said user input information; and

information about said additional user provided information being associated with the user provided audio information.

12. Method according to claim 9, wherein said generating of said recording context information comprises:

encoding said meta-information as a document being encoded in accordance with a markup language.

13. Method according to claim 9, wherein said meta-information is displayed by assigning graphical elements to each information entry included in said meta-information and predicting said graphical elements illustrating the content of the meta-information and showing associations defined in said meta-information.

14. Method according to claim 1, further comprising:

receiving a request for retrieving, said request comprising request information for instructing to retrieve certain user provided information;

comparing said request information with said meta-information being assigned to said user provided information which is provided by said information history functionality;

retrieving said user provided information being assigned to said meta-information which matches said request information;

generating a response comprising said retrieved user provided information; and

directing transmission of said response.

15. Method according to claim 14, wherein said retrieving of said user provided information comprises:

retrieving referencing information being associated with said user provided information to be retrieved, said referencing information comprising address information which addresses said part of user provided information being stored in a second storage area; and

retrieving said part of user provided information from said second storage area.

16. Method according to claim 14, wherein said retrieving of said user provided information comprises:

evaluating said user provided information being retrieved on the basis of said membership grade values obtained from calendar entries;

said membership grade value defining a measure which allows to estimate a reliability for retrieval.

17. Method according to claim 1, further comprising:

receiving a request to store at least a part of said user provided information, wherein said request comprises said part of said user provided information and referencing information;

directing storage of said part of said user provided information and said referencing information such that said part of said user provided information is retrievable in conjunction with said referencing information;

receiving a request for retrieving at least said part of said user provided information, wherein said request comprises referencing information;

retrieving said part of said user provided information in accordance with said referencing information; and

generating a response including said retrieved part of said user provided information.

18. A computer readable storage medium embedded with a computer program comprising programming code for carrying out the operations of claim 1.

19. The method according to claim 1, wherein said deriving said membership grade value further comprises at least one out of averaging said membership function over said first period of time, determining a maximum of said membership function over said first period of time, or determining a minimum of said membership function over said first period of time.

20. The method according to claim 1, wherein said membership grade value indicates said timely relatedness between said user provided information and one of said matching calendar entries.

21. A method, comprising:

obtaining user provided information in consequence to any user operation on a mobile terminal device, wherein said user provided information includes user provided audio information;

obtaining context information associated with said user provided information, wherein said context information is related to at least one current condition of the mobile terminal device at the time of said any user operation;

providing code basis representing a plurality of coding symbols, said code basis comprising a pre-defined number of pre-defined frequencies, wherein a plurality of coding symbols represents a character and symbol code table employable for coding said meta-information, wherein said code basis is defined within a first frequency range, which is one frequency range of a plurality of frequency ranges forming a total frequency range being applicable to said user provided audio information;

repeating said code basis within at least one further frequency range out of said plurality of frequency ranges;

coding said meta-information in accordance with said code basis defined within said first frequency range and repeated within said at least one further frequency range to obtain redundancy;

combining said user provided audio information and said coded meta-information by embedding said coded meta-information into said user provided audio information; and

directing storage of said user provided information with said coded meta-information in a history storage in order to establish an information history functionality;

wherein said meta-information is employable for retrieval of said user provided information by matching request information of a retrieval request with said meta-information for selecting a user provided information assigned to said meta-information matched to said request information.

22. Method according to claim 21, wherein said combining comprises:

obtaining a modulation signal from said coded meta-information;

obtaining a modulated signal by combining said user provided audio information and said modulation signal in a frequency domain; and

obtaining said user provided audio information having embedded said meta-information by combining said modulated signal with said user provided audio information in a time domain.

23. Method according to claim 22, wherein said obtaining of said modulated signal comprises:

obtaining said modulated signal by multiplying said user provided audio information and said modulation signal in said frequency domain.

24. Method according to claim 22, wherein said obtaining of said user provided audio information having embedded said meta-information comprises:

obtaining said user provided audio information having embedded said meta-information by adding said modulation signal and said user provided audio information in said time domain.

25. Method according to claim 21, further comprising:

providing a correlation basis comprising each frequency being included in a set of code bases representing a plurality of coding symbols, said coding symbols being employed for coding said meta-information; and

applying said correlation basis onto said user provided audio information having embedded said meta-information to extract said meta-information;

wherein said extracted meta-information being available for retrieval.

26. Method according to claim 25, wherein said applying comprises:

obtaining a modulation signal from said correlation basis;

obtaining a correlation signal from said user provided audio information by convoluting said modulation signal and said user provided audio information having embedded said meta-information in a frequency domain;

extracting magnitude signal values from said correlation signal, said magnitude signal values corresponding to each code basis of said set of code bases; and

evaluating said magnitude signal values to retrieve said meta-information from said user provided audio information.

27. Method according to claim 26, wherein said evaluating of said magnitude signals comprises:

obtaining magnitude ratio values of each pair of said magnitude signal values;

normalizing said magnitude ratio values; and

evaluating said normalized magnitude ratio values of said magnitude signal values to extract said meta-information from said user provided audio information.

28. Method according to claim 26, wherein said correlation basis being defined in a first frequency range and said obtaining of said correlation signal comprises:

providing a correlation basis comprising all frequencies of said set of code bases within a first frequency range of a plurality of frequency ranges;

mapping said correlation basis into each frequency range of a plurality of frequency ranges, said plurality of frequency ranges forming a total frequency range being applicable to said user provided audio information to obtain said modulation signal; and

obtaining said correlation signal from said user provided audio information by convoluting said modulation signal and said user provided audio information in said frequency domain.

29. A computer readable storage medium embedded with a computer program comprising programming code for carrying out the operations of claim 21.

30. An apparatus comprising at least one processor and at least one memory storing computer program code, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to at least:

obtain user provided information and context information associated with said user provided information, in consequence to any user operation against said apparatus, wherein said context information is related to at least one current condition of the apparatus at the time of said any user operation and includes calendar information from an electronic calendar implemented at said apparatus;

obtain a first time information in accordance with said user provided information;

obtain one or more calendar entries included in said calendar information, wherein each calendar entry comprises a second time information with a start time and an end time;

compare said first time information with each of said second time information to identify matching calendar entries by assigning a membership function to each of said second time information and deriving a membership grade value from said membership function in accordance with said first time information, wherein each membership function comprises a function in time, which rises from zero value at a predetermined moment in time before said start time of a respective calendar entry and becomes zero value at a predefined moment in time after said end time of said respective calendar entry;

obtain meta-information from each matching calendar entry of said one or more obtained calendar entries;

assign meta-information obtained from said context information and said membership grade value to said user provided information, said membership grade value defining a measure which allows estimation of a reliability of a timely relatedness for retrieval; and

direct storage of said user provided information and said meta-information in a history storage in order to establish an information history functionality;

wherein said meta-information is employable for retrieval of said user provided information by matching request information of a retrieval request with said meta-information for selecting a user provided information being assigned to said meta-information matched to said request information.

31. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

obtain user provided audio information and additional user provided information;

record said user provided audio information;

direct storage of said user provided audio information;

obtain context information in parallel to said user provided audio information, said context information comprising user input information generated in consequence on a user action against said apparatus; and

generate meta-information comprising information about said additional user provided information and said context information.

32. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to obtain said user provided information at least in part by at least one of:

receiving user input information being generated by user operation of any input of said mobile terminal device or

receiving transaction information and/or communication information being received via any communication interface of said mobile terminal device,

wherein at least one of said user input information, said transaction information or said communication information represent said user provided information.

33. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to assign said meta-information to said user provided information at least in part by:

extracting at least one of labeling information or indexing information from said context information;

assigning said at least one of labeling information or indexing information to said user provided information; and

directing storage of said at least one of labeling information or indexing information and said user provided information;

wherein said at least one of labeling information or indexing information is employed as said meta-information for establishing retrieval functionality of said user input history.

34. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to associate said context information with said user provided information at least in part by:

generating referencing information for at least a part of said user provided information;

directing storage of said referencing information in a first storage area; and

directing storage of said part of said user provided information in a second storage area;

wherein said referencing information comprises address information which allow to retrieve said part of user provided information from said second storage area.

35. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

partition said user provided information into a plurality of information parts, each information part relating to a maximum pre-defined period of time, each information part being matched separately.

36. The apparatus according to claim 30, wherein, in case said first period of time exceeds said second period of time, the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

section said user provided information into at least two information sections, one of said at least two information sections fitting with said second period of time, said one fitting information section being matched.

37. The apparatus according to claim 30, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to derive said membership grade value by at least one out of averaging said membership function over said first period of time, determining a maximum of said membership function over said first period of time, or determining a minimum of said membership function over said first period of time.

38. The apparatus according to claim 30, wherein said membership grade value indicates said timely relatedness between said user provided information and one of said matching calendar entries.

39. An apparatus comprising at least one processor and at least one memory storing computer program code, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to at least:

obtain user provided information and context information associated with said user provided information, in consequence to any user operation against said apparatus, wherein said context information is related to at least one current condition of the apparatus at the time of said any user operation, wherein said user provided information includes user provided audio information;

access a code basis representing a plurality of coding symbols, said code basis comprising a pre-defined number of pre-defined frequencies, wherein said plurality of coding symbols represents a character and a symbol code table employable for coding said meta-information wherein said code basis is defined within a first frequency range, wherein said first frequency range is one frequency range of a plurality of frequency ranges forming a total frequency range within the frequency range of said user provided audio information, wherein said code basis is repeated within at least one further frequency range out of said plurality of frequency ranges;

code said meta-information in accordance with said code basis defined within said first frequency range and repeated within said at least one further frequency range to obtain redundancy;

combine user provided audio information and said coded meta-information by embedding said coded meta-information into said user provided audio information; and

direct storage of said user provided information having embedded said embedded meta-information in a history storage in order to establish an information history functionality;

wherein said meta-information is employable for retrieval of said user provided information by matching request information of a retrieval request with said meta-information for selecting a user provided information being assigned to said meta-information matched to said request information.

40. The apparatus according to claim 39, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to combine said user provided audio information and said coded meta-information at least in part by:

obtaining a modulation signal from said coded meta-information;

obtaining a modulated signal by combining said user provided audio information and said modulation signal in a frequency domain; and

obtaining said user provided audio information having embedded said meta-information by combining said modulated signal with said user provided audio information in a time domain.

41. The apparatus according to claim 40, wherein said obtaining of said modulated signal comprises:

obtaining said modulated signal by multiplying said user provided audio information and said modulation signal in said frequency domain.

42. The apparatus according to claim 40, wherein said obtaining of said user provided audio information having embedded said meta-information comprises:

obtaining said user provided audio information having embedded said meta-information by adding said modulation signal and said user provided audio information in said time domain.

43. The apparatus according to claim 39, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, further cause the apparatus to:

provide a correlation basis comprising each frequency being included in a set of code bases representing a plurality of coding symbols, said coding symbols being employed for coding said meta-information; and

apply said correlation basis onto said user provided audio information having embedded said meta-information to extract said meta-information;

wherein said extracted meta-information being available for retrieval.

44. The apparatus according to claim 43, wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to apply said correlation basis at least in part by:

obtaining a modulation signal from said correlation basis;

obtaining a correlation signal from said user provided audio information by convoluting said modulation signal and said user provided audio information having embedded said meta-information in a frequency domain;

extracting magnitude signal values from said correlation signal, said magnitude signal values corresponding to each code basis of said set of code bases; and

evaluating said magnitude signal values to retrieve said meta-information from said user provided audio information.

45. The apparatus according to claim 44, wherein said evaluating of said magnitude signals comprises:

obtaining magnitude ratio values of each pair of said magnitude signal values;

normalizing said magnitude ratio values; and

evaluating said normalized magnitude ratio values of said magnitude signal values to extract said meta-information from said user provided audio information.

46. The apparatus according to claim 44, wherein said correlation basis being defined in a first frequency range and wherein the at least one memory and stored computer program code are configured to, with the at least one processor, cause the apparatus to obtain said correlation signal at least in part by:

providing a correlation basis comprising all frequencies of said set of code bases within a first frequency range of a plurality of frequency ranges;

mapping said correlation basis into each frequency range of a plurality of frequency ranges, said plurality of frequency ranges forming a total frequency range being applicable to said user provided audio information to obtain said modulation signal; and

obtaining said correlation signal from said user provided audio information by convoluting said modulation signal and said user provided audio information in said frequency domain.