Method and communication system for continuously recording sounding information

• US 8,571,529 B2
• Filed: 08/31/2007
• Issued: 10/29/2013
• Est. Priority Date: 08/31/2006
• Status: Active Grant

• Current Assignee: InterDigital Patent Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Swisscom AG (Government of Switzerland)
• Inventors: Lauper, Eric
• Primary Examiner(s): Brandt, Christopher M
• Application Number:
  US11/848,519
• Publication Number:
  US 20080057911A1
• Time in Patent Office: 2,251 Days
• Field of Search: 455/412.1, 455/412.2, 455/414.1, 455/414.2, 455/414.4, 455/41.2, 455/572, 455/574, 340/539.1, 340/539.11, 340/539.13, 340/539.14, 340/539.16, 340/539.17, 340/539.3
• US Class Current: 455/414.4
• CPC Class Codes: G06F 16/64 Browsing; Visualisation therefor generation of a list or set of audio data G06F16/638
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: H04L 67/00 Network arrangements or protocols for supporting network services or applications user-to-user messaging H04L51/00; network arrangements, protocols or services for supporting real-time applications in data packet communications networks H04L65/00
• CPC Class Codes: H04M 1/72451 according to schedules, e.g. using calendar applications
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

A method is disclosed for continuously recording surrounding information by a mobile device (11). The mobile device (11) of a user (1) continuously records surrounding information such as surrounding noises or surrounding image data and at least one file (7) is generated from the recording and sent to a remote server (4). Said file or a file derived therefrom (7) is automatically subdivided and/or indexed automatically depending on the contents of the file (7) and/or on metadata (M₁, M₂, M₃) generated from external events (e₁, e₂, e₃) in relation to the mobile device (11) and/or by the user (1). Advantageously, the user (1) searches the subdivided and/or indexed file (7) so that he can easily retrieve the recorded information.

Claims

1. Method for continuously recording surrounding information by a mobile device with the following method steps:

the mobile device of a user continuously recording surrounding audio information and sending resulting data to a remote server, wherein;

the data is classified and/or indexed automatically by the remote server such that said classification and/or indexing is determined in an automated manner depending on the content of the data and/or on metadata that are generated from external events in relation to the mobile device and/or by the user,

the classified and/or indexed data is formatted and stored to be electronically searchable by the user accessing said remote server from a remote location, and

the mobile device records the data and if the data is recognized as being meaningless before the digital conversion, the mobile device switches to a sleep mode.

2. The method of claim 1, wherein, from the recorded information, one or a plurality of files are generated in the mobile device or in the remote server.

3. The method of claim 2, further comprising the step of associating the metadata temporally with the file generated or derived therefrom, according to the external events or this file is subdivided through the metadata in such a manner that the file can be searched according to the associated metadata.

4. The method of claim 2, wherein the mobile device is connected to a mobile radio network and all recorded surrounding information is sent to a remote server and said file is stored there together with the metadata in a memory area allocated to the user.

5. The method of claim 1, wherein the external events are recorded automatically, are preprogrammed and/or are set individually by the user.

6. The method of claim 1, wherein the content of the data and the external events include the location or changes of the location of the mobile device, features in the room, data, time, events within the mobile device or data from sensors connected to the mobile device, predetermined noises within the file that are to be detected in the vicinity and/or recorded conversations or interlocutors of the user.

7. The method of claim 1, wherein the mobile device is in the sleep mode if no useful wanted signals is received.

8. The method of claim 1, wherein the mobile device records, converts digitally and analyzes all data and if these are not recognized as meaningful, they are not transmitted to the remote server.

9. The method of claim 1, wherein the sleep mode of the recording function is independent of the sleep mode of the mobile device.

10. The method of claim 1, wherein switching to the sleep mode or awakening from the sleep mode can be initiated automatically by a noise or an image, or automatically depending on said events, or by the user, or automatically periodically or by the remote server.

11. The method of claim 1, wherein events can also be transmitted by the mobile device to the remote server when the mobile device is in the sleep mode.

12. The method of claim 1, further comprising the step of local buffering of received signals or file parts in the memory of the mobile device before transmission to the remote server takes place.

13. The method of claim 1, wherein a voice recognition system converts the recorded data into a text file and the metadata are associated temporally to the generated text file according to the external events.

14. The method of claim 1, wherein a voice recognition system converts the recorded data into a text file and recognizes different conversation parts of a conversation and the participation of at least one interlocutor is used as metadata.

15. The method of claim 13, wherein the used voice recognition system uses user-dependent language models.

16. The method of claim 13, wherein new language models are established through new conversations for the individual interlocutors.

17. The method of claim 13, wherein the voice recognition occurs locally in the mobile device or in the remote server.

18. The method of claim 13, wherein the recorded data and/or the text file are stored.

19. The method of claim 1, wherein the audio, video or text data transmitted to the remote server are signed by means of a certificate by the mobile device in order to guarantee the authenticity of the data.

20. The method of claim 1, wherein the user adds further data to said recorded data.

21. The method of claim 1, wherein the recorded data are marked by the user at a later stage after the recording with metadata.

22. The method of claim 1, wherein different persons have access rights to different parts of the data that are marked with certain metadata.

23. The method of claim 1, wherein the user has access to the recorded data from his mobile device, over an Internet site or over a calendar program and searches these data according to contents or according to metadata.

24. The method of claim 1, wherein the search is performed over a voice command.

25. The method of claim 1, wherein a program or applet is provided on the mobile device that performs the continuous recording and the processing of the recorded data.

26. The method of claim 1, wherein a service provider offers a user of a mobile device for performing said method and bills a corresponding service to the user.

27. The method of claim 1, wherein a video recording is made permanently by the mobile device or by a device connected to the mobile device and the data are also sent to the remote server.

28. The method of claim 1, wherein the subdividing and/or indexing of the recorded data occurs through conversion into a markup language file, which performs a subdivision in different files, where each file is connected with metadata or by adding a marked index in an audio file.

29. The method of claim 1, wherein the search results are reproduced as text or as synthetic voice.

30. The method of claim 1, wherein said resulting data is buffered during or prior to said sending, with said sending step including streaming said resulting data to the remote server.

31. The method of claim 1, wherein said surrounding information is sound that is recorded by said mobile device.

32. Communication system including:

a mobile device for continuously recording surrounding information and streaming resulting data to a remote server, wherein the mobile device records the data and if the data is recognized as being meaningless before the digital conversion, the mobile device switches to a sleep mode,

means for automatically classifying and/or indexing the data, said classifying and/or indexing performed automatically depending on the content of the data and/or on metadata that are generated from external events in relation to the mobile device and/or by the user, and

means for the user remotely searching the classified and/or indexed data, wherein the classified and/or indexed data is formatted and stored.

33. The communication system of claim 32, characterized by means in the mobile device or in the remote server for generating from the recorded data one or a plurality of files.

34. The communication system of claim 33, characterized by means for temporally associating the metadata with the file generated or derived therefrom, according to the external events or for subdividing this file through the metadata in such a manner that the file can be searched according to the associated metadata.

35. The communication system of claim 32, wherein the mobile device is a mobile connected to a mobile radio network and a remote server is connected to the mobile radio network and said data can be stored in the remote server together with the metadata in a memory area allocated to the user.

36. The communication system of claim 32, characterized by a voice recognition system that converts the recorded data into a text file, means being provided for temporally associating the metadata with the generated text file according to the external event.

37. The communication system of claim 32, wherein the used voice recognition system uses user-dependent language models.

38. The communication system of claim 32, characterized by access rights for different people to different portions of the data that are marked with certain metadata.

39. Method for continuously recording surrounding information using a server connected to a mobile device:

the server receiving, via a communication network from a remote location, recorded data from a mobile device that is continuously recording surrounding information and streaming resulting data to the server, wherein the mobile device records the data and if the data is recognized as being meaningless before the digital conversion, the mobile device switches to a sleep mode,

the server automatically classifying and/or indexing the recorded data in an automated manner based on the content of the recorded data and/or based on metadata received by the server that are generated from external events in relation to the mobile device and/or by the user; and

the server providing a capability to a user located remotely from the server to search the classified and/or indexed data via a connection with the mobile device connected to said server or another device connected to said server.

40. Method for continuously recording surrounding information using a server connected to a mobile device:

the server receiving, via a communication network from a remote location, recorded data from a mobile device of a user that is continuously recording information surrounding the user and streaming resulting data to the server, wherein the mobile device records the data and if the data is recognized as being meaningless before the digital conversion, the mobile device switches to a sleep mode,

the server automatically classifying and/or indexing the recorded data in an automated manner based on the content of the recorded data and/or based on metadata received by the server that are generated from external events in relation to the mobile device and/or by the user; and

the server providing a capability to the user located remotely from the server to search the classified and/or indexed data via a connection with the mobile device connected to said server, wherein such search is based on the content of the recorded data and/or based on metadata.