Feature extraction in a networked portable device

• US 20050215239A1
• Filed: 03/26/2004
• Published: 09/29/2005
• Est. Priority Date: 03/26/2004
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Makipaa, Mikko, Kopra, Toni, Vaananen, Mauri
• Application Number:
  US10/810,924
• Publication Number:
  US 20050215239A1
• Time in Patent Office: Days
• US Class Current: 455/414.100
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04M 1/656 for recording conversations
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72442 for playing music files
• CPC Class Codes: H04M 2203/105 Financial transactions and auctions, e.g. bidding
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 7/0036 where the data service is an information service

Abstract

A mobile station MS has a transducer or other interface to receive a media sample, a processor to extract features from a digital version of the media sample, a transmitter to transmit those extracted features, and preferably a buffer so feature extraction can begin on a portion of music received prior to a user input to identify the media sample. The features describe the content of the digital version of the media file. A server receives the extracted features from the MS, extracts additional features which may be higher order features extracted from the MS-extracted features and/or lower order features extracted from a segment of the digital media sample sent from the MS. The server uses all extracted features to search a database and identify the underlying file from which the media sample is taken. A single user entry at the MS initiates the feature extraction, link setup, and transmittal of features. A further entry purchases a copy of the entire media file that is downloaded to the MS.

Claims

1. A mobile station MS comprising:

an interface to receive a media sample;

a processor to extract at least one feature from a digital version of the media sample;

a transmitter to transmit the at least one extracted feature over a wireless communication link.

2. The MS of claim 1 wherein the interface comprises a transducer.

3. The MS of claim 2 wherein the transducer comprises a microphone and the media sample comprises an audio sample.

4. The MS of claim 2 wherein the transducer comprises a camera and the media sample comprises a visual sample.

5. The MS of claim 1 wherein the interface comprises one of a cable and a wireless link.

6. The MS of claim 5 wherein the media sample that the interface receives is the digital version.

7. The MS of claim 1 wherein said transmitter is further to transmit a message that includes the at least one extracted feature and no portion of the digital version of the media sample.

8. The MS of claim 1 wherein the processor is further to adaptively select a number of features to extract based on the digital version of the media sample.

9. The MS of claim 1 wherein the processor is further to adaptively select at least one type of feature to extract based on the digital version of the media sample, the processor extracts at least one feature of the adaptively selected type, and wherein the transmitter is further to transmit an identifier of the selected type of feature.

10. The MS of claim 1 wherein the digital version of the received media sample defines a first time-bounded segment and a second time-bounded segment, and the processor extracts the at least one feature from only the first segment and further transmits the second segment and not the first segment.

11. The MS of claim 10 wherein the processor further extracts a second feature from the second segment.

12. The MS of claim 1 further comprising a user interface by which a user may initiate the processor to so extract, and a buffer to store at least a portion of the digital version of the media sample prior to the user so initiating.

13. The MS of claim 1 wherein the at least one feature defines a timepoint within the digital version of the media sample, the processor is to extract a plurality of n timepoints from the digital version of the media sample, and the transmitter is to transmit at least n spectral slices of the digital version of the media sample and an identifier that links each spectral slice to at least one timepoint.

14. The MS of claim 13 wherein each (n+1) spectral slice corresponds to a larger portion of the digital version of the media sample than a preceding n^th slice.

15. The MS of claim 13 further comprising a receiver for receiving a sample identification message, and the processor is further to terminate transmitting further timepoints and spectral slices in response to receipt of the sample identification message.

16. The MS of claim 1 further comprising a user interface by which a single user input initiates:

the processor to extract the at least one feature, a wireless communications link to be established between the MS and a communication service, and the at least one extracted feature to be transmitted over the wireless communications link.

17. The MS of claim 16 wherein the single user input further initiates a buffer disposed between the transducer and the processor to begin storing at least a portion of the digital version of the media sample.

18. The MS of claim 1 wherein the processor is to extract a series of MPEG-7 descriptors from the digital version of the media sample.

19. The MS of claim 1 wherein the at least one feature is non-reconstructive of the digital version of the media sample.

20. The MS of claim 1 wherein all extracted features for which the transmitter is to transmit are non-reconstructive of the digital version of the media sample.

21. The MS of claim 1 further comprising a buffer disposed between the transducer and the processor to store at least a portion of one of the digital version of the media sample and the media sample.

22. The MS of claim 21 further comprising a receiver to receive a request for further features message, wherein, in response to receiving said request message, the processor extracts at least a second feature from said at least a portion stored in the buffer;

and further wherein the transmitter transmits the at least second feature.

23. A computer program, embodied on a computer readable medium within a mobile station, to process a media sample comprising:

a first set of computer instructions to extract at least one feature from a digital media sample; and

a second set of computer instructions to transmit the at least one extracted feature over a wireless communications link.

24. The computer program of claim 23 wherein the second set of computer instructions is further to transmit a message carrying the at least one extracted feature and no portion of the digital media sample.

25. The computer program of claim 23 wherein the first set of computer instructions is to adaptively select a number of features to extract based on the digital media sample, and extracting the selected number.

26. The computer program of claim 23 wherein

the first set of computer instructions is to adaptively select a type of feature to extract based on the digital media sample and to extract at least one feature of the adaptively selected type, and

the second set of computer instructions is to transmit the at least one extracted feature and an identifier of the selected type of feature.

27. The computer program of claim 23 wherein

the first set of computer instructions is to extract the feature from a first time-bounded segment of the digital media sample, and

the second set of computer instructions is to transmit a second time-bounded segment and not the first time-bounded segment.

28. The computer program of claim 27 wherein the first set of computer instructions is further to extract a second feature from the second segment.

29. The computer program of claim 23 wherein the first set of computer instructions is to extract at least one feature in response to a user input at the mobile station, the computer program further comprising a third set of computer instructions to continuously store at least a portion of the digital media sample in a buffer, and wherein the first set of computer instructions is further to extract the feature from that portion of the digital media sample stored in the buffer.

30. The computer program of claim 23 wherein the at least one feature defines a timepoint, the first set of computer instructions is to extract a plurality of n timepoints from the digital media sample, and the second set of computer instructions is to transmit at least n spectral slices of the digital media sample and an identifier that links each spectral slice to a timepoint.

31. The computer program of claim 30 wherein each (n+1) spectral slice corresponds to a larger portion of the digital media sample than a preceding n^th slice.

32. The computer program of claim 30 wherein the second set of computer instructions include instructions to terminate transmission of the n spectral slices of the digital media sample and an identifier that links each spectral slice to a timepoint upon receipt of a reply message that identifies the digital media sample.

33. The computer program of claim 23 wherein the first set of computer instructions begins extracting at least one feature in response to a single user input at the mobile station.

34. The computer program of claim 33 wherein the first set of computer instructions further includes instructions to begin storing in a buffer of the MS at least a portion of the digital media sample.

35. The computer program of claim 23 wherein the first set of computer instructions is to extract at least one feature from a digital media sample that is non-reconstructive of that digital media sample.

36. The computer program of claim 23 wherein the first set of computer instructions further is to store in a buffer of the MS at least a portion of the digital media sample.

39. The computer program of claim 37 wherein a communication link between a sender of the message that includes the received features and the computer program of claim 34 remains open between at least a first time defined by receipt of the message that includes received features and a second time defined by transmission of the reply.

37. A computer program embodied on a computer readable medium to uniquely match a plurality of extracted features to a feature set stored in a database comprising:

a first set of computer instructions to receive over a network a message that includes received features;

a second set of computer instructions to extract additional features from the message; and

a third set of computer instructions to search a database of feature sets until a plurality of received features combined with extracted additional features uniquely matches only one feature set of the database.

38. The computer program of claim 37 wherein each feature set is associated with a media file title, the computer program further comprising a fourth set of computer instructions to transmit, over the network to a sender of the message, a reply that includes the media file title.

40. The computer program of claim 38 wherein the third set of computer instructions further is to determine a link address for a media file uniquely associated with the only one feature set, and wherein the fourth set of computer instructions is further to transmit the link address in the reply.

41. The computer program of claim 37 wherein the third set of computer instructions includes instructions to search the database of feature sets using only the received features, followed by searching the database with the extracted additional features.

42. The computer program of claim 41 wherein the third set of computer instructions includes instructions to search the database of file feature sets using only the received features and to simultaneously extract additional features from the message.

43. The computer program of claim 37 wherein the second set of computer instructions is to extract additional features from the received features.

44. The computer program of claim 37 wherein the second set of computer instructions is to extract additional features from a portion of the message that includes a segment of a digital media sample.

45. The computer program of claim 44 wherein the second set of computer instructions is further to extract additional features from the received features.

46. The computer program of claim 37 further comprising a fifth set of computer instructions to send a request message requesting further features when the third set of computer instructions fails to find a unique match in the database, and to re-execute at least the first and third sets of computer instructions upon receiving a second message that includes received further features.

47. The computer program of claim 46 wherein the request message includes at least one of a number and a type of the further features.

48. A mobile station comprising:

means for receiving a media sample;

means for extracting at least one feature from a digital version of the media sample;

means for transmitting the at least one extracted feature over a wireless communication link.

49. The mobile station of claim 48, wherein the means for receiving comprises a transducer, and the means for extracting comprises a digital processor.

50. A method for signaling information about a media file to a remote database, comprising:

at a portable wireless device, receiving a media sample;

at the portable wireless device, extracting a plurality of features from a digital version of the media sample;

transmitting from the portable wireless device a message that includes the extracted plurality of features and that excludes the digital version of the media sample.