System and method for communication between mobile devices using digital/acoustic techniques

• US 8,750,473 B2
• Filed: 09/03/2009
• Issued: 06/10/2014
• Est. Priority Date: 09/03/2008
• Status: Active Grant

• Current Assignee: Smule, Inc.
• Original Assignee: Smule, Inc.
• Inventors: Cook, Perry R.
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US12/553,788
• Publication Number:
  US 20100053169A1
• Time in Patent Office: 1,741 Days
• Field of Search: 379/344, 379/93.37, 455/11.1, 455/20, 455/22, 455/41.2
• US Class Current: 379/93.37
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: H04B 11/00 Transmission systems employing sonic, ultrasonic or infrasonic waves
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04W 76/14 Direct-mode setup

Abstract

Techniques have been developed for transmitting and receiving information conveyed through the air from one portable device to another as a generally unperceivable coding within an otherwise recognizable acoustic signal. For example, in some embodiments in accordance with the present invention(s), information is acoustically communicated from a first handheld device toward a second by encoding the information in a signal that, when converted into acoustic energy at an acoustic transducer of the first handheld device, is characterized in that the acoustic energy is discernable to a human ear yet the encoding of the information therein is generally not perceivable by the human. The acoustic energy is transmitted from the acoustic transducer of the first handheld device toward the second handheld device across an air gap that constitutes a substantially entirety of the distance between the devices. Acoustic energy received at the second handheld device may then be processed using signal processing techniques tailored to detection of the particular information encodings employed.

Claims

1. A peer-to-peer method of acoustically and bi-directionally communicating information between a first portable device and a second, the method comprising:

encoding first information in a signal that, when converted into first acoustic energy at an acoustic transducer of the first portable device, is characterized in that the first acoustic energy is discernable to a human ear yet the encoding of the first information therein is not perceivable by the human apart from the first acoustic energy itself;

transmitting the first acoustic energy from the acoustic transducer of the first portable device toward the second portable device across an air gap that constitutes a substantially entirety of the distance therebetween; and

receiving at the acoustic transducer of the first portable device second acoustic energy, the second acoustic energy being from the second portable device, wherein the second acoustic energy is also characterized in that the second acoustic energy is itself discernable to a human ear yet an encoding of second information therein is not perceivable by the human.

10. The method of claim 1,

wherein the first information encoded in the acoustic transmission conveys an identifier associated or associable with a user of the first portable device.

11. The method of claim 10, wherein the user identifier encodes one or more of:

a signature;

a vcard; and

a resource identifier, the resource identifier usable by the second portable device to retrieve information associated with the user.

12. The method of claim 1,

wherein the first information encoded in the acoustic transmission conveys an identifier associated or associable with the first portable device.

13. The method of claim 12,

wherein the first portable device includes a radio frequency (RF) communication interface;

and

wherein the identifier conveyed by the acoustic transmission facilitates RF communication with the first portable device.

14. The method of claim 13,

wherein the identifier conveyed constitutes a beacon that identifies the first portable device and is broadcast to the second portable device and to at least one other device acoustically proximate to the first portable device.

15. The method of claim 13,

wherein the identifier conveyed by the first information encoded in the acoustic transmission solicits or facilitates a pairing interchange over the radio frequency (RF) communication interface in accord with a wireless personal area network protocol.

16. The method of claim 1, further comprising:

at the first portable device, synthesizing the first acoustic energy to present to the human ear a sound having a generally recognizable physical analog.

17. The method of claim 16,

wherein the first acoustic energy includes the encoded first information at least partially as a sinusoid not employed in the synthesis of sound corresponding to the generally recognizable physical analog, but detectable at the second portable device.

18. The method of claim 16,

wherein the first acoustic energy includes the encoded first information at least partially as one or more of frequency, phase and amplitude modulation of a sinusoid employed in the synthesis of sound corresponding to the generally recognizable physical analog.

19. The method of claim 16,

wherein the first acoustic energy includes the encoded first information at least partially as a selection plural non-harmonic overtones thereof.

20. The method of claim 16,

wherein the first acoustic energy includes the encoded first information at least partially as frequency, phase and amplitude modulation of plural non-harmonic overtones thereof.

21. The method of claim 16,

wherein the first acoustic energy includes harmonics of one or more fundamental frequencies associated with the generally recognizable physical analog;

and

wherein the encoded first information is represented at least partially as a frequency, phase and amplitude perturbation of at least one of the harmonics.

22. The method of claim 16,

wherein the first acoustic energy includes noise;

and

wherein the encoded first information is represented at least partially as a structure imparted to the noise.

23. The method of claim 16,

wherein the first acoustic energy includes a plurality of constituents presented in phase in a stereo field together with an additional constituent that is out of phase with the plurality of constituents;

wherein the encoded first information is represented at least partially using the out-of-phase constituent.

24. A computer program product encoded in one or more media, the computer program product including instructions executable on a processor of the first portable device to cause the first portable device to perform the method of claim 1.

25. The computer program product of claim 24, wherein the one or more media are readable by the first portable device or readable incident to a computer program product conveying transmission to the first portable device.

26. The method of claim 1, further comprising:

sampling a signal transduced from the second acoustic energy; and

processing the sampled signal at the first portable device to extract the second information encoded therein.

33. The method of claim 1,

wherein the second acoustic energy presents to the human ear a sound having a generally recognizable physical analog.

35. The method of claim 33,

wherein the received acoustic energy includes the encoded information at least partially as one or more of frequency, phase and amplitude modulation of a sinusoid employed in a synthesis of sound corresponding to the generally recognizable physical analog.

36. The method of claim 33,

wherein the received acoustic energy includes the encoded information at least partially as a selection plural non-harmonic overtones thereof.

37. The method of claim 33,

wherein the received acoustic energy includes the encoded information at least partially as frequency, phase and amplitude modulation of plural non-harmonic overtones thereof.

38. The method of claim 33,

wherein the received acoustic energy includes harmonics of one or more fundamental frequencies associated with the generally recognizable physical analog;

and

wherein the encoded information is represented at least partially as a frequency, phase and amplitude perturbation of at least one of the harmonics.

39. The method of claim 33,

wherein the received acoustic energy includes a plurality of constituents presented in phase in a stereo field together with an additional constituent that is out of phase with the plurality of constituents;

wherein the encoded information is represented at least partially using the out-of-phase constituent.

2. A method of acoustically communicating information from a first portable device toward a second, the method comprising:

encoding the information in a signal that, when converted into acoustic energy at an acoustic transducer of the first portable device, is characterized in that the acoustic energy is discernable to a human ear yet the encoding of the information therein is not perceivable by the human apart from the acoustic energy itself; and

transmitting the acoustic energy from the acoustic transducer of the first portable device toward the second portable device across an air gap that constitutes a substantially entirety of the distance therebetween,

wherein the information encoded in the acoustic transmission conveys one of character and state for a physical object simulated at the first portable device.

3. The method of claim 2,

wherein the simulated physical process includes a flame for which a dynamic visual presentation is displayed on the first portable device;

wherein the acoustic energy discernable to the human ear includes one or more of pops and crackles consonant with the displayed flame; and

wherein the information not perceivable by the human but coded in the acoustic energy includes one or more of size, orientation, simulated heat and color of the displayed flame.

4. The method of claim 3,

wherein the information not perceivable by the human but coded in the acoustic energy further characterizes a wind field in which the simulated flame burns.

5. The method of claim 2,

wherein the simulated physical object is a musical instrument characterized, at least in part, by plural non-harmonic overtones produced thereby.

6. The method of claim 5,

wherein the physical object simulated is one of a Tibetan prayer bowl, clock/church tower bells and a bell-type object.

7. The method of claim 5, further comprising:

propagating an instance of the characterized musical instrument to the second portable device for simulation thereon,

the characterizing non-harmonic overtones not individually perceivable by the human.

8. The method of claim 5, further comprising:

multi-dimensionally coding the information as one or more of frequency, phase and amplitude modulations of at least some of the characterizing non-harmonic overtones.

9. The method of claim 8, wherein the multi-dimensional coding conveys one or more of:

a 6-tuple that corresponds to a media access control (MAC) address of the first portable device; and

a serial number or other identifier.

27. A method of receiving acoustically communicated information from a first portable device, the method comprising:

receiving acoustic energy at an acoustic transducer of a second portable device, the received acoustic energy transmitted from the first portable device across an air gap that constitutes a substantially entirety of the distance between the first and second portable devices, wherein the received acoustic energy is characterized in that the acoustic energy is itself discernable to a human ear yet an encoding of information therein is not perceivable by the human;

sampling a signal transduced from the received acoustic energy; and

processing the sampled signal at the second portable device to extract the information encoded therein,

wherein the information encoded in the received acoustic energy conveys one of character and state for a physical object simulated at the first portable device.

28. A computer program product encoded in one or more media, the computer program product including instructions executable on a processor of the second portable device to cause the second portable device to perform the method of claim 27.

29. The computer program product of claim 27, wherein the one or more media are readable by the first portable device or readable incident to a computer program product conveying transmission to the second portable device.

32. The method of claim 27, wherein the processing includes one or more of:

applying one or more resonant tuned resonant filters to the sampled signal;

peak picking in a range of spectral magnitudes computed for the sampled signal using a fast Fourier transform (FFT);

applying a quadrature heterodyne filter (QHF) to the sampled signal; and

computing a power spectral density (PSD) for the sampled signal.

30. A portable device comprising:

one or more acoustic transducers capable of converting a first signal into outgoing acoustic energy and capable of converting into a second signal incoming acoustic energy received from across an air gap that constitutes a substantially entirety of the distance between the portable device and a second portable device;

a transmit path that encodes in the first signal, first information destined for one or more remote receivers, wherein the first signal is characterized in that outgoing acoustic energy converted therefrom is discernable to a human ear yet the encoding of the first information therein is not perceivable by the human apart from the acoustic energy itself; and

a receive path that extracts second information from the second signal transduced from the incoming acoustic energy, wherein the incoming acoustic energy is discernable to the human ear yet the second information encoded therein is not perceivable by the human apart from the acoustic energy itself.

31. The portable device of claim 30, embodied as one or more of a handheld mobile device, a mobile phone, a laptop or notebook computer, a personal digital assistant, a smart phone, a media player, a netbook, and a book reader.

41. The portable device of claim 30,

wherein either or both of the outgoing and incoming acoustic energy presents to the human ear a sound having a generally recognizable physical analog.

34. A method of receiving acoustically communicated information from a first portable device, the method comprising:

receiving acoustic energy at an acoustic transducer of a second portable device, the received acoustic energy transmitted from the first portable device across an air gap that constitutes a substantially entirety of the distance between the first and second portable devices, wherein the received acoustic energy is characterized in that the acoustic energy is itself discernable to a human ear yet an encoding of information therein is not perceivable by the human;

sampling a signal transduced from the received acoustic energy; and

processing the sampled signal at the second portable device to extract the information encoded therein,

wherein the received acoustic energy presents to the human ear a sound having a generally recognizable physical analog, and

wherein the received acoustic energy includes the encoded information at least partially as a sinusoid not employed in a synthesis of the sound corresponding to the generally recognizable physical analog, but nonetheless detectable at the second portable device.

40. A portable device comprising:

one or more acoustic transducers capable of converting a first signal into outgoing acoustic energy and capable of converting into a second signal incoming acoustic energy received from across an air gap that constitutes a substantially entirety of the distance between the portable device and a second portable device;

a transmit path that encodes in the first signal, first information destined for one or more remote receivers, wherein the first signal is characterized in that outgoing acoustic energy converted therefrom is discernable to a human ear yet the encoding of the first information therein is not perceivable by the human apart from the acoustic energy itself; and

a receive path that extracts second information from the second signal transduced from the incoming acoustic energy, wherein the incoming acoustic energy is discernable to the human ear yet the second information encoded therein is not perceivable by the human apart from the acoustic energy itself,

wherein the information encoded in either or both of the outgoing and incoming acoustic energy conveys one of character and state for a physical object simulated at the first portable device.

42. A method of acoustically communicating information from a first portable device toward a second, the method comprising:

encoding the information in a signal that, when converted into acoustic energy at an acoustic transducer of the first portable device, is characterized in that the acoustic energy is discernable to a human ear yet the encoding of the information therein is not perceivable by the human apart from the acoustic energy itself;

transmitting the acoustic energy from the acoustic transducer of the first portable device toward the second portable device across an air gap that constitutes a substantially entirety of the distance therebetween; and

at the first portable device, synthesizing the acoustic energy to present to the human ear a sound having a generally recognizable physical analog,

wherein the acoustic energy includes at least one or more selected from the group consisting of;

(i) the encoded information at least partially as a sinusoid not employed in the synthesis of sound corresponding to the generally recognizable physical analog, but detectable at the second portable device,

(ii) the encoded information at least partially as one or more of frequency, phase and amplitude modulation of a sinusoid employed in the synthesis of sound corresponding to the generally recognizable physical analog,

(iii) the encoded information at least partially as a selection plural non-harmonic overtones thereof,

(iv) the encoded information at least partially as frequency, phase and amplitude modulation of plural non-harmonic overtones thereof,

(v) harmonics of one or more fundamental frequencies associated with the generally recognizable physical analog, wherein the encoded information is represented at least partially as a frequency, phase and amplitude perturbation of at least one of the harmonics,

(vi) noise, wherein the encoded information is represented at least partially as a structure imparted to the noise, and

(vii) a plurality of constituents presented in phase in a stereo field together with an additional constituent that is out of phase with the plurality of constituents, wherein the encoded information is represented at least partially using the out-of-phase constituent.

43. A method of receiving acoustically communicated information from a first portable device, the method comprising:

receiving acoustic energy at an acoustic transducer of a second portable device, the received acoustic energy transmitted from the first portable device across an air gap that constitutes a substantially entirety of the distance between the first and second portable devices, wherein the received acoustic energy is characterized in that the acoustic energy is itself discernable to a human ear yet an encoding of information therein is not perceivable by the human;

sampling a signal transduced from the received acoustic energy; and

processing the sampled signal at the second portable device to extract the information encoded therein,

wherein the received acoustic energy presents to the human ear a sound having a generally recognizable physical analog,

wherein the received acoustic energy includes at least one or more selected from the group consisting of;

(i) the encoded information at least partially as one or more of frequency, phase and amplitude modulation of a sinusoid employed in a synthesis of sound corresponding to the generally recognizable physical analog,

(ii) the encoded information at least partially as a selection plural non-harmonic overtones thereof,

(iii) the encoded information at least partially as frequency, phase and amplitude modulation of plural non-harmonic overtones thereof,

(iv) harmonics of one or more fundamental frequencies associated with the generally recognizable physical analog, wherein the encoded information is represented at least partially as a frequency, phase and amplitude perturbation of at least one of the harmonics, and

(v) a plurality of constituents presented in phase in a stereo field together with an additional constituent that is out of phase with the plurality of constituents, wherein the encoded information is represented at least partially using the out-of-phase constituent.