Messaging by writing an image into a spectrogram

• US 9,406,305 B2
• Filed: 12/20/2013
• Issued: 08/02/2016
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Kamath, Ajith M.
• Primary Examiner(s): Bhatnagar, Anand
• Application Number:
  US14/137,663
• Publication Number:
  US 20140185862A1
• Time in Patent Office: 956 Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 20/3829 involving key management
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal

Abstract

A message is communicated between devices by passing a host signal, such as an audio signal, with the message written into the spectrogram of the host signal. For host audio signals, the message is authored, converted to an image format (if not already in suitable image form) and written into the spectrogram of the audio signal. The resulting audio signal can be communicated in a variety of ways, but in one particular embodiment, the audio signal is played from a speaker (e.g., of a sending mobile device) and captured through the microphone of a receiving device. The spectrogram of the received audio signal is computed and then rendered to a display screen (e.g., the display of a receiving mobile device).

Claims

1. A messaging method comprising:

providing an image;

writing the image into a spectrogram of a host signal to produce a modified host signal; and

transferring the modified host signal, in which a human readable message is conveyed by the image within the spectrogram;

wherein providing an image comprises taking image-based message input from a user via a user input device, and producing a corresponding two dimensional image; and

wherein the host signal comprises a host audio signal.

2. The method of claim 1 wherein the writing comprises inserting the two dimensional image into the spectrogram of the host audio signal by modifying frequency components in the spectrogram, such that the two dimensional image is visible to a human when the spectrogram is generated and displayed as a two dimensional image on a display device.

3. The method of claim 2 further comprising:

obtaining the host audio signal in response to selection by a user.

4. The method of claim 3 wherein the host audio signal comprises a song selected by the user.

5. The method of claim 3 wherein the host audio signal comprises audio input captured from a user via a microphone.

6. The method of claim 5 further including:

modifying the host audio signal with special effects selected by a user prior to inserting the two dimensional image into the spectrogram of the host audio signal.

7. The method of claim 2 including:

preconditioning the spectrogram before inserting the two dimensional image into the spectrogram by adding spectral content into the spectrogram in an area of the spectrogram where the two dimensional image is to be written.

8. The method of claim 1 further including converting the modified host signal into an audio signal and sending the audio signal to a speaker to produce audio output for device to device message communication.

9. The method of claim 1 implemented in a mobile device application executing on a mobile device, the mobile device application including a message authoring module, executing on the mobile device, enabling a user to author a message and providing that message in image form as the image, and a message writing module, executing on the mobile device, for writing the image into a spectrogram of a host audio signal.

10. The method of claim 9 wherein the mobile device application further includes a message rendering module, executing on the mobile device, for receiving an audio signal captured by a microphone of the mobile device, converting the captured audio signal into a spectrogram and sending the spectrogram in image form to a display device on the mobile device for display.

11. The method of claim 1 wherein the transferring comprises transferring to a messaging network service in which the modified host signal is posted, and is made available for one or more destination devices.

12. A message rendering method comprising:

receiving an audio signal;

converting the audio signal into a spectrogram;

rendering the spectrogram for display, wherein an image written into the spectrogram is displayed, the image having been written into the spectrogram by inserting an array of two dimensional image elements to corresponding two dimensional time-frequency locations within the spectrogram.

13. The method of claim 12 wherein the receiving comprises receiving the audio signal from a microphone of a mobile device, the microphone capturing the audio signal from a neighboring device in which the image was written into the spectrogram of the audio signal.

14. The method of claim 12 wherein the receiving comprises receiving a voice telephone call over a telecommunications network.

15. The method of claim 12 wherein the receiving, converting and rendering is performed by a mobile application, executing on a mobile device, and the mobile application including a message authoring module, executing on the mobile device, enabling a user to author a message and providing that message in image form as the image, and a message writing module, executing on the mobile device, for writing the image into a spectrogram of a host audio signal.

16. The method of claim 12, further including:

decoding machine readable information from a private channel in the audio signal, and using the machine readable information to determine authorized recipient of the image written into the spectrogram.

17. The method of claim 12, further including:

decoding machine readable information from a private channel in the audio signal to extract control data for determining whether a recipient has access to view the image; and

decoding machine readable information from a public channel in the audio signal to control distribution of a message within the spectrogram of the audio signal.

18. A non-transitory computer readable medium, on which is stored instruction, which when executed by one or more processors, perform a messaging method comprising:

providing an image;

writing the image into a spectrogram of a host signal to produce a modified host signal; and

transferring the modified host signal, in which a human readable message is conveyed by the image within the spectrogram;

wherein providing an image comprises taking image-based message input from a user via a user input device, and producing a corresponding two dimensional image;

wherein the host signal comprises a host audio signal; and

wherein the writing comprises inserting the two dimensional image into the spectrogram of the host audio signal by modifying frequency components in the spectrogram, such that the two dimensional image is visible to a human when the spectrogram is generated and displayed as a two dimensional image on a display device.