System and method for coding audio information in images

• US 6,687,383 B1
• Filed: 11/09/1999
• Issued: 02/03/2004
• Est. Priority Date: 11/09/1999
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Pickover, Clifford A., Maes, Stephane, Zlatsin, Alexander, Kanevsky, Dimitri
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US09/436,163
• Time in Patent Office: 1,547 Days
• Field of Search: 382/100, 382/232, 382/240, 380/51, 380/54, 380/210, 380/252, 380/287, 380/59, 380/73.1, 380/201, 380/205, 713/176, 713/179, 348/460, 348/461, 348/463, 709/217
• US Class Current: 382/100
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal

Abstract

A system and method for encoding sound information in image sub-feature sets comprising pixels in a picture or video image. Small differences in intensity of pixels in this image set are not detectable by eyes, but are detectable by scanning devices that measure these intensity differences between closely situated pixels in the sub-feature sets. These encoded numbers are mapped into sound representations allowing for the reproduction of sound.

Claims

1. A system for embedding audio information in image data corresponding to a whole image for display or print, said image data comprising pixels, the system comprising:

device for characterizing a sub-area in said whole image as a pixel block comprising a predetermined number of pixels, each pixel block including first and second complementary sets of pixels representing respective first and second image sub-feature sets, a first image sub-feature set including pixels comprising whole image content to be displayed or printed; and

, a second image sub-feature including pixels comprising coded audio information; and

,

audio-video transcoding device for associating said second image sub-feature set with units of audio information, said transcoding being performed so that image sub-features in the second set satisfy constraints related to visibility of said whole image.

2. The system as claimed in claim 1, wherein said whole image corresponds to a digital space associated with a digital information presentation device including a memory storage and a CPU, each said pixel comprising a unit of computer memory and including predefined number of data bits.

3. The system as claimed in claim 2, wherein each said pixel value includes a first predefined number of data bytes of memory storage representing whole image content and a second predefined number of data bytes representing coded audio information, said second predefined number of data bytes being smaller than said first predefined number of data bytes.

4. The system as claimed in claim 3, wherein each byte of said first predefined number of data bytes of memory storage represents a color or intensity of a color of said image.

5. The system as in claim 2, wherein an amount of said second set of pixels having values comprising coded audio information in said pixel block is less than an amount of said first set of pixels in said pixel block.

6. The system as in claim 2, wherein pixel locations in a pixel block comprise indices into a table of values for said pixel, said table including pixel values corresponding to whole image content and audio information.

7. The system as claimed in claim 2, wherein said digital information presentation device includes electronic paper.

8. The system as claimed in claim 1, where each sub-area is characterized as having a shape according to one selected from shapes including:

square, rectangle, triangle, circle, polygon, oval.

9. The system as claimed in claim 1, further comprising means for specifying constraints related to visibility of said whole image, said constraints specified in accordance with prioritization of visual image content.

10. The system as claimed in claim 9, wherein said transcoding device includes audio-to-video transcoder for transforming audio data into video data, and inserting said video data as video sub-features in the second set according to said constraints related to visibility of said whole image.

11. The system as claimed in claim 1, wherein said transcoding device for associating said second image sub-feature set with units of audio information further includes:

means for mapping video sub-features of said second image sub-feature set into indexes of units of audio information;

said video sub-features being ordered in a predetermined fashion, wherein said mapping means induces an order of units of audio information for providing a global audio information content.

12. The system as claimed in claim 11, wherein said means for mapping video sub-features into indexes of units of audio information includes:

means for relating video-sub features to number values, an order of sub-features inducing an order of said number values;

means for constructing a sequence of new number values based on sequences of prior ordered number values; and

,

table means having entry indexes according to said sequence of new number values.

13. The system as claimed in claim 12, wherein said new number values are constructed applying algebraic formulae to sequences of prior number values.

14. The system as claimed in claim 12, wherein said means for relating video-sub features to number values comprises:

means for classifying sub-features according to physical quantities represented by said sub-features, and assigning number values to said classes, said number values representing intensity of said classified physical quantity.

15. The system as claimed in claim 14, where physical quantities are one of the following:

color, waveform type, wavelength, frequency, thickness.

16. The system as claimed in claim 1, further comprising:

a video-image processing device for extracting said audio information that is embedded in said second image sub-feature set.

17. The system as claimed in claim 14, wherein said extracting means comprises:

means for determining said second image sub-feature set areas of said image comprising said coded audio data, said video sub-features in said second sub-feature set being ordered in a predetermined fashion;

means for determining content of video sub-features in video data as indexes to units of audio information and inducing an order on the units of audio information; and

,

means for processing units of audio information in the induced order to produce an audio message from an audio playback device.

18. The system as claimed in claim 16, wherein said audio information includes conversational mark-up language (CML) data accessible via a speech browser for playback therefrom.

19. A method for embedding audio information in image data corresponding to a whole image for display or print, said image data comprising pixels, the method steps comprising:

characterizing a sub-area in said whole image as a pixel block comprising a predetermined number of pixels, each pixel block including first and second complementary sets of pixels representing respective first and second image sub-feature sets, a first image sub-feature set including pixels comprising whole image content to be displayed or printed; and

, a second image sub-feature including pixels comprising coded audio information; and

,

encoding pixels of said first image sub-feature set with whole image content to be displayed or printed and pixels of said second image sub-feature set with coded audio information, said encoding of said audio data performed such that image sub-features in the second set satisfy constraints related to visibility of said whole image.

20. The method as claimed in claim 19, wherein said whole image corresponds to a digital space associated with a digital information presentation device including a memory storage and a CPU, each said pixel comprising a unit of computer memory and including a predefined data bit value.

21. The method as claimed in claim 20, wherein pixel locations in a pixel block comprise indices into a table of values for said pixel, said table including pixel values corresponding to whole image content and audio information.

22. The method as claimed in claim 21, wherein said encoding step includes the step of:

specifying constraints related to visibility of said whole image, said constraints specified in accordance with prioritization of visual image content.

23. The method as claimed in claim 22, wherein said encoding step includes the steps of:

transforming audio data into video data; and

,

inserting said video data as video sub-features in the second set according to said constraints related to visibility of said whole image.

24. The method as claimed in claim 22, wherein said encoding step includes the steps of:

mapping video sub-features of said second image sub-feature set into indexes of units of audio information, said video sub-features being ordered in a predetermined fashion; and

,

inducing an order of units of audio information for providing a global audio information content.

25. The method as claimed in claim 24, wherein said mapping of video sub-features into indexes of units of audio information includes:

relating video-sub features to number values, an order of sub-features inducing an order of said number values; and

constructing a sequence of new number values based on sequences of prior ordered number values; and

,

entering said sequence of new number values as indexes to a table look-up device.

26. The method as claimed in claim 25, wherein said new number values are constructed according to algebraic formulae applied to sequences of prior number values.

27. The method as claimed in claim 25, wherein said relating step further comprises the steps of:

classifying sub-features according to physical quantities represented by said sub-features; and

,

assigning number values to said classes, said number values representing intensity of said classified physical quantity, wherein said classified physical quantities include one selected from the following;

color, waveform type, wavelength, frequency, thickness.

28. The method as claimed in claim 19, further comprising steps of:

scanning an image having audio information embedded in said second image sub-feature set; and

,

extracting said embedded audio information via a playback device.

29. The method as claimed in claim 28, wherein said extracting step comprises:

determining said second image sub-feature set areas of said image comprising said coded audio data, said video sub-features in said second sub-feature set being ordered in a predetermined fashion;

determining content of video sub-features in video data as indexes to units of audio information and inducing an order on the units of audio information; and

,

processing said units of audio information in the induced order to produce an audio message.

30. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for embedding audio information in image data corresponding to a whole image for display or print, said image data comprising pixels, the method steps comprising:

dividing each of one or more image pixels into first and second complementary sets of pixel components representing respective first and second image sub-feature sets;

encoding pixels of said first image sub-feature set with whole image content to be displayed or printed and pixels of said second image sub-feature set with coded audio information, said encoding of said audio data performed such that image sub-features in the second set satisfy constraints related to visibility of said whole image.