Generating sound files and transcriptions for use in spreadsheet applications

• US 9,934,215 B2
• Filed: 06/30/2016
• Issued: 04/03/2018
• Est. Priority Date: 11/02/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Canton, Christian, Campbell, John, Radakovitz, Samuel C., Fichtner, Matthew Hart, Otero, Carlos Augusto, Rampson, Benjamin Edward
• Primary Examiner(s): RIES, LAURIE ANNE
• Application Number:
  US15/199,968
• Publication Number:
  US 20170124045A1
• Time in Patent Office: 642 Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06F 40/197 Version control for software G06F8/71
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/26 : Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G11B 27/00 : Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel

Abstract

Systems for associating audio files with cells of a spreadsheet are provided. Both audio files and data may be associated with a single cell of the spreadsheet. An audio file may be recorded, retrieved from storage, or converted from a document (e.g., using text-to-speech technology) for association with a spreadsheet. Upon association, audio parameters may be viewed and/or manipulated by a user, providing audio processing functionality within a spreadsheet. Controls may be provided for listening to the audio file and/or playing the audio file in response to spreadsheet data satisfying a condition. Text transcriptions (e.g., speech-to-text) of audio files may be inserted into the spreadsheet. For low vision users, audio transcriptions (e.g., text-to-speech) of data may be generated and “played” for the user. Spreadsheet operations (e.g., sort and/or filter operations) may also be performed on a range of cells based on audio parameters of associated audio files.

Claims

1. A system comprising:

at least one processing unit; and

at least one memory storing computer executable instructions that, when executed by the at least one processing unit, cause the system to perform a method, the method comprising;

receiving a selection of at least one cell of a spreadsheet;

receiving a selection of a document;

transcribing the document;

creating at least one audio file;

creating a compound data type including the at least one audio file and the document and

associating the compound data type with the at least one cell of a spreadsheet.

2. The system of claim 1, wherein the document comprises one of:

textual content; and

a musical score.

3. The system of claim 2, wherein the document comprises textual content, the method further comprising:

converting text to speech to create the at least one audio file.

4. The system of claim 2, wherein the document comprises a musical score, the method further comprising:

converting the musical score to music to create the at least one audio file.

5. A method of associating an audio file with at least one cell of a spreadsheet, comprising:

receiving a selection of at least one cell of a spreadsheet;

recording sound;

creating at least one audio file based on the recorded sound;

transcribing the at least one audio file into text;

creating a compound data type including the at least one audio file and the text transcription; and

associating the compound data type with the at least one cell of the spreadsheet.

6. The method of claim 5, wherein the sound is speech.

7. The method of claim 6, the method further comprising:

transcribing the speech into text.

8. The method of claim 6, wherein the method further comprising:

transcribing the speech into text; and

associating the text transcription as a comment bubble with the at least one cell.

9. The method of claim 5, wherein the sound is recorded using a microphone on a mobile device.

10. The method of claim 5, further comprising:

referencing the at least one audio file in a function associated with the spreadsheet;

determining that a condition of the function is satisfied; and

playing the at least one audio file.

11. The method of claim 10, wherein playing the at least one audio file comprises one of:

playing the at least one audio file from the at least one cell in the spreadsheet;

sending the at least one audio file in a message to a user; and

sending the at least one audio file as a voicemail to a user.

12. A method of associating a text transcription of an audio file with a spreadsheet, comprising:

receiving at least one audio file;

associating the at least one audio file with a cell of the spreadsheet;

transcribing the at least one audio file into text;

creating a compound data type including the at least one audio file and the text transcription; and

associating the compound data type with the cell of the spreadsheet.

13. The method of claim 12, wherein the at least one audio file is received by one of:

recording the at least one audio file;

pasting the at least one audio file into the cell;

dropping the at least one audio file into the cell;

retrieving the at least one audio file from storage;

converting a range of values in the spreadsheet into the at least one audio file;

converting a document into the at least one audio file; and

converting a musical score into the at least one audio file.

14. The method of claim 12, further comprising:

referencing the at least one audio file in a function associated with the spreadsheet;

determining that a condition of the function is satisfied; and

providing the text transcription in a popup bubble associated with the at least one cell.

15. The method of claim 12, further comprising:

identifying a plurality of tracks for the at least one audio file;

splicing the audio file into the plurality of tracks; and

performing audio processing on one or more tracks of the plurality of tracks.

16. A computer-readable storage medium storing computer executable instructions that, when executed by at least one processing unit, cause a computing device to:

receive a selection of a range of cells within a spreadsheet, wherein each cell of the range of cells comprises a value;

map each value for each cell to an audio pitch;

create an audio file based on each mapped audio pitch;

creating a compound data type including the audio file and the value for each cell;

associate the compound data type with the range of cells.

17. The computer-readable storage medium of claim 16, wherein the range of cells is selected automatically.

18. The computer-readable storage medium of claim 16, wherein the audio file is played by a low-vision user to identify the values associated with the range of cells.

19. The computer-readable storage medium of claim 18, wherein the sound is played using a speaker on a mobile device.

20. The computer-readable storage medium of claim 16, wherein an audio file is created for each range of cells having values within the spreadsheet.