MEDIA CAPTURE AND PROCESS SYSTEM

• US 20180182379A1
• Filed: 12/22/2016
• Published: 06/28/2018
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: MONTANTES, James
• Granted Patent: US 10,276,155 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H03G 5/165 Equalizers; Volume or gain control in limited frequency bands
• CPC Class Codes: H04R 1/326 for microphones H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/04 for correcting frequency response

Abstract

A user media device may include a microphone array and a communication interface. The microphone array may include an omnidirectional microphone and a directional microphone. The microphone array may be selectively switchable. The communication interface may communicatively couple the user media device with a computer and may transmit audio captured by the microphone array to the computer for transfer to a remote service. The remote service may generate text of the processed audio via natural language processing. The remote service may further perform semantic reasoning of the processed audio via a semantic reasoning engine. The remote service may also generate content based at least in part on the semantic reasoning performed on the processed audio. The curated content may include a report having results of the semantic reasoning organized to demonstrate the results in a meaningful way with respect to the processed audio.

Claims

1. A user media device comprising:

a microphone array including an omnidirectional microphone and a directional microphone, the microphone array being selectively switchable; and

a communication interface to communicatively couple the user media device with a computer and to transmit audio captured by the microphone array to the computer to transfer to a remote service for;

generating text of the processed audio via natural language processing;

performing semantic reasoning of the processed audio via a semantic reasoning engine; and

generating curated content based at least in part on the semantic reasoning performed on the processed audio, wherein the curated content includes a report having results of the semantic reasoning organized to demonstrate the results in a meaningful way with respect to the processed audio.

2. The user media device of claim 1, wherein the microphone array includes a second omnidirectional microphone and a second directional microphone.

3. The user media device of claim 1, wherein audio captured by the omnidirectional microphone is employed to perform noise cancelation relative to the audio captured by the directional microphone.

4. The user media device of claim 1, further comprising a camera.

5. The user media device of claim 1, wherein processed audio is received by the user media device from the computer via the communication interface and is generated by a speaker of the user media device.

6. The user media device of claim 1, wherein the user media device is configured to be worn on a head of a user.

7. The user media device of claim 1, wherein the microphone array is selectively switchable based at least in part on control commands received by the user media device from the computer.

8. A method of processing audio captured by a user media device, the method comprising:

capturing audio at a microphone array of the user media device;

transmitting the captured audio from the user media device to a computer communicatively coupled to the user media device;

processing the captured audio at the computer;

transmitting the processed audio from the computer to the user media device; and

generating the processed audio at a speaker of the user media device.

9. The method of claim 8, wherein processing the captured audio includes filtering the captured audio.

10. The method of claim 8, wherein processing the captured audio includes equalizing the captured audio.

11. The method of claim 8, further comprising:

capturing visual media at a camera of the user media device; and

transmitting the captured visual media from the user media device to the computer.

12. The method of claim 11, further comprising transmitting the captured visual media and one or more tags associated with the captured visual media from the computer to a remote service.

13. The method of claim 8, further comprising:

transmitting the processed audio from the computer to a remote service;

generating, at the remote service, text of the processed audio via natural language processing;

performing, at the remote service, semantic reasoning of the processed audio via a semantic reasoning engine; and

generating, at the remote service, curated content based at least in part on the semantic reasoning performed on the processed audio.

14. A media capture and process system comprising:

a user media device including;

a microphone array; and

a communication interface for communicatively coupling the user media device with a computer and for transmitting audio captured by the microphone array to the computer; and

a remote service for receiving the captured audio from the computer and processing the captured audio, the remote service including;

a semantic reasoning engine for performing semantic analysis on the captured audio;

a data curation engine for curating data associated with the captured audio; and

an analytics engine for generating reports based on the captured audio.

15. The media capture and process system of claim 14, wherein the microphone array includes an omnidirectional microphone and a directional microphone, the microphone array being selectively switchable.

16. The media capture and process system of claim 14, wherein the remote service includes a natural language engine for converting the captured audio to text.

17. The media capture and process system of claim 14, wherein curating the results includes organizing results of the semantic analysis to demonstrate the results in a meaningful way with respect to the captured audio.

18. The media capture and process system of claim 14, wherein the reports are accessible via an application programming interface (API).

19. The media capture and process system of claim 14, wherein the analytics engine generates the reports based on the captured audio at least in part via statistical analysis of results of the semantic analysis.

20. The media capture and process system of claim 14, wherein the user media device further includes a camera for capturing visual media and the remote service includes a deep learning engine for performing, on the captured visual media, at least one of object detection, context analysis, and image processing.