Methods and Systems for Audiovisual Communication

• US 20160321506A1
• Filed: 07/26/2015
• Published: 11/03/2016
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: KAMI Vision Incorporated
• Original Assignee: ANTS TECHNOLOGY (HK) LIMITED
• Inventors: Zhao, Lili, Fridental, Ron, Blayvas, Ilya
• Granted Patent: US 10,565,455 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04M 2201/50 Telephonic communication in combination with video communication
• CPC Class Codes: H04M 2242/04 for emergency applications
• CPC Class Codes: H04M 2242/30 Determination of the location of a subscriber
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Disclosed are systems and devices for facilitating audiovisual communication. According to embodiments, there may be provided a communication device including or otherwise functionally associated with a video camera to acquire visual information from a scene and to convert the visual information into a video stream. A video processing circuit may (a) receive the video stream, (b) extract user related features from the video stream; and (c) detect and characterize one or more user actions. A controller to receive the video processing circuit output and responsive to an indication of one or more detected events within the video stream to trigger a communication session between said communication device and an addressee device, wherein different addressee devices may be associated with different detected actions.

Claims

1. A communication device comprising:

a video camera to acquire visual information from a scene and to convert the visual information into a video stream;

a video processing circuit to;

(a) receive the video stream, (b) extract user related features from the video stream; and

(c) detect and characterize one or more user actions; and

a controller to receive one or more outputs of the video processing circuit and, responsive to an indication of one or more detected events within the video stream, trigger a communication session between said communication device and an addressee device, wherein different addressee devices are associated with different detected actions.

2. The device according to claim 1, further comprising:

a microphone; and

an audio processing circuit for characterizing acoustic information collected by said microphone;

wherein said controller selects an addressee device at least partially based on an output of the audio processing circuit.

3. The device according to claim 2, wherein said audio processing circuit or said video processing circuit include user identification functionality to identify specific humans.

4. The device according to claim 2, further comprising a user specific addressee contacts data storage to contain user specific records, which records correlate specific detected actions or specific detected utterances of a specific user with identifiers of one or more specific addressee devices.

5. The device according to claim 4, wherein said contacts data storage includes multiple device identifiers for a given addressee and an indication of a preferred order of devices to use in attempting a communication session with the given addressee.

6. The device according to claim 5, wherein said controller, upon receiving an indication of a user action or user utterances associated with the given addressee, attempts a communication session with the given addressee in accordance with the preferred order of addressee devices.

7. The device according to claim 6, wherein, if a communication attempt by said controller with a first device in the order fails, the controller attempts a communication session with a second addressee device in the order.

8. The device according to claim 7, wherein the first addressee device is of a different type from and utilizes different communication channels than the second addressee device.

9. The device according to claim 1, further comprising one or more three dimensional sensors, wherein an output of the one or more three dimensional sensors is used by said device to:

(a) extract user related features from the video stream; and

(b) detect and characterize one or more user actions.

10. The device according to claim 1, further comprising a display and an interface generator adapted to render control elements on said display, and wherein said controller is adapted to use outputs of the video processing circuit to allow a user to interact with rendered control elements using physical movements and/or gestures.

11. A communication device comprising:

an audio-video acquisition assembly including;

(a) a camera to acquire visual information from a scene and to convert the visual information into a video stream, and (b) a microphone to acquire acoustic signals from the scene and to convert the acoustic signals into an audio stream;

audio-video processing circuits to;

(a) extract user related features from the video stream;

(b) detect and characterize one or more user actions; and

(c) characterize information within the audio stream; and

a controller to receive outputs of said audio-video processing circuits and, responsive to a detection of an event in the received outputs, trigger a communication session between said communication device and an addressee device, wherein different addressee devices are associated with different combinations of detected actions and utterances, such that uttered subjective addressee designations are resolved by said device prior to communication session initiation.

12. The device according to claim 11, wherein a subjective addressee designation by a user is resolved using a user specific contacts data storage holding user specific records, which records correlate detected actions or utterances of a specific user with an identifier of a specific addressee device.

13. The device according to claim 11, further comprising one or more three dimensional sensors to generate signals indicative of user movements or gestures.

14. The device according to claim 13, wherein output from said one or more three dimensional sensors is processed to detect an action of a user or an event occurring to a user.

15. A communication device comprising:

an audio-video acquisition assembly including;

(a) a camera to acquire visual information from a scene and to convert the visual information into a video stream, and (b) a microphone to acquire acoustic signals from the scene and to convert the acoustic signals into an audio stream;

a data buffer to store at least a portion of the audio or video streams;

audio-video processing circuits to;

(a) extract user related features from the video stream;

(b) detect and characterize one or more user actions within the video stream; and

(c) characterize information within the audio stream;

a controller to receive outputs of said audio-video processing circuits and, responsive to an identification of a an event of a specific event type, trigger a communication session between said communication device and an addressee device, wherein the communication session includes an auto generated audio-video synapses relating to the event and based on data stored in the data buffer.

16. The device according to claim 15, further comprising a detected event specific contacts data storage, containing event specific records, which event specific records correlate detected event types or detected utterances of a specific user with an identifier of a specific addressee device.

17. The device according to claim 16, wherein said triggering event type is selected from the group consisting of:

(a) a fall, (b) a stumble, (c) a distress gesture, (d) presence of an unknown person in the scene, (e) loud noise, (f) user speech at lowered volume and/or clarity, and (g) stalled or stopped breathing

18. The device according to claim 16, further comprising a content auto-generator and wherein said event specific contact data-storage records provide to said auto-generator instructions relating to auto-generating an audio-video synapses for at least one event type.

19. The device according to claim 18, wherein synapses auto-generation instructions for a specific detected event type include instructions to identify within the buffered audio or video streams specific portions including specific features, specific actions or specific events relevant to an understanding of the detected event type;

and to append the identified portions to the synapsis.

20. The device according to claim 15, wherein said device is adapted to concurrently transmit to the addressee device the auto-generated synapsis and a live audio-video feed of the scene.

21. The device according to claim 15, wherein said device establishes a bidirectional communication session with the addressee device.