Systems and methods of providing contextual features for digital communication

• US 20160191958A1
• Filed: 12/28/2015
• Published: 06/30/2016
• Est. Priority Date: 12/26/2014
• Status: Abandoned Application

• Current Assignee: Krush Technologies, LLC
• Original Assignee: Krush Technologies, LLC
• Inventors: Free, Linsey Ann, Riggins, Gary T., Nauseef, John P., Wire, Christopher S., Faust, Brian T., Farrell, Matthew J., Murray, Patrick M., Nesbitt, John C., Clinard, Dustin L.
• Application Number:
  US14/980,769
• Publication Number:
  US 20160191958A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image

Abstract

Embodiments disclosed herein may be directed to a video communication server for: receiving, using a communication unit comprised in at least one processing device, video content of a video communication connection between a first user of a first user device and a second user of a second user device; analyzing, using a graphical processing unit (GPU) comprised in the at least one processing device, the video content in real time; identifying, using a recognition unit comprised in the at least one processing device, at least one object of interest comprised in the video content; identifying, using a features unit comprised in the least one processing device, at least one contextual feature associated with the at least one identified object of interest; and presenting, using an input/output (I/O) device, the at least one contextual feature to at least one of the first user device and the second user device.

Claims

1. A video communication server comprising:

at least one memory comprising instructions; and

at least one processing device configured for executing the instructions, wherein the instructions cause the at least one processing device to perform the operations of;

receiving, using a communication unit comprised in the at least one processing device, video content of a video communication connection between a first user of a first user device and a second user of a second user device;

analyzing, using a graphical processing unit (GPU) comprised in the at least one processing device, the video content in real time;

identifying, using a recognition unit comprised in the at least one processing device, at least one object of interest comprised in the video content;

identifying, using a features unit comprised in the least one processing device, at least one contextual feature associated with the at least one identified object of interest; and

presenting, using an input/output (I/O) device, the at least one contextual feature to at least one of the first user device and the second user device.

2. The video communication server of claim 1, wherein the at least one object of interest comprises at least one of a facial feature, a facial gesture, a vocal inflection, a vocal pitch shift, a change in word delivery speed, a keyword, an ambient noise, an environment noise, a landmark, a structure, a physical object, and a detected motion.

3. The video communication server of claim 1, wherein identifying the at least one object of interest comprises:

identifying, using the recognition unit, a facial feature of the first user in the video content at a first time;

identifying, using the recognition unit, the facial feature of the first user in the video content at a second time; and

determining, using the recognition unit, movement of the facial feature from a first location at a first time to a second location at a second time,

wherein the determined movement of the facial feature comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

4. The video communication server of claim 1, wherein identifying the at least one object of interest comprises:

identifying, using the recognition unit, a first vocal pitch of the first user in the video content at a first time;

identifying, using the recognition unit, a second vocal pitch of the first user in the video content at a second time; and

determining, using the recognition unit, a change of vocal pitch of the first user,

wherein the determined change of vocal pitch comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

5. The video communication server of claim 1, wherein identifying the at least one object of interest comprises:

identifying, using the recognition unit, a landmark in the video content, wherein the landmark is associated with a geographic region;

identifying, using the recognition unit, a speaking accent of the first user in the video content, wherein the accent is associated with the geographic region; and

determining, using the recognition unit and based at least in part on the landmark and the accent, the first user device is located in the geographic region,

wherein the at least one contextual feature is associated with the geographic region.

6. The video communication server of claim 1, wherein presenting the at least one contextual feature to at least one of the first user device and the second user device comprises:

identifying, using the recognition unit, at least one reference point in the video content;

tracking, using the recognition unit, movement of the at least one reference point in the video content; and

overlaying, using the features unit, the at least one contextual feature onto the at least one reference point in the video content.

7. The video communication server of claim 1, wherein identifying the at least one object of interest comprises:

determining, using the GPU, a numerical value of at least one pixel associated with the at least one object of interest.

8. A non-transitory computer readable medium comprising code, wherein the code, when executed by at least one processing device of a video communication server, causes the at least one processing device to perform the operations of:

receiving, using a communication unit comprised in the at least one processing device, video content of a video communication connection between a first user of a first user device and a second user of a second user device;

analyzing, using a graphical processing unit (GPU) comprised in the at least one processing device, the video content in real time;

identifying, using a recognition unit comprised in the at least one processing device, at least one object of interest comprised in the video content;

identifying, using a features unit comprised in the least one processing device, at least one contextual feature associated with the at least one identified object of interest; and

presenting, using an input/output (I/O) device, the at least one contextual feature to at least one of the first user device and the second user device.

9. The non-transitory computer readable medium of claim 8, wherein the at least one object of interest comprises at least one of a facial feature, a facial gesture, a vocal inflection, a vocal pitch shift, a change in word delivery speed, a keyword, an ambient noise, an environment noise, a landmark, a structure, a physical object, and a detected motion.

10. The non-transitory computer readable medium of claim 8, wherein the non-transitory computer readable medium further comprises code that, when executed by the at least one processing device of the video communication server, causes the at least one processing device to perform the operations of:

identifying, using the recognition unit, a facial feature of the first user in the video content at a first time;

identifying, using the recognition unit, the facial feature of the first user in the video content at a second time; and

determining, using the recognition unit, movement of the facial feature from a first location at a first time to a second location at a second time,

wherein the determined movement of the facial feature comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

11. The non-transitory computer readable medium of claim 8, wherein the non-transitory computer readable medium further comprises code that, when executed by the at least one processing device of the video communication server, causes the at least one processing device to perform the operations of:

identifying, using the recognition unit, a first vocal pitch of the first user in the video content at a first time;

identifying, using the recognition unit, a second vocal pitch of the first user in the video content at a second time; and

determining, using the recognition unit, a change of vocal pitch of the first user,

wherein the determined change of vocal pitch comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

12. The non-transitory computer readable medium of claim 8, wherein the non-transitory computer readable medium further comprises code that, when executed by the at least one processing device of the video communication server, causes the at least one processing device to perform the operations of:

identifying, using the recognition unit, a landmark in the video content, wherein the landmark is associated with a geographic region;

identifying, using the recognition unit, a speaking accent of the first user in the video content, wherein the accent is associated with the geographic region; and

determining, using the recognition unit and based at least in part on the landmark and the accent, the first user device is located in the geographic region,

wherein the at least one contextual feature is associated with the geographic region.

13. The non-transitory computer readable medium of claim 8, wherein the non-transitory computer readable medium further comprises code that, when executed by the at least one processing device of the video communication server, causes the at least one processing device to perform the operations of:

identifying, using the recognition unit, at least one reference point in the video content;

tracking, using the recognition unit, movement of the at least one reference point in the video content; and

overlaying, using the features unit, the at least one contextual feature onto the at least one reference point in the video content.

14. The non-transitory computer readable medium of claim 8, wherein the non-transitory computer readable medium further comprises code that, when executed by the at least one processing device of the video communication server, causes the at least one processing device to perform the operations of:

determining, using the GPU, a numerical value of at least one pixel associated with a facial feature identified in the video content.

15. A method comprising:

receiving, using a communication unit comprised in at least one processing device, video content of a video communication connection between a first user of a first user device and a second user of a second user device;

analyzing, using a graphical processing unit (GPU) comprised in the at least one processing device, the video content in real time;

identifying, using a recognition unit comprised in the at least one processing device, at least one object of interest comprised in the video content;

identifying, using a features unit comprised in the least one processing device, at least one contextual feature associated with the at least one identified object of interest; and

presenting, using an input/output (I/O) device, the at least one contextual feature to at least one of the first user device and the second user device.

16. The method of claim 15, wherein the at least one object of interest comprises at least one of a facial feature, a facial gesture, a vocal inflection, a vocal pitch shift, a change in word delivery speed, a keyword, an ambient noise, an environment noise, a landmark, a structure, a physical object, and a detected motion.

17. The method of claim 15, wherein the method further comprises:

identifying, using the recognition unit, a facial feature of the first user in the video content at a first time;

identifying, using the recognition unit, the facial feature of the first user in the video content at a second time; and

determining, using the recognition unit, movement of the facial feature from a first location at a first time to a second location at a second time,

wherein the determined movement of the facial feature comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

18. The method of claim 15, wherein the method further comprises:

identifying, using the recognition unit, a first vocal pitch of the first user in the video content at a first time;

identifying, using the recognition unit, a second vocal pitch of the first user in the video content at a second time; and

determining, using the recognition unit, a change of vocal pitch of the first user, wherein the determined change of vocal pitch comprises a gesture associated with a predetermined emotion, and

wherein the at least one contextual feature is associated with the predetermined emotion.

19. The method of claim 15, wherein the method further comprises:

identifying, using the recognition unit, a landmark in the video content, wherein the landmark is associated with a geographic region;

identifying, using the recognition unit, a speaking accent of the first user in the video content, wherein the accent is associated with the geographic region; and

determining, using the recognition unit and based at least in part on the landmark and the accent, the first user device is located in the geographic region,

wherein the at least one contextual feature is associated with the geographic region.

20. The method of claim 15, wherein the method further comprises:

identifying, using the recognition unit, at least one reference point in the video content;

tracking, using the recognition unit, movement of the at least one reference point in the video content; and

overlaying, using the features unit, the at least one contextual feature onto the at least one reference point in the video content.