Device and method for automatic participant identification in a recorded multimedia stream

• US 8,390,669 B2
• Filed: 12/15/2009
• Issued: 03/05/2013
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Catchpole, Jason, Cockerton, Craig
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US12/638,635
• Publication Number:
  US 20100149305A1
• Time in Patent Office: 1,176 Days
• Field of Search: 348/14.08, 348/14.09, 348/14.16, 348/515, 382/173, 382/107, 704/231, 704/236, 704/256, 704/276, 709/204, 709/231, 710/8, 725/19, 725/22
• US Class Current: 348/14.08
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

The present disclosure discloses a method for identifying individuals in a multimedia stream originating from a video conferencing terminal or a Multipoint Control Unit, including executing a face detection process on the multimedia stream; defining subsets including facial images of one or more individuals, where the subsets are ranked according to a probability that their respective one or more individuals will appear in a video stream; comparing a detected face to the subsets in consecutive order starting with a most probable subset, until a match is found; and storing an identity of the detected face as searchable metadata in a content database in response to the detected face matching a facial image in one of the subsets.

Claims

1. A device comprising:

a receiver configured to receive a video conference format coded data stream;

a conversion unit configured to convert the video conference format coded data stream to a multimedia stream in a defined multimedia streaming format and store the multimedia stream and searchable metadata in a content database;

a user database including identities of known individuals stored with associated facial images; and

a face recognition unit configured to,

detect a face in a video stream included in the multimedia stream,

define subsets including facial images of one or more of the known individuals, where the subsets are ranked according to a probability that their respective one or more known individuals will appear in the video stream,

compare the detected face to the subsets in consecutive order starting with a most probable subset, until a match is found, and

store an identity of the detected face as searchable metadata in the content database in response to the detected face matching a facial image in one of the subsets.

2. The device according to claim 1, further comprising:

a ranking unit configured to rank the subsets based on protocol information extracted from a communication session and based on groups in the user database,

wherein the user database further includes one or more unique addresses of known terminals, the identities and unique addresses being associated with one or more of the groups, and at least one of the unique addresses being related to one or more of the known individuals.

3. The device according to claim 2, wherein the protocol information includes the one or more unique addresses of one or more terminals which are an origin of the video conference format coded data stream.

4. The device according to claim 1, wherein the face recognition unit is further configured to create a thumbnail image of the detected face and store the thumbnail image with the multimedia stream in the content database.

5. The device according to claim 1, further comprising a graphical user interface including a listing of available multimedia streams in the content database, the listing including a link to a stored multimedia stream and the searchable metadata associated with the multimedia stream, including the names of recognized participants.

6. The device according to claim 5, wherein the listing further includes one or more stored thumbnail images of individuals detected in each multimedia stream.

7. The device according to claim 1, wherein the video conference format coded data stream is a H.323, H.320, or Session Initiated Protocol coded data stream.

8. The device according to claim 2, wherein the face recognition unit is further configured to rank the subsets based on the one or more unique addresses according to a first subset including the known individuals related to the one or more unique addresses, a second subset including all known individuals associated with a first group also associated with the one or more unique addresses, and a third subset including all known individuals associated with a second group and associated with the one or more unique addresses.

9. The device according to claim 8,

wherein the first group is selected from at least one of a department, a geographical location, a building, an office, and a floor, and

wherein the second group is selected from at least one of a company, an organization, and a set of known individuals.

10. The device according to claim 8, further comprising:

a ranking unit ranking a fourth subset higher than the second subset, the fourth subset including all known individuals associated with a user histogram group and associated with the one or more unique addresses of the known terminals,

wherein the face recognition unit is further configured to generate a user histogram group for each terminal, wherein previous users of each terminal are associated with the user histogram group.

11. The device according to claim 8, further comprising:

a ranking unit configured to rank a fifth subset higher than the second subset, the fifth subset including facial images of known users in a list of probable users,

wherein the face recognition unit is further configured to generate the list of probable users of each of the known terminals at a given time, based on information from at least one of a scheduling server and a presence server.

12. A method for identifying individuals in a multimedia stream originating from a video conferencing terminal or a Multipoint Control Unit, the method comprising:

executing, at a device, a face detection process on the multimedia stream;

defining, at the device, subsets including facial images of one or more individuals, where the subsets are ranked according to a probability that their respective one or more individuals will appear in a video stream;

comparing, at the device, a detected face to the subsets in consecutive order starting with a most probable subset, until a match is found; and

storing, at the device, an identity of the detected face as searchable metadata in a content database in response to the detected face matching a facial image in one of the subsets.

13. The method according to claim 12, further comprising:

receiving a video conference format coded data stream;

storing, in a user database, one or more unique addresses of known terminals and identities of known individuals, wherein the identities of known individuals and unique addresses are associated with one or more groups, and wherein at least one of the unique addresses are related to one or more of the known individuals;

extracting protocol information from a communication session; and

ranking the subsets based on the protocol information and at least one of the groups in the user database.

14. The method according to claim 13, wherein the protocol information includes the one or more unique addresses of one or more terminals which are an origin of the video conference format coded data stream.

15. The method according to claim 12, further comprising:

creating a thumbnail image of the detected face; and

storing the thumbnail image with the multimedia stream in the content database.

16. The method according to claim 12, further comprising providing a graphical user interface including a listing of available multimedia streams in the content database, the listing including a link to a stored multimedia stream and the searchable metadata associated with the multimedia stream, including the names of recognized participants.

17. The method according to claim 16, wherein the listing further includes one or more stored thumbnail images of individuals detected in each multimedia stream.

18. The method according to claim 13, wherein the video conference format coded data stream is a H.323, H.320, or Session Initiated Protocol coded data stream.

19. The method according to claim 13, further comprising ranking the subsets according to a first subset including the known individuals related to the one or more unique addresses, a second subset including all known individuals associated with a first group also associated with the one or more unique addresses, and a third subset including all known individuals associated with a second group and associated with the one or more unique addresses.

20. The method according to claim 19,

wherein the first group is selected from at least one of a department, a geographical location, a building, an office, and a floor, and

wherein the second group is selected from at least one of a company, an organization, and a set of known individuals.

21. The method according to claim 19, further comprising:

generating a user histogram group for each terminal, wherein previous users of a terminal are associated with the user histogram group; and

ranking a fourth subset higher than the second subset, the fourth subset including all known individuals associated with the user histogram group and associated with the one or more unique addresses of the known terminals.

22. The method according to claim 19, further comprising:

generating a list of probable users of each of the known terminals at a given time, based on information from at least one of a scheduling server and a presence server; and

ranking a fifth subset higher than the second subset, the fifth subset including facial images of known users in the list of probable users.