Conference information processing apparatus, and conference information processing method and storage medium readable by computer

• US 7,809,792 B2
• Filed: 09/03/2004
• Issued: 10/05/2010
• Est. Priority Date: 03/22/2004
• Status: Expired due to Fees

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Miyazaki, Jun, Hirata, Kazutaka
• Primary Examiner(s): Vaughn, Jr.; William C
• Assistant Examiner(s): Gupta; Muktesh G
• Application Number:
  US10/933,228
• Publication Number:
  US 20050210105A1
• Time in Patent Office: 2,223 Days
• Field of Search: 709/205, 370/256, 715/716
• US Class Current: 709/205
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/329 on a disc [VTOC]
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A conference information processing apparatus includes an extracting unit that extracts partial conference audio and/or visual information from conference audio and/or visual information. The partial conference audio and/or visual information has one or more physical events of a conference participant. The apparatus also has a providing unit that provides an index for the partial conference audio and/or visual information in accordance with a functional action abstracted from the one or more physical events.

Claims

1. A conference information processing apparatus comprising:

an extracting unit which extracts partial conference audio and/or visual information from conference audio and/or visual information, the partial conference audio and/or visual information comprising at least one of audio data and video data of one or more physical actions performed by a conference participant in the conference audio and/or visual information, and which extracts at least one of audio and visual information of each physical action from the partial audio and/or visual information; and

a providing unit which generates an index relating the partial conference audio and/or visual data to the physical actions, the index comprising;

information relating the one or more physical actions with a time duration over which the one or more physical actions occur;

information relating one or more physical actions with one or more functional actions, each functional action abstracted from one more physical actions; and

information relating the partial conference audio and/or visual information with one or more functional actions, the partial conference audio and/or visual information abstracted from one or more functional actions; and

a display unit which displays a summary of the conference audio and/or visual information, the summary comprising;

the one or more physical actions based on the time duration over which the one or more physical actions occur;

the one or more functional actions hierarchically arranged from the one or more physical actions; and

the partial conference audio and/or visual data hierarchically arranged from one or more functional actions.

2. The conference information processing apparatus as claimed in claim 1, further comprising

a producing unit that produces a summarized conference video of the conference audio and/or visual information in accordance with the index provided by the providing unit.

3. The conference information processing apparatus as claimed in claim 2, further comprising

a recording unit that records the conference video summarized by the producing unit.

4. The conference information processing apparatus as claimed in claim 2, further comprising

a display unit that displays the conference video summarized by the producing unit.

5. The conference information processing apparatus as claimed in claim 1, further comprising

a synchronization unit that synchronizes conference videos with each other, the conference videos being captured in the same period of time.

6. The conference information processing apparatus as claimed in claim 1, further comprising

a graphical user interface that displays the index provided by the providing unit.

7. The conference information processing apparatus as claimed in claim 1, wherein the providing unit calculates the time duration of the functional action from the logical sum of the time durations of the one or more physical actions.

8. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least a gesture made by the conference participant, the gesture being contained in the partial conference audio and/or visual information.

9. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least movements of the mouse of the conference participant, the movements of the mouse being contained in the partial conference audio and/or visual information.

10. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least movements of the eyes of the conference participant, the movements of the eyes being contained in the partial conference audio and/or visual information.

11. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least movements of the head of the conference participant, the movements of the head being contained in the partial conference audio and/or visual information.

12. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least a writing action of the conference participant, the writing action being contained in the partial conference audio and/or visual information.

13. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least the action of the conference participant standing up from his/her seat, the action being contained in the partial conference audio and/or visual information.

14. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least the action of the conference participant typing on a predetermined input device, the action being contained in the partial conference audio and/or visual information.

15. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least a facial expression of the conference participant, the facial expression being contained in the partial conference audio and/or visual information.

16. The conference information processing apparatus as claimed in claim 1, wherein the providing unit identifies the functional action by at least voice data of the conference participant, the voice data being contained in the partial conference audio and/or visual information.

17. The conference information processing apparatus as claimed in claim 2, wherein the producing unit produces conference videos for reviewing, the conference videos including active functional actions that are speaking, questioning, speech-type public information space using, question-type public information space using, and non-speech-type public information space using, and a functional action of speech-type thinking.

18. The conference information processing apparatus as claimed in claim 2, wherein the producing unit produces conference videos for public viewing, the conference videos including functional actions that are speaking, questioning, speech-type thinking, and speech-type public information space using.

19. The conference information processing apparatus as claimed in claim 2, wherein the producing unit produces conference videos for executives, the conference videos including functional actions that are speaking, speech-type thinking, and speech-type public information space using.

20. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface hierarchically displays the physical events and the functional actions abstracted from the physical actions.

21. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface displays scenes hierarchically with respect to the functional actions, each of the scenes being abstracted from one or more of the functional actions.

22. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface represents the index in the form of a time line or a table.

23. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface represents the functional actions in colors.

24. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface emphatically displays a change of audio speakers, a change of physical speakers, or a change of audio and physical speakers, by putting a color or a shape or giving a moving picture to the graphical representation of each corresponding one of the functional actions.

25. The conference information processing apparatus as claimed in claim 6, wherein the graphical user interface displays the data as to index identifiers, indexing start times, indexing end times, functional action names, roles of the conference participants, intended directions of the actions.

26. The conference information processing apparatus as claimed in claim 2, wherein the producing unit produces each conference video by summarizing an image portion that does not exceed a predetermined time duration in the corresponding partial conference audio and/or visual information.

31. The conference information processing apparatus as claimed in claim 1, wherein the index is generated manually by a user.

32. The conference information processing apparatus as claimed in claim 1, wherein the one or more physical actions in the summary of the conference comprise a media layer,

wherein the one or more functional actions hierarchically arranged from the one or more physical actions in the summary of the conference comprise a functional action layer, the functional action layer hierarchically arranged in the summary at a level higher the media layer, and

wherein the partial conference audio and/or visual data hierarchically arranged from one or more functional actions comprise a scene layer, the scene layer hierarchically arranged in the summary at a level higher than the functional action layer.

27. A conference information processing method, comprising:

extracting partial conference audio and/or visual information from conference audio and/or visual information, the partial conference audio and/or visual information comprising at least one of audio data and video data of one or more physical actions performed by a conference participant in the conference audio and/or visual information;

extracting at least one of audio and visual information of each physical action from the partial audio and/or visual information;

generating an index relating the partial conference audio and/or visual data to the physical actions, the index comprising;

information relating the one or more physical actions with a time duration over which the one or more physical actions occur;

information relating one or more physical actions with one or more functional actions, each functional action abstracted from one or more physical actions; and

information relating the partial conference audio and/or visual information with one or more functional actions, the partial conference audio and/or visual information abstracted from one or more functional actions; and

displaying summary of the conference audio and/or visual information on a display, the summary comprising;

the one or more physical actions based on the time duration over which the one or more physical actions occur;

the one or more functional actions hierarchically arranged from the one or more of the physical actions; and

the partial conference audio and/or visual data hierarchically arranged from one or more functional actions.

28. The conference information processing method as claimed in claim 27, further comprising:

producing a summarized conference video of the conference audio and/or visual information in accordance with the index.

33. The conference information processing method as claimed in claim 27, wherein the one or more physical actions in the summary of the conference comprise a media layer,

wherein the one or more functional actions hierarchically arranged from the one or more physical actions in the summary of the conference comprise a functional action layer, the functional action layer hierarchically arranged in the summary at a level higher the media layer, and

wherein the partial conference audio and/or visual data hierarchically arranged from one or more functional actions comprise a scene layer, the scene layer hierarchically arranged in the summary at a level higher than the functional action layer.

29. A storage medium readable by a computer, the storage medium storing a program of instructions executable by the computer to perform a conference information processing method comprising:

extracting partial conference audio and/or visual information from conference audio and/or visual information, the partial conference audio and/or visual information comprising at least one of audio data and video data of one or more physical actions performed by a conference participant in the conference audio and/or visual information;

extracting at least one of audio and visual information of each physical action from the partial audio and/or visual information;

generating an index relating the partial conference audio and/or visual data to the physical actions, the index comprising;

information relating the one or more physical actions with a time duration over which the one or more physical actions occur;

information relating one or more physical actions with one or more functional actions, each functional action abstracted from one or more physical actions; and

information relating the partial conference audio and/or visual information with one or more functional actions, the partial conference audio and/or visual information abstracted from one or more functional actions; and

displaying a summary of the conference audio and/or visual information on a display, the summary comprising;

the one or more physical actions based on the time duration over which the one or more physical actions occur;

the one or more functional actions hierarchically arranged from the one or more physical actions; and

the partial conference audio and/or visual data hierarchically arranged from one or more functional actions.

30. The storage medium as claimed in claim 29, further causing the computer to execute instructions to

produce a summarized conference video of the conference audio and/or visual information in accordance with the index provided.

34. The storage medium as claimed in claim 29, wherein the one or more physical actions in the summary of the conference comprise a media layer,

wherein the one or more functional actions hierarchically arranged from the one or more physical actions in the summary of the conference comprise a functional action layer, the functional action layer hierarchically arranged at a level higher the media layer in the summary, and

wherein the partial conference audio and/or visual data hierarchically arranged from one or more functional actions comprise a scene layer, the scene layer hierarchically arranged at a level higher than the functional action layer in the summary.