Computer controlled display system using a graphical replay device to control playback of temporal data representing collaborative activities

• US 6,332,147 B1
• Filed: 11/03/1995
• Issued: 12/18/2001
• Est. Priority Date: 11/03/1995
• Status: Expired due to Term

• Current Assignee: Spatium Innovations LLC
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Chiu, Patrick, Moran, Thomas P., Kurtenbach, Gordon P., van Melle, William J.
• Primary Examiner(s): Hong, Stephen S.
• Application Number:
  US08/552,648
• Publication Number:
  US 20020002562A1
• Time in Patent Office: 2,237 Days
• Field of Search: 395/806, 395/807, 395/173, 395/174, 395/327, 395/328, 395/329, 395/330, 395/331, 395/334, 395/348, 395/615, 364/514.R, 364/514.A, 364/715.02, 348/13, 348/15, 345/302, 345/473, 345/474, 345/327, 345/328, 345/329, 345/330, 345/331, 345/334, 345/348, 707/104, 707/500.1, 707/512
• US Class Current: 715/203
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/489 using time information

Abstract

A system for controlling the playback of a recorded session utilizing a window acting as a player for playing back a timestream for a shared representation media, such as an electronic whiteboard. Timestream and event information for the session is captured by one or more recording devices. Playback of a session is performed under the control of a session access device which utilizes event information to create a user interface for controlling session replay. The user interface includes the window for playing back the timestream of a shared representation media. Each graphical object in the window is associated with one or more events (e.g. a creation event, an editing event, a move event or a deletion event). Selection of a graphical object enables playback time to be moved to the time for one of the associated events. Selection may also be made on an area of the window. This will enable the user to view all events related to graphical objects in the selected area (as some graphical objects may be erased or moved).

Claims

1. A computer controlled display system for controlling the replay of a session captured on a plurality of media capture devices, wherein at last one of said plurality of media capture devices is a shared representation media, said shared representation media for creating and displaying graphical objects responsive to a user input, said computer controlled display system coupled to one or more players for replaying said session, said computer controlled display system comprising:

display

session input means for receiving temporal data of said captured session, said temporal data comprising timestreams and events;

user input means for enabling a user to interact with said computer controlled display system;

synchronization means for synchronizing playback of said session by said one or more players;

processing circuitry for generating display information from a timestream representing operations performed on said shared representation media, said display information for display in a window on said display, said window operating as a player for replaying the temporal sequence of operations performed on said shared representation media, said replay causing one or more graphical objects created by a user of said shared representation media during said session capture to be displayed in said window, and said window further operating as a user interface for controlling the playback of said session;

window control circuitry for detecting a user interaction on at least one of said one or more graphical objects presented during replay in said window, and user interaction causing replay of the temporal sequence of operations on said shared representation media to move to a new point in said temporal sequence of operations.

2. The computer controlled display system as recited in claim 1 wherein said window operating as a player has an operating mode that recreates the interactions with the shared representation media in the same temporal sequence that they occurred.

3. The computer controlled display system as recited in claim 1 wherein said window operating as a player has an operating mode that starts with an end state of the graphical objects on the shared representation media and moves a cursor to point to a particular graphical object when an event associated with the particular graphical object occurs.

4. The computer controlled display system as recited in claim 3 wherein said visual representation of said cursor will change in accordance with an event occurring with respect to the particular graphical object pointed to.

5. The computer controlled display system as recited in claim 1 wherein one of said operations performed on said shared representation is an erase operation for erasing graphical objects and which are subsequently represented by a ghost graphical object.

6. The computer controlled display system as recited in claim 1 wherein said window control circuitry further comprises:

means for identifying a graphical object that a cursor is point to;

means for identifying events associated with said pointed to graphical object, said events having associated therewith a time said event occurred in said session;

means for determining which event to move playback to;

and wherein said playback control signal is one that moves playback to the time associated with said determined event.

7. The computer controlled display system as recited in claim 6 wherein said means for determining which event to move playback to is comprised of means for displaying a menu of all the events associated with said graphical object and means for detecting an event selected from said menu.

8. The computer controlled display system as received in claim 1 wherein said window control circuitry further comprises:

means for identifying a graphical object that a cursor is pointing to;

means for identifying a creating event for said graphical object; and

said playback control signal is one that moves playback to the time associated with said creation event.

9. In a computer controlled display system for controlling playback of a session, a method for changing the playback time of said session, said session captured on a plurality of media capture devices, wherein at least one of said plurality of media capture devices in a shared representation media, and shared representation media for creating and displaying graphical objects responsive to a user input, said method comprising the steps of:

a) analyzing a timestream of said shared representation media to identify a plurality of events, each of said plurality of events corresponding to an object created on the shared representation media during said session, each of said events having a corresponding a point in time in said session;

b) replaying said timestream of said shared representation media in a replay window on a display coupled to said computer controlled display system;

c) detecting that a user has selected a graphical object in said replay window presented during replay of said timestream of said shared representation media, said graphical object representing a marking made on said shared representation media during said session capture;

d) identifying events associated with said graphical object;

e) determining which of said events will be used to move playback time to; and

f) cuing players to move playback of said session to the time associated with said event determined in step e).

10. The method as recited in claim 9 wherein said step of replaying a representation of said shared representation media in a replay window on a display coupled to said computer controlled display system is further comprised of the steps of:

b1) displaying objects in said replay window representing an end state of the shared representative media at the end of said session; and

b2) moving a cursor to point to an object that is associated with an event during the playback of said session.

11. The method as recited in claim 10 further comprising the step of:

b3) changing the visual representation of said cursor to correspond to the type of event occurring on an object.

12. The method as recited in claim 9 wherein said step of replaying said timestream of said shared representation media in a replay window on a display coupled to said computer controlled display system is further comprised on the step of displaying graphical objects in said replay window in the temporal sequence that they were created.

13. The method as recited in claim 12 wherein said step of replaying said timestream of said shared representation media in a replay window on a display coupled to said computer controlled display system is further comprised of the step of displaying a ghost object at the spatial location where an object is deleted.

14. The method as recited in claim 9 wherein said step of determining which of said events will be used to move playback time to is further comprised of the steps of:

e1) displaying a list of events associated with said graphical object so that a user may select one of said events;

e2) detecting that a user has selected one of the events in said list of events; and

e3) providing said selected event as said determined event.