Interactive system for producing, storing and retrieving information correlated with a recording of an event

• US 5,564,005 A
• Filed: 10/15/1993
• Issued: 10/08/1996
• Est. Priority Date: 10/15/1993
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Poon, Alex D., Weber, Karon A., Moran, Thomas P.
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): Ho, Ruay Lian
• Application Number:
  US08/138,549
• Time in Patent Office: 1,089 Days
• Field of Search: 395/161, 395/132, 395/100, 395/500, 364/237.1, 364/413.29, 364/926.1, 358/461, 360/22, 382/13, 382/43
• US Class Current: 715/863
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An interactive processor-controlled system for entering and storing user information about an event that is being concurrently recorded in real time correlates the user information to user-requested addresses, such as time values stored with the recorded signals. The system includes a recording device for producing signals indicating recorded event data that includes periodic addresses, each of which indicates a portion of the recorded signals. An address source, such as a clock, is connected for providing address values to both the processor and the recording device. Using a pen-based user interface in one embodiment of the system, the system user specifically requests an address value in order to correlate user information to the signals being concurrently recorded. In response to a request for an address, the system establishes a spatial region on the display, called a time zone, into which the system user may enter information, in the form of handwritten strokes, about the event. All information entered into a time zone is stored and correlated with the address value represented by the time zone, in effect creating a log of the recorded event that may be used to access the portions of the recorded signals. While time zones are created by the user in sequential time order, information can be entered in to time zones in any sequence, thereby facilitating editing and annotation of previously-entered information. The ability to correlate the recorded signals to keywords designated in the user-supplied information is also provided.

Claims

1. A processor-controlled system for correlating event data being recorded in real time to event information entered into the system by a system user;

the system including

a user input device for producing signals indicating actions of a system user;

a display having a display area for presenting images;

a processor connected for receiving the signals from the user input device, and connected for providing images to the display;

a recording device in communication with a storage medium for producing a plurality of signals indicating recorded event data about an event;

the recording device currently producing the plurality of signals indicating the recorded event data and storing the signals on the storage medium;

an address source connected for simultaneously providing address data items to the processor of the system and to the recording device;

memory for storing data;

the data stored in the memory including instruction data indicating instructions the processor executes;

the processor being further connected for accessing the data stored in the memory;

the processor, in executing the instructions, responding to a first signal from the user input device indicating a request from the system user to obtain a user-requested address value from the address source by

accessing the address source and obtaining an address data item indicating the user-requested address value;

the address source simultaneously providing the user-requested address value to the recording device for storing with the recorded event data;

presenting a first image in the display area;

the first image including an address zone display feature positioned at an address marking location in the display area so that the address zone display feature is perceived by the system user as marking a spatial region in the display area;

the first image further including display features representing the user-requested address value positioned in the spatial region so that the user-requested address value is perceived as being associated with the spatial region; and

producing an address zone data structure for storing in the memory of the system and storing the address data item and the address marking location therein;

the address zone data structure being represented by the spatial region in the display area and being accessible by the processor using either the address marking location indicating the spatial region or the address data item indicating the user-requested address value;

the recording device, in response to receiving a recording address value from the address source, storing a plurality of signals indicating a first portion of recorded event data on the storage medium such that the recording address value provided by the address source indicates the first portion of the recorded event data stored on the storage medium; and

the processor, further in executing the instructions, responding to a second signal from the user input device indicating user-produced event data entered by the system user in a spatial region by

presenting a second image in the spatial region;

the second image including display features representing the user-produced event data entered by the system user; and

storing the user-produced event data in the address zone data structure represented by the spatial region such that all of the user-produced event data entered by the system user within the spatial region marked in the display area is accessible by the processor in the address zone data structure using the user-requested address value indicated by the address data item obtained when the spatial region was first produced;

the user-requested event data indicated by the user-requested address value being correlated with the portion of the recorded event data indicated by the recording address value simultaneously provided to the recording device by the address source.

2. The system of claim 1 further including an output device suitable for presentation of the plurality of signals indicating the recorded event data stored on the storage medium;

the output device being connected for receiving operational signals from the processor for operating the output device to present the recorded event data to the system user; and

wherein the processor, further in executing the instructions, when recording of the event has terminated, responds to a third signal from the user input device indicating a selected search address value and a retrieval request by the system user to obtain the recorded event data stored on the storage medium indicated by the search address value;

the selected search address value being selected by the system user from user-requested address values associated with user-produced event data entered in spatial regions in the display area;

the retrieval request causing the processor to respond by

operating the output device to obtain from the storage medium and present the plurality of signals indicating the recorded event data indicated by a recording address value nearest in value to the selected search address value;

the recorded event data being correlated with the user-produced event data entered into the spatial region associated with the selected search address value.

3. The method of claim 1 wherein the address source is a clock source simultaneously providing time data items to the processor of the system indicating the user-requested address value and to the recording device recording the event indicating the recording time value.

4. The system of claim 1 wherein the first signal received from the user is received from a stroke inputting device connected for interacting with the display to produce a first stroke as the signal indicating the request of the system user.

5. The system of claim 1 wherein the second signal received from the user is received from a stroke inputting device connected for interacting with the display to produce a stroke as signal;

the user-produced event data being a stroke entered on the surface of the display by the system user using the stroke inputting device.

6. The system of claim 5 wherein the processor, when storing the user-produced event data in the address zone data structure, stores signals indicating the stroke entered on the surface of the display by the system user using the stroke inputting device.

7. The system of claim 1 wherein the user input device for providing the signals indicating the actions of the system user is a stylus device connected for interacting with the display to produce the signals in the form of a handwritten stroke.

8. The system of claim 1 wherein the processor, further in executing the instructions, responding to a third signal from the user input device indicating a data designation action by the system user designating a portion of the user-produced event data for use as an information designator;

the portion hereafter referred to as the information designator;

the third signal indicating the system user'"'"'s action of creating an information designator;

by

presenting a third image in the display area;

the third image including a designator display object positioned in a designator location in the display area;

the designator display object including display features representing the information designator; and

producing an information designator data structure and storing therein the information designator and the designator location of the designator display object in;

the designator location indicating the information designator such that the information designator data structure is accessible by the processor when the system user subsequently selects the designator display object in the display area;

the system user having the capability of designating any user-produced event data entered in a spatial region as an information designator.

9. The system of claim 8 wherein the processor, further in executing the instructions, responds to a fourth signal from the user input device indicating the system user'"'"'s action of associating a selected information designator with a target spatial region;

the fourth signal including a selection action selecting a designator display object as a selected information designator and a designator target action indicating a target spatial region in the display area;

the processor responding by

presenting a fourth image in the display area;

the fourth image including display features representing the selected information designator and being presented in the target spatial region in the display area; and

modifying the address zone data structure represented by the target spatial region to include the selected information designator such that the user-produced event data and the user-requested address value stored in the address zone data structure of the target spatial region are accessible by the processor and retrievable by the system user using the selected information designator;

the selected information designator in the address zone data structure of the target spatial region providing access to recorded event data at the recording address value correlated with the user-requested address value of the target spatial region.

10. The system of claim 8 wherein the processor, in executing the instructions, further responds to the third signal from the user input device indicating the data designation action by the system user by assigning a unique information designator identifier to the information designator, and by storing the unique information designator identifier in the information designator data structure such that a selected information designator may be retrieved from an information designator data structure using the designator location thereof or using the unique information designator identifier thereof;

and

wherein the processor, further in executing the instructions, responding to a fourth signal from the user input device indicating the system user'"'"'s action of associating a selected information designator with a target spatial region;

the fourth signal including a selection action selecting a unique information designator identifier as a selected information designator identifier and a designator target action indicating a target spatial region in the display area;

the processor responding by

presenting a fourth image in the display area;

the fourth image including display features representing the selected information designator identifier and being presented in the target spatial region in the display area; and

modifying the address zone data structure represented by the target spatial region to include the selected information designator identifier such that the user-produced event data and the user-requested address value stored in the address zone data structure of the target spatial region are accessible by the processor and retrievable by the system user using the selected information designator identifier;

the selected information designator identifier in the address zone data structure of the target spatial region providing access to recorded event data at the recording address value correlated with the user-requested address value of the target spatial region.

11. The system of claim 10 wherein the processor, further in responding to the fourth signal, modifies the information designator data structure of the selected information designator identifier to indicate the user-requested address value stored in the address zone data structure of the target spatial region such that, when the system user associates a selected information designator identifier with a respective spatial region, all user-requested address values of all spatial regions having the selected information designator associated therewith are accessible to the processor and retrievable by the system user;

the processor using user-requested address values indicated by an information designator data structure to access recording address values matching the user-requested address values to present recorded event data correlated therewith.

12. The system of claim 11 further including an output device suitable for presentation of the plurality of signals indicating the recorded event data stored on the storage medium;

the output device being connected for receiving operational signals from the processor for operating the output device to present to the system user the recorded event data stored on the storage medium; and

wherein the processor, further in executing the instructions, when recording of the event has terminated, responding to a fifth signal from the user input device indicating the system user'"'"'s action to retrieve recorded event signals using an unique information designator identifier, the fifth signal including a selection action selecting the display features representing a selected information designator identifier;

the processor responding to the fifth signal by

obtaining a user-requested address value from the information designator data structure using the selected information designator identifier; and

operating the output device to obtain from the storage medium and present the plurality of signals indicating the recorded event data indicated by a recording address value nearest in value to the user-requested address value;

the recorded event data being automatically correlated with a respective information designator associated with a spatial region when the system user associates a selected information designator identifier indicating the respective information designator with the spatial region.

13. A processor-controlled machine for synchronizing recorded data to information entered into the machine by a machine user;

the machine including

a user input device for producing signals indicating actions of a machine user;

a display having a display area for presenting images;

a processor connected for receiving the signals from the user input device, and connected for providing images to the display;

a storage medium access device in communication with a storage medium adapted for storing thereon a plurality of signals indicating recorded data;

the recorded data being stored on the storage medium with a plurality of time data items indicating recording time values;

each recording time value being stored such that the recording time value indicates a portion of the recorded data;

a clock source connected for providing time data items to the processor;

the clock source being synchronized with the storage medium access device such that a time data item provided by the clock source indicates a time value equal to a recorded time value stored with the recorded data on the storage medium when the storage medium access device is accessing the recorded data; and

memory for storing data;

the data stored in the memory including instruction data indicating instructions the processor executes;

the processor being further connected for accessing the data stored in the memory;

the processor, in executing the instructions, responding to a first signal from the user input device indicating a request from the machine user to obtain a user-requested time value from the clock source by

accessing the clock source and obtaining a time data item indicating the user-requested time value;

presenting a first image in the display area;

the first image including a time zone display feature positioned at a time marking location in the display area such that the time zone display feature is perceived by the machine user as marking a spatial region, referred to as a time zone, in the display area; and

producing a time zone data structure for storing in the memory of the machine and storing the time data item and the time marking location therein;

the time zone data structure being represented by the time zone in the display area and being accessible by the processor using either the time marking location indicating the time zone or the time data item indicating the user-requested time value;

the storage medium access device accessing the storage medium at a recorded time value equal to the user-requested time value when the processor obtains the user-requested time value;

the processor, further in executing the instructions, responding to a second signal from the user input device indicating user-produced data entered by the machine user in a time zone by

presenting a second image in the time zone;

the second image including display features representing the user-produced data entered by the machine user; and

storing the user-produced data in the time zone data structure represented by the time zone such that all of the user-produced data entered by the machine user within the time zone marked in the display area is accessible by the processor in the time zone data structure using the user-requested time value obtained when the time zone data structure was first produced;

the user-produced data stored in a time zone data structure being synchronized with the portion of the recorded data indicated by the recording time value being accessed by the storage medium access device when the user-requested time value was first obtained and the time zone data structure was first produced.

14. The processor-controlled machine of claim 13 wherein the processor, further in executing instructions responding to the first signal indicating the request from the machine user, causes the first image to further include display features representing the user-requested time value positioned in the time zone so that the user-requested time value is perceived by the machine user as being associated with the time zone.

15. The processor-controlled machine of claim 13 wherein

the storage medium access device is a signal presentation device adapted for presenting the plurality of signals indicating the recorded data stored on the storage medium in a manner perceivable by the machine user;

the recorded data having been stored on the storage medium prior to the machine user first requesting a user-requested time value, and being referred to as previously recorded data;

the signal presentation device currently presenting the previously recorded data;

wherein the time data items provided to the processor by the clock source are recording time values previously stored on the storage medium with the previously recorded data being presented to the machine user by the signal presentation device; and

wherein the user-produced data stored in a time zone data structure is synchronized with a portion of the previously recorded data indicated by the recording time value being accessed by the signal presentation device when the user-requested time value was first obtained and the time zone data structure was first produced.

16. The machine of claim 15 wherein the signal presentation device is adapted to presenting image signals;

wherein the plurality of signals indicating the previously recorded data stored on the storage medium is a plurality of previously recorded images being presented on the signal presentation device and observed by the machine user;

the user-produced data entered by the machine user and stored in the time zone data structure being synchronized with a portion of previously recorded image data.

17. The machine of claim 16 wherein the previously recorded image data represent video images.

18. The machine of claim 16 wherein the signal recording device is adapted to recording image signals;

wherein the plurality of signals indicating the recorded data stored on the storage medium is a plurality of recorded images being recorded on the signal recording device;

the user-produced data entered by the machine user and stored in the time zone data structure being synchronized with a portion of recorded image data.

19. The machine of claim 18 wherein the recorded image data represent video images.

20. The machine of claim 15 wherein the signal presentation device is adapted to presenting audio signals;

wherein the plurality of signals indicating the previously recorded data stored on the storage medium is a plurality of previously recorded sounds being presented on the signal presentation device and heard by the machine user;

the user-produced data entered by the machine user and stored in the time zone data structure being synchronized with a portion of previously recorded audio data.

21. The processor-controlled machine of claim 13 wherein

the storage medium access device is a signal recording device adapted for recording the plurality of signals indicating the recorded data stored on the storage medium;

the signal recording device currently recording the recorded data;

the recorded data being stored on the storage medium as the machine user requests user-requested time values and as time zone data structures are produced;

wherein the time data items provided to the processor by the clock source are recording time values indicated by the signal recording device and being stored on the storage medium with the recorded data being recorded by the signal recording device; and

wherein the user-produced data stored in a time zone data structure is being synchronized with a portion of recorded data indicated by the recording time value being recorded with the recorded data by the signal recording device at the time a user-requested time value is obtained and the time zone data structure is first produced.

22. The machine of claim 21 wherein the signal recording device is adapted to recording audio signals;

wherein the plurality of signals indicating the recorded data stored on the storage medium is a plurality of recorded sounds being recorded on the signal recording device;

the user-produced data entered by the machine user and stored in the time zone data structure being synchronized with a portion of recorded audio data.

23. The machine of claim 13 wherein the user input device for providing the signals indicating the actions of the machine user is a stylus device connected for interacting with the display to produce the signals in the form of a handwritten stroke.

24. The machine of claim 23 wherein the processor, in executing the instructions for storing the user-produced data in the time zone data structure, stores signals indicating the stroke entered on the surface of the display by the machine user using the stylus device.

25. The machine of claim 24 wherein the signals indicating the stroke entered on the surface of the display by the machine user using the stylus device and stored as the user-produced data indicate image definition data defining the display features representing the user-produced data entered by the machine user and included in the second image.