System and method for visualizing connected temporal and spatial information as an integrated visual representation on a user interface

• US 7,499,046 B1
• Filed: 03/14/2005
• Issued: 03/03/2009
• Est. Priority Date: 03/15/2003
• Status: Active Grant

• Current Assignee: Pen-Link Ltd.
• Original Assignee: Uncharted Software, Inc.
• Inventors: Wright, William, Kapler, Thomas
• Primary Examiner(s): Chauhan; Ulka
• Assistant Examiner(s): Hajnik; Daniel F
• Application Number:
  US11/078,330
• Time in Patent Office: 1,450 Days
• Field of Search: None
• US Class Current: 345/418
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline

Abstract

A system includes a visualization manager for assembling the group of data elements and for assigning a connection visual element in the visual representation between a first visual element representing a first data element of the group and a second visual element representing a second data element of the group. The system also has a spatial visualization component configured for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions. The reference surface is for relating the first visual element to a first location of interest in the spatial reference frame and for relating the second visual element to a second location of interest in the spatial reference frame. The system also has a temporal visualization component configured for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest. The temporal domain includes a first time track, such as a timeline, coupled to the first location of interest and a second time track coupled to the second location of interest, such that the first visual element is positioned on the first time track and the second visual element is positioned on the second time track. Each of the time tracks are configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface.

Claims

1. A method for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as a corresponding plurality of visual elements, at least two of the data elements of the group linked by at least one association, the method comprising the steps of:

generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating a first visual element of the plurality of visual elements representing a first data element of the group to a first location of interest in the spatial reference frame and relating a second visual element of the plurality of visual elements representing a second data element of the group to a second location of interest in the spatial reference frame, the first and second data elements being defined as positioned with respect to their respective location of interest for time defined in a temporal reference frame;

generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing the common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest;

linking the first data element to the second data element using the at least one association, the at least one association configured for representing a connection between the first data element and the second data element, the connection having at least one property in the spatial domain and including time as a temporal property;

assigning a connection visual element in the visual representation between the first visual element and the second visual element, the connection visual element for representing the at least one association; and

aggregating the first data element and the second data element into an aggregated element group according to a selected aggregation parameter, the aggregated element group configured for reducing the visual complexity of the visual elements for display in the visual representation by displaying the aggregated element group on a third time track and removing the display of the first and second time tracks from the visual representation

wherein the visual representation is configured for display on a user interface for subsequent interaction with user events wherein the user interface uses a processor and a memory.

2. The method of claim 1, wherein the visual representation of the group of data elements is selected from the group comprising;

a concurrent time and geographic context and a concurrent time and diagrammatic context.

3. The method of claim 2, wherein the diagrammatic context includes a plurality of nodes of a defined process, such that one of the nodes is positioned at the first location of interest and a second one of the nodes is positioned at the second location of interest.

4. The method of claim 2 further comprising the step of arranging the plurality of the visual elements along the time tracks according to the times at which the corresponding data elements occurred in the temporal reference frame, the type of the data elements selected from the group comprising:

entitles;

events;

locations; and

associations.

5. The method of claim 2, wherein the location in the temporal reference frame of each of the plurality of visual elements on the time tracks is proportional to the distance from the locations of interest associated with the reference surface, the type of the visual elements selected from the group comprising:

entitles;

events;

locations; and

associations.

6. The method of claim 5, wherein the time tracks are represented as timelines in the visual representation.

7. The method of claim 2, wherein types of the data elements are selected from the group comprising;

entity, location, association, and event.

8. The method of claim 7, wherein the event data element type represents an action taking place at a particular one of the locations of interest in the spatial reference frame and at a particular time in the temporal reference frame.

9. The method of claim 8, wherein the event data element type has data properties and display properties selected from the group comprising;

a short text label, description, location, start-time, end-time, general event type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, default and user-set color, reference document, and reference photograph.

10. The method of claim 7, wherein the entity data element type represents an actor involved in a selected event.

11. The method of claim 10, wherein the entity data element type has data properties and display properties selected from the group comprising;

short text label, description, general entity type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, default and user-set color, and reference document, and reference photograph.

12. The method of claim 7, wherein the location data element type represents a location within the spatial reference frame.

13. The method of claim 12, wherein the location data element type has data properties and display properties selected from the group comprising;

position coordinates, a label, description, color information, precision information, location type, non-geospatial flag and user comments, reference document, and reference photograph.

14. The method of claim 13, wherein the location data element type is selected from the group comprising;

a physical location on a geospatial map, a physical location as a node in a diagram, and a virtual location related to a geospatial map such that the virtual location is not assigned to a specific physical location on the geospatial map.

15. The method of claim 2, wherein the at least one association describes a pairing between two or more of the data elements for providing an information data object related to both of the two or more data elements.

16. The method of claim 15 wherein the connection visual element for the at least one association is a selected line.

17. The method of claim 15 wherein the information data object is selected from the group comprising:

a communication connection describing communication details transferred between the two or more data elements;

a financial transaction or other transaction; and

a relationship connection describing social details in common between the two or more data elements.

18. The method of claim 17, wherein the information data object is selected from the group comprising:

a direct connection; and

an indirect connection.

19. The method of claim 15, wherein the at least one association is predefined or is created due to user interaction with the visualization representation.

20. The method of claim 19, wherein the user interaction generates user events for modifying visual properties of the connection visual element

21. The method of claim 20 further comprising the step of applying a filtering function to the visual elements and the at least one related association to select a subgroup thereof, selection of the subgroup according to a method selected from the group comprising;

criteria matching, algorithmic methods, and manual selection.

23. The method of claim 4 wherein the first and second data elements represent entitles connected to the same event by the at least one association.

24. The method of claim 16, wherein the line type further includes a vector property for representing a quantity selected from the group comprising:

a direction of movement;

polarity; and

a sequence.

25. The method of claim 2 further comprising the step of dynamically adding an annotation to the visual representation by the user, the annotation selected from the group comprising a geometrical object and a text abject, the annotation having at least one property in the spatial domain and at least one property in the temporal domain.

26. The method of claim 25 further comprising the step of dynamically changing a visual property of the annotation based on a selected range in the spatial domain.

27. The method of claim 25 further comprising the step of dynamically changing a visual property of the annotation based on a selected range in the temporal domain.

28. The method of claim 25 wherein the geometrical object is selected from the group comprising:

a line; and

a closed shape.

29. The method according to claim 1, wherein the aggregation parameter is selected from the group comprising:

a time range in the temporal reference frame;

a spatial range in the spatial reference frame;

a location of interest;

a combination of locations of interest;

a hierarchy of locations of interest;

an event;

a combination of events;

an entity;

a combination of entities;

a level of detail for the visual complexity according to a scale of the visual representation;

an association; and

a combination of associations.

30. The method according to claim 29, wherein the method of selection of the aggregation parameter is selected from the group comprising:

predefined; and

dynamically specified by a user.

31. The method according to claim 29 further comprising the step of selecting the combination of locations according to a selected region in the spatial reference frame.

32. The method according to claim 31 wherein the selected region is selected from the group comprising:

a point; and

a set of points.

33. The method according to claim 1 further comprising the step of locating the third time track at a different location of interest calculated as a combination of the positions of the first and second locations of interest.

34. The method according to claim 1, wherein both the first data element and the third data element are selected from the first time track.

35. The method according to claim 34, wherein the method of selection of the aggregation parameter is selected from the group comprising:

predefined; and

dynamically specified by a user.

36. The method according to claim 1 further comprising the step of attaching a label to the aggregated element group displayed in the visual representation.

37. The method according to claim 1, wherein the aggregation parameter is specified using a predefined filter.

38. The method according to claim 1 further comprising the step of indicating that the aggregated element group is a result of aggregation of multiple events.

39. The method according to claim 37, wherein the predefined filter is selected from the group comprising:

a specified degree of separation between the first and third data elements;

a specified spatial boundary including the locations of interest of the first and third data elements;

a specified spatial proximity to the locations of interest of the first and third data elements;

a specified hierarchy indicating a parent-child relationship between the first and third data elements; and

at least one specified association between the first and third data elements.

40. The method of claim 16, wherein the selected line type is different for each association type.

64. The method of claim 1, wherein the first data element is further defined as having a first time span in the temporal reference frame.

22. A computer implemented system for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as a corresponding plurality of visual elements, at least two of the data elements of the group linked by at least one association, the system comprising:

a processor and a memory;

a spatial visualization component configured for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating a first visual element of the plurality of visual elements representing a first data element of the group to a first location of interest in the spatial reference frame and relating a second visual element of the plurality of visual elements representing a second data element of the group to a second location of interest in the spatial reference frame, the first and second data elements being defined as positioned with respect to their respective location of interest for time defined in a temporal reference frame;

a temporal visualization component configured for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing the common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time trades configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest;

a visualization manager for linking the first data element to the second data element using the at least one association and for assigning a corresponding connection visual element in the visual representation between the first visual element and the second visual element, the at least one association configured for representing a connection between the first data element and the second data element and having at least one property in the spatial domain and including time as a temporal property; and

an aggregation manager configured for aggregating the first data element and the second data element into an aggregated element group according to a selected aggregation parameter, the aggregated element group configured for reducing the visual complexity of the visual elements for display in the visual representation, the aggregated element group being displayed on a third time track and the display of the first and second time tracks being removed from the visual representation;

wherein the visual representation is displayed on a user interface for subsequent interaction with user events.

41. The system of claim 22 wherein the visual representation of the group of data elements is selected from the group comprising;

a concurrent time and geographic context and a concurrent time and diagrammatic context.

42. The system of claim 41, wherein the diagrammatic context includes a plurality of nodes of a defined process, such that one of the nodes is positioned at the first location of interest and a second one of the nodes is positioned at the second location of interest.

43. The system of claim 41 wherein types of the data elements are selected from the group comprising;

entity, location, association, and event.

44. The system of claim 41, wherein the at least one association describes a pairing between two or more of the data elements for providing an information data object related to both of the two or more data elements.

45. The system of claim 44, wherein the connection visual element for the at least one association is a selected line type.

46. The system of claim 44, wherein the information data object is selected from the group comprising:

a communication connection describing communication details transferred between the two or more data elements;

a financial transaction or other transaction; and

a relationship connection describing social details in common between the two or more data elements.

47. The system of claim 46, wherein the information data object is selected from the group comprising:

a direct connection; and

an indirect connection.

48. The system of claim 44, wherein the at least one association is predefined or is created due to user interaction with the visualization representation.

49. The system of claim 48 wherein the user interaction generates user events for modifying visual properties of the connection visual element.

50. The system of claim 22, wherein the first and second data elements represent entitles connected to the same event by the at least one association.

51. The system of claim 22, further comprising a user interface configured for dynamically adding an annotation to the visual representation by the user, the annotation selected from the group comprising a geometrical object and a text object, the annotation having at least one property in the spatial domain and at least one property in the temporal domain.

52. The system of claim 51, wherein the geometrical object is selected from the group comprising:

a line; and

a closed shape.

53. The system according to claim 22, wherein the aggregation parameter is selected from the group comprising:

a time range in the temporal reference frame;

a spatial range in the spatial reference frame;

a location of interest;

a combination of locations of interest;

a hierarchy of locations of interest;

an event;

a combination of events;

an entity;

a combination of entities;

a level of detail for the visual complexity according to a scale of the visual representation;

an association; and

a combination of associations.

54. The system according to claim 53, wherein the aggregation parameter is selected from the group comprising:

predefined; and

dynamically specified by a user.

55. The system according to claim 53 further comprising the aggregation manager configured for selecting the combination of locations according to a selected region in the spatial reference frame.

56. The system according to claim 55, wherein the selected region is selected from the group comprising:

a point; and

a set of points.

57. The system according to claim 22 further comprising the aggregation manager configured for locating the third time track at a different location of interest calculated as a combination of the positions of the first and second locations of interest.

58. The system according to claim 22, wherein both the first data element and the third data element are selected from the first time track.

60. The system according to claim 22 further comprising the aggregation manager configured for attaching a label to the aggregated element group displayed in the visual representation.

61. The system according to claim 22 further comprising a predefined filter for specifying the aggregation parameter.

62. The system according to claim 22 further comprising the aggregation manager configured for indicating that the aggregated element group is a result of aggregation of multiple events.

63. The system according to claim 61, wherein the predefined filter is selected from the group comprising:

a specified degree of separation between the first and third data elements;

a specified spatial boundary including the locations of interest of the first and third data elements;

a specified spatial proximity to the locations of interest of the first and third data elements;

a specified hierarchy indicating a parent-child relationship between the first and third data elements; and

at least one specified association between the first and third data elements.

59. The system according to clam 58, wherein the aggregation parameter is selected from the group comprising:

predefined; and

dynamically specified by a user.