System and method for visualizing connected temporal and spatial information as an integrated visual representation on a user interface

• US 7,180,516 B2
• Filed: 03/29/2004
• Issued: 02/20/2007
• Est. Priority Date: 03/15/2003
• Status: Active Grant

• Current Assignee: Pen-Link Ltd.
• Original Assignee: Uncharted Software, Inc.
• Inventors: Wright, William, Kapler, Thomas
• Primary Examiner(s): Chauhan; Ulka
• Assistant Examiner(s): HAJNIK, DANIEL F
• Application Number:
  US10/810,680
• Publication Number:
  US 20050012743A1
• Time in Patent Office: 1,058 Days
• Field of Search: None
• US Class Current: 345/418
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline

Abstract

A system includes a visualization manager for assembling the group of data elements and for assigning a connection visual element in the visual representation between a first visual element representing a first data element of the group and a second visual element representing a second data element of the group. The system also has a spatial visualization component configured for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions. The reference surface is for relating the first visual element to a first location of interest in the spatial reference frame and for relating the second visual element to a second location of interest in the spatial reference frame. The system also has a temporal visualization component configured for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest. The temporal domain includes a first time track, such as a timeline, coupled to the first location of interest and a second time track coupled to the second location of interest, such that the first visual element is positioned on the first time track and the second visual element is positioned on the second time track. Each of the time tracks are configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface.

Claims

1. A method for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as a corresponding plurality of visual elements, the method comprising the steps of:

generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating a first visual element of the plurality of visual elements representing a first data element of the group to a first location of interest in the spatial reference frame and relating a second visual element of the plurality of visual elements representing a second data element of the group to a second location of interest in the spatial reference frame;

generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks extending with respect to the reference surface and configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest;

positioning a respective instant of focus at the intersection region between each of the time tracks and the reference surface at the respective locations of interest, the instant of foci being used for visually distinguishing a range of past time in the temporal domain preceding the instant of foci and a range of future time in the temporal domain after the instant of foci, each of the instant of foci configured for synchronous repositioning along their respective one of the time tracks; and

assigning a connection visual element in the visual representation between the first visual element and the second visual element, the connection visual element for representing an association distributed in at least one of the domains between the first visual element and the second visual element;

wherein the visual representation is displayed on a user interface for subsequent interaction with user events.

2. The method of claim 1, wherein the visual representation of the group of data elements is selected from the group comprising;

a concurrent time and geographic context and a concurrent time and diagrammatic context.

3. The method of claim 2, wherein the diagrammatic context includes a plurality of nodes of a defined process, such that one of the nodes is positioned at the first location of interest and a second one of the nodes is positioned at the second location of interest.

4. The method of claim 3 further comprising the step of specifying a time range of the temporal reference frame selected from the group comprising:

the range of past time of the temporal sequence preceding the instant of focus; and

the range of future time of the temporal sequence after the instant of focus.

5. The method of claim 1, further comprising the step of intersecting each of the time tracks through their respective location of interest such that the past ranges of the time tracks extend from one side of the reference surface and the future ranges of the time tracks extends from the other side of the reference surface.

6. The method of claim 5 further comprising the step of arranging the plurality of the visual elements along the time tracks according to the times at which the corresponding data elements occurred in the temporal reference frame, at least one of the plurality of the visual elements being independent and as such being not related to either of the first and second visual elements.

7. The method of claim 5 further comprising the step of arranging the plurality of the visual elements along the time tracks according to the times at which the corresponding data elements occurred in the temporal reference frame, the type of the data elements selected from the group comprising:

entitles;

events;

locations; and

associations.

8. The method of claim 6, wherein the location in the temporal reference frame of each of the plurality of visual elements on the time tracks is proportional to the distance from the instant of foci associated with the reference surface, the type of the visual elements selected from the group comprising:

entitles;

events;

locations; and

associations.

9. The method of claim 8, wherein the time tracks are represented as timelines in the visual representation.

10. The method of claim 9, wherein the temporal reference frame has a scale selected from the group comprising linear and logarithmic.

11. The method of claim 2 further comprising the step of maintaining an orientation of the time tracks with respect to the reference surface such that changes in the viewing orientation of the reference surface in response to the user events results in a corresponding change in the viewing orientation of the time tracks.

12. The method of claim 11, wherein the orientation angle between the time tracks and the reference surface is 90 degrees.

13. The method of claim 2 further comprising the step of maintaining an orientation of the time tracks with respect to the reference surface such that changes in the viewing orientation of the reference surface in response to the user events does not result in a corresponding change in the viewing orientation of the time tracks such that the time tracks are maintained in a vertical orientation.

14. The method of claim 13, wherein the orientation of the time tracks is such that the length of the time tracks is maximized as perceived by a user of the user interface.

15. The method of claim 2 further comprising the step of overlapping a time chart on the first time track and on the second time track, the time chart having a time axis and a spatial axis for representing the temporal reference frame and at least one of the spatial dimensions respectively, such that visual reference elements between the time axis and the time tracks are maintained as a new spatial orientation of the spatial reference frame is selected and the time tracks in the time chart remain continuously connected across the time chart and the spatial reference frame so that the time tracks in the time chart are repositioned to correspond to the new spatial orientation.

16. The method of claim 2, wherein types of the data elements are selected from the group comprising;

entity, location, association, and event.

17. The method of claim 16, wherein the event data element type represents an action taking place at a particular one of the locations of interest in the spatial reference frame and at a particular time in the temporal reference frame.

18. The method of claim 17, wherein the event data element type has data properties and display properties selected from the group comprising;

a short text label, description, location, start-time, end-time, general event type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, default+and user-set color, reference document, and reference photograph.

19. The method of claim 16, wherein the entity data element type represents an actor involved in a selected event.

20. The method of claim 19, where in the entity data element type has data properties and display properties selected from the group comprising;

short text label, description, general entity type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, default+and user-set color, and reference document, and reference photograph.

21. The method of claim 16, wherein the location data element type represents a location within the spatial reference frame.

22. The method of claim 21, wherein the location data element type has data properties and display properties selected from the group comprising;

position coordinates, a label, description, color information, precision information, location type, non-geospatial flag and user comments, reference document, and reference photograph.

23. The method of claim 22, wherein the location data element type is selected from the group comprising;

a physical location on a geospatial map, a physical location as a node in a diagram, and a virtual location related to a geospatial map such that the virtual location is not assigned to a specific physical location on the geospatial map.

24. The method of claim 2, wherein the association describes a pairing between two or more of the data elements for providing an information data object related to both of the two or more data elements.

25. The method of claim 24, wherein the connection visual element is a selected line type representing a direct connection between the first visual element and the second visual element.

26. The method of claim 24, wherein the information data object is selected from the group comprising:

a communication connection describing communication details transferred between the two or more data elements;

a financial transaction or other transaction; and

relationship connection describing social details in common between the two or more data elements.

27. The method of claim 24, wherein the association is predefined or is created due to user interaction with the visualization representation.

28. The method of claim 1, further comprising the step of manipulating the position of the instant of foci on the time tracks for facilitating synchronous modification of the visual properties of the plurality of visual elements on the time tracks.

29. The method of claim 28, wherein synchronous modification of the visual properties of the plurality of visual elements includes concurrent and continuous updating as the instant of foci are repositioned on the time tracks.

30. The method of claim 28 further comprising the step of adding at least one further visual element on one of the time tracks in response to the manipulation of the instant of foci with respect to a selected time range in the temporal reference frame.

31. The method of claim 28, further comprising the step of changing the displayed visual properties of the connection visual element in response to the manipulation of the instant of foci.

32. The method of claim 31, wherein the connection visual element is associated with movement of an entity visual element across the visual representation between the first location of interest and the second location of interest, the entity visual element representing an actor involved in a selected event.

33. The method of claim 28 further comprising the step of applying a filtering function to the visual elements and the at least one related association to select a subgroup thereof, selection of the subgroup according to a method selected from the group comprising;

criteria matching, algorithmic methods, and manual selection.

35. The method of claim 4 further comprising the step of specifying the range of past time independently from the range of future time.

36. The method of claim 4 further comprising the step of specifying the range of past time and the range of future time independently from the instant of foci.

37. The method of claim 5 further comprising the step of configuring a visual translucency of the reference surface in order to facilitate showing the time tracks extending below and above the reference surface.

38. The method of claim 7, wherein the visual elements positioned on the time tracks are icons.

39. The method of claim 23 further comprising the step of positioning the virtual location at an edge of the geospatial map in the spatial reference frame.

40. The method of claim 28 further comprising the step of removing at least one of the plurality of visual elements from at least one of the time tracks in response to the manipulation of the instant of foci with respect to a selected time range in the temporal reference frame.

41. The method of claim 28 further comprising the step of removing one of the time tracks and the respective visual elements positioned thereon from the visualization representation in response to the respective visual elements becoming outside of a selected time range in the temporal reference frame in response to the repositioning of the instant of foci.

42. The method of claim 28 further comprising the step of reassigning associated ones of the visual elements from the range of past time of the time tracks on one side of the reference surface to the range of future time of the time tracks on the other side of the reference surface in response to manipulation of the position of the instant of foci.

43. The method of claim 42 further comprising the step of positioning selected ones of the visual elements on the reference surface associated with ones of the data elements having a present time in the temporal reference frame corresponding to the time represented by the instant of foci.

44. The method of claim 42 further comprising the step of positioning selected ones of the visual elements above the reference surface associated with ones of the data elements having a present time in the temporal reference frame corresponding to the time represented by the instant of foci.

45. The method of claim 43, wherein the position of each of the selected visual elements is selected from the group comprising:

at the location of interests; and

interpolated between the location of interests.

46. The method of claim 43 further comprising the step of animating the change in position of each of the selected visual elements on the reference surface as the positions of the instant of foci are manipulated, such that the visual properties of the plurality of visual elements in the visualization representation are updated in both the temporal and spatial domains.

34. A computer program product for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as a corresponding plurality of visual elements, the computer program product comprising:

a computer readable medium;

a visualization module stored on the computer readable medium for assembling the group of data elements and for assigning a connection visual element in the visual representation between a first visual element of the plurality of visual elements representing a first data element of the group and a second visual element of the plurality of visual elements representing a second data element of the group;

a spatial visualization module stored on the computer readable medium for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating the first visual element to a first location of interest in the spatial reference frame and relating the second visual element to a second location of interest in the spatial reference frame;

a temporal visualization module stored on the computer readable medium for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks extending with respect to the reference surface and configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest; and

an interactive control component for positioning a respective instant of focus at the intersection region between each of the time tracks and the reference surface at the respective locations of interest, the instant of foci being used for visually distinguishing a range of past time in the temporal domain preceding the instant of foci and range of future time in the temporal domain after the instant of foci, each of the instant of foci configured for synchronous repositioning along their respective one of the time tracks;

wherein the connection visual element represents an association distributed in at least one of the domains between the first visual element and the second visual element such that the visual representation is displayed on a user interlace for subsequent interaction with user events.

47. The computer program produce of claim 34, wherein the visual representation of the group of data elements is selected from the group comprising;

a concurrent time and geographic context and a concurrent time and diagrammatic context.

48. The computer program product of claim 47, wherein the diagrammatic context includes a plurality of nodes of a defined process, such that one of the nodes is positioned at the first location of interest and a second one of the nodes is positioned at the second location of interest.

49. The computer program product of claim 48 further comprising a time range indicator of the temporal reference frame for specifying a range selected from the group comprising:

the range of past time of the temporal sequence preceding the instant of focus; and

the range of future time of the temporal sequence after the instant of focus.

50. The computer program product of claim 48, further comprising a time range indicator of the temporal reference frame for specifying the range of past time independently from the range of future time.

51. The computer program product of claim 48, further comprising a time range indicator of the temporal reference frame for specifying the range of past time and the range of future time independently from the instant of foci.

52. The computer program product of claim 34, wherein the plurality of the visual elements are arranged along the time tracks according to the times at which the corresponding data elements occurred in the temporal reference frame, at least one of the plurality of the visual elements being independent and as such being not related to either of the first and second visual elements.

53. The computer program product of claim 34, wherein the location data element type is selected from the group comprising;

a physical location on a geospatial map, a physical location as a node in a diagram, and a virtual location related to a geospatial map such that the virtual location is not assigned to a specific physical location on the geospatial map.

54. The computer program product of claim 53, wherein the virtual location is positioned at an edge of the geospatial map in the spatial reference frame.

55. The computer program product of claim 48, wherein the association describes a pairing between two or more of the data elements for providing an information data object related to both of the two or more data elements.

56. The computer program product of claim 55, wherein the information data object is selected from the group comprising:

a communication connection describing communication details transferred between the two or more data elements;

a financial transaction or other transaction; and

a relationship connection describing social details in common between the two or more data elements.

57. The computer program product of claim 34, wherein the position of the instant of foci is configured for manipulation on the time tracks for facilitating synchronous modification of the visual properties of the plurality of visual elements on the time tracks.

58. The computer program product of claim 57, wherein synchronous modification of the visual properties of the plurality of visual elements includes concurrent and continuous updating as the instant of foci are repositioned on the time tracks.

59. The computer program product of claim 57, wherein the visualization module is further configured for adding at least one further visual element on one of the time tracks in response to the manipulation of the instant of foci with respect to a selected time range in the temporal reference frame.

60. The computer program product of claim 57, wherein the visualization module is further configured for removing at least one of the plurality of visual elements from at least one of the time tracks in response to the manipulation of the instant of foci with respect to a selected time range in the temporal reference frame.

61. The computer program product of claim 57, wherein the visualization module is further configured for removing one of the time tracks and the respective visual elements positioned thereon from the visualization representation in response to the respective visual elements becoming outside of a selected time range in the temporal reference frame in response to the repositioning of the instant of foci.

62. The computer program product of claim 57, wherein the visualization module is further configured for reassigning associated ones of the visual elements from the range of past time of the time tracks on one side of the reference surface to the range of future time of the time tracks on the other side of the reference surface in response to manipulation of the position of the instant of foci.

63. The computer program product of claim 62, wherein the visualization module is further configured for positioning selected ones of the visual elements on the reference surface associated with ones of the data elements having a present time in the temporal reference frame corresponding to the time represented by the instant of foci.

64. The computer program product of claim 62, wherein the visualization module is further configured for positioning selected ones of the visual elements above the reference surface associated with ones of the data elements having a present time in the temporal reference frame corresponding to the time represented by the instant of foci.

65. The computer product of claim 63, wherein the position of each of the selected visual elements is selected from the group comprising:

at the location of interests; and

interpolated between the location of interests.

66. The computer program product of claim 63, wherein the visualization module is further configured for animating the change in position of each of the selected visual elements on the reference surface as the positions of the instant of foci are manipulated, such that the visual properties of the plurality of visual elements in the visualization representation are updated in both the temporal and spatial domains.

67. The computer program of claim 57, wherein the visualization module is further configured for changing the displayed visual properties of the connection visual element in response to the manipulation of the instant of foci.

68. The computer program product of claim 67, wherein the connection visual element is associated with movement of an entity visual element across the visual representation between the first location of interest and the second location of interest, the entity visual element representing an actor involved in a selected event.

69. The computer program product of claim 57, where the visualization module is further configured to apply a filtering function to the visual elements and the at least one related association to select a subgroup thereof, selection of the subgroup according to a method selected from the group comprising;

criteria matching, algorithmic methods, and manual selection.