System and method for visualizing connected temporal and spatial information as an integrated visual representation on a user interface

• US 20050012743A1
• Filed: 03/29/2004
• Published: 01/20/2005
• Est. Priority Date: 03/15/2003
• Status: Active Grant

• Current Assignee: Pen-Link Ltd.
• Original Assignee: Uncharted Software, Inc.
• Inventors: Wright, William, Kapler, Thomas
• Granted Patent: US 7,180,516 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline

Abstract

As data about events and objects become more commonly available, analyzing and understanding of interrelated temporal and spatial information is increasingly a concern for military commanders, intelligence analysts and business analysts. A system and method is provided for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties. The data elements are included in the visual representation as corresponding visual elements, such that the data elements of the group linked by at least one association. The system includes a visualization manager for assembling the group of data elements using the at least one association and for assigning a connection visual element in the visual representation between a first visual element representing a first data element of the group and a second visual element representing a second data element of the group. The system also has a spatial visualization component, such as a sprite, configured for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions. The reference surface is for relating the first visual element to a first location of interest in the spatial reference frame and for relating the second visual element to a second location of interest in the spatial reference frame. The system also has a temporal visualization component, such as a sprite, configured for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest. The temporal domain includes a first time track, such as a timeline, coupled to the first location of interest and a second time track coupled to the second location of interest, such that the first visual element is positioned on the first time track and the second visual element is positioned on the second time track. Each of the time tracks configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface. In implementation of the method, the connection visual element represents a distributed association in at least one of the domains between the first visual element and the second visual element such that the visual representation is displayed on a user interface for subsequent interaction with user events, including animation of the visual elements to help in the analysis of the data contained in the visual representation.

Claims

1. A method for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as corresponding visual elements, the data elements of the group linked by at least one association, the method comprising the steps of:

assembling the group of data elements using the at least one association;

generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating a first visual element representing a first data element of the group to a first location of interest in the spatial reference frame and relating a second visual element representing a second data element of the group to a second location of interest in the spatial reference frame;

generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface; and

assigning a connection visual element in the visual representation between the first visual element and the second visual element, the connection visual element for representing a distributed association in at least one of the domains between the first visual element and the second visual element;

wherein the visual representation is displayed on a user interface for subsequent interaction with user events.

2. The method of claim 1, wherein the visual representation of the group of data elements is selected from the group comprising;

a concurrent time and geographic context, and a concurrent time and diagrammatic context.

3. The method of claim 2 further comprising the step of configuring the reference surface for providing an instant of focus in the temporal reference frame for at least some of the temporal and spatial properties of the group of data elements, the instant of focus coupled to the locations of interest of the reference surface, the instant of focus selected by the user events.

4. The method of claim 3 further comprising the step of configuring the temporal reference frame for providing a time range selected from the group comprising;

a past range of the temporal sequence preceding the instant of focus, and a future range of the temporal sequence after the instant of focus.

5. The method of claim 4 further comprising the step of intersecting the first time track through the first location of interest such that the past range of the first time track extends from one side of the reference surface and the future range of the first time track extends from the other side of the reference surface, the instant of focus located on the first time track at the intersection point between the first time track and the reference surface.

6. The method of claim 5 further comprising the step of arranging a first plurality of the visual elements along the first time track according to the times at which the visual elements occurred in the temporal reference frame.

7. The method of claim 6 further comprising the step of intersecting the second time track through the second location of interest such that the past range of the second time track extends from one side of the reference surface and the future range of the first time track extends from the other side of the reference surface, the second instant of focus located on the second time track at the intersection point between the second time track and the reference surface.

8. The method of claim 7 further comprising the step of arranging a second plurality of the visual elements along the second time track according to the times at which the visual elements occurred in the temporal reference frame.

9. The method of claim 6, wherein the location in the temporal reference frame of each of the plurality of visual elements on the time tracks is proportional to the distance from the instant of focus associated with the reference surface.

10. The method of claim 9, wherein the time tracks are represented as timelines in the visual representation.

11. The method of claim 10, wherein the temporal reference frame has a scale selected from the group comprising linear and logarithmic.

12. The method of claim 2 further comprising the step of maintaining an orientation of the time tracks with respect to the reference surface such that changes in the attitude of the reference surface in response to the user events results in a corresponding change in the orientation of the time tracks.

13. The method of claim 12, wherein the orientation angle between the time tracks is 90 degrees.

14. The method of claim 2 further comprising the step of maintaining an orientation of the time tracks with respect to the reference surface such that changes in the attitude of the reference surface in response to the user events does not result in a corresponding change in the orientation of the time tracks.

15. The method of claim 14, wherein the orientation of the time tracks is such that the length of the time tracks is maximized as perceived by a user of the user interface.

16. The method of claim 2 further comprising the step of overlapping a time chart on the first time track and the second time track, the time chart having a time axis and a spatial axis for representing the temporal reference frame and at least one of the spatial dimensions respectively.

17. The method of claim 16 further comprising the step of maintaining an orientation of the time chart with respect to the reference surface such that changes in the attitude of the reference surface in response to the user events does not result in a corresponding change in the orientation of the time chart.

18. The method of claim 17, wherein the time chart is represented as a rectangular region.

19. The method of claim 16 further comprising the step of configuring the time chart for providing an instant of focus in the temporal reference frame for at least some of the temporal and spatial properties of the group of data elements, the instant of focus coupled to the locations of interest of the reference surface, the instant of focus selected by the user events.

20. The method of claim 19 further comprising the step of configuring the time chart for providing a time range selected from the group comprising;

a past range of the temporal sequence preceeding the instant of focus, and a future range of the temporal sequence after the instant of focus.

21. The method of claim 20, wherein the time chart has a superimposed grid for representing the time axis and the spatial axis.

22. The method of claim 2, wherein types of the data elements are selected from the group comprising;

entity, location, and event.

23. The method of claim 22, wherein the event data element type represents an action taking place at a particular one of the locations of interest in the spatial reference frame and at a particular time in the temporal reference frame.

24. The method of claim 23, wherein the event data element type has data properties and display properties selected from the group comprising;

a short text label, description, location, start-time, end-time, general event type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, and default+user-set color.

25. The method of claim 22, wherein the entity data element type represents an actor involved in a selected event.

26. The method of claim 25, wherein the entity data element type has data properties and display properties selected from the group comprising;

short text label, description, general entity type, icon reference, visual layer settings, priority, status, user comment, certainty value, source of information, and default+user-set color.

27. The method of claim 22, wherein the location data element type represents a location within the spatial reference frame.

28. The method of claim 27, wherein the location data element type has data properties and display properties selected from the group comprising;

position coordinates, a label, description, color information, precision information, location type, non-geospatial flag and user comments.

29. The method of claim 28, wherein the location data element type is selected from the group comprising;

a physical location on a geospatial map, a physical location as a node in a diagram, and a virtual location related to a geospatial map.

30. The method of claim 22, wherein the at least one association describes a pairing between two or more of the data elements.

31. The method of claim 30, wherein the connection visual element is a solid line representing a direct connection between the first visual element and the second visual element.

32. The method of claim 31, wherein the solid line has a pointer for indicating a vector property of the visual connection element.

33. The method of claim 31, wherein the connection visual element is a trail represented as a series of connected lines tracing known locations in the spatial reference frame of a selected entity visual element over time represented by the temporal reference frame.

34. The method of claim 2 further comprising the step of updating the visual elements in the visual representation in response to the user events.

35. The method of claim 34, wherein the user events are generated in response to manipulation by the user of an interactive control for modifying visual properties of the visual representation.

36. The method of claim 35, wherein the interactive control is selected from the group comprising a time range selector and an instant of focus selector.

37. The method of claim 35 further comprising the step of animating the display of at least one of the visual elements of the visual representation in response to the manipulation of the interactive control.

38. The method of claim 37, wherein the display of the connection visual element of the visual representation is animated in response to the manipulation of the interactive controls.

39. The method of claim 38, wherein the connection visual element is coupled to the movement of an entity visual element across the visual representation between the first location of interest and the second location of interest, entity visual element representing an actor involved in a selected event.

40. The method of claim 35 further comprising the step of applying a filtering function to the visual elements and the at least one related association to select a subgroup thereof.

41. The method of claim 40 further comprising the step of selecting the subgroup according to a method selected from the group comprising;

criteria matching, algorithmic methods, and manual selection.

42. The method of claim 40 further comprising the step of processing the selected subgroup of visual elements to change the presentation of the visual representation selected from the group comprising;

highlighting the subgroup and removing the subgroup.

43. A system for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as corresponding visual elements, the data elements of the group linked by at least one association, the system comprising:

a visualization manager for assembling the group of data elements using the at least one association and for assigning a connection visual element in the visual representation between a first visual element representing a first data element of the group and a second visual element representing a second data element of the group;

a spatial visualization component configured for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating the first visual element to a first location of interest in the spatial reference frame and relating the second visual element to a second location of interest in the spatial reference frame; and

a temporal visualization component configured for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface; and

wherein the connection visual element represents a distributed association in at least one of the domains between the first visual element and the second visual element such that the visual representation is displayed on a user interface for subsequent interaction with user events.

44. A computer program product for creating a multidimensional visual representation of a group of data elements having integrated temporal and spatial properties, the data elements being included in the visual representation as corresponding visual elements, the data elements of the group linked by at least one association, the computer program product comprising:

a computer readable medium;

a visualization module stored on the computer readable medium for assembling the group of data elements using the at least one association and for assigning a connection visual element in the visual representation between a first visual element representing a first data element of the group and a second visual element representing a second data element of the group;

a spatial visualization module stored on the computer readable medium for generating a spatial domain of the visual representation to include a reference surface for providing a spatial reference frame having at least two spatial dimensions, the reference surface for relating the first visual element to a first location of interest in the spatial reference frame and relating the second visual element to a second location of interest in the spatial reference frame; and

a temporal visualization module stored on the computer readable medium for generating a temporal domain of the visual representation operatively coupled to the spatial domain, the temporal domain for providing a common temporal reference frame for the locations of interest, the temporal domain including a first time track coupled to the first location of interest and a second time track coupled to the second location of interest, the first visual element positioned on the first time track and the second visual element positioned on the second time track, each of the time tracks configured for visually representing a respective temporal sequence of a plurality of the data elements at each of the locations of interest of the reference surface; and

wherein the connection visual element represents a distributed association in at least one of the domains between the first visual element and the second visual element such that the visual representation is displayed on a user interface for subsequent interaction with user events.