Method for tracking and assessing program participation

• US 20030061231A1
• Filed: 09/18/2002
• Published: 03/27/2003
• Est. Priority Date: 09/18/2001
• Status: Active Grant

• Current Assignee: Victoria M. Lovegren
• Original Assignee: Victoria M. Lovegren
• Inventors: Lovegren, Victoria M.
• Granted Patent: US 7,269,579 B2
• Time in Patent Office: Days
• US Class Current: 707/103Y
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Disclosed is a computerized decision support system and method for a) tracking participation within programs, b) capturing participant'"'"'s participation activity and assessment information in a format that can be easily analyzed and c) distilling the participation and assessment data into useful management and evaluation information. The repository of longitudinal data can be analyzed and reported for case-management and program-evaluation purposes. An assessment module enables analyzable assessment instruments to be custom-defined by the system user, e.g. a program manager. The customized assessment instrument is used to provide answer-restricted questions during an assessment interview, enabling virtually any data item to be tracked historically. The system captures date/time-stamped participation information at various levels of detail and stores this information in a way that can be easily retrieved and analyzed for program and participant-focused analysis. A set of industry-standard participation events can be tracked, with supporting detail, as well as less-data-intensive ad hoc user-defined activities. The data model underlying the system, and the implementation of this model within a relational database system, provides a great degree of flexibility, adaptability and efficient navigation through the database for analysis and reporting. Though numerous program-evaluation reports are provided, a set of intermediary aggregations of data is also available for efficient evaluation of additional program outcome measures.

Claims

1. A method for performing an aggregate assessment of a group of subjects given subject-specific assessments performed using assessment instruments and data-validation rules comprising the steps of:

a) providing, within an assessment-instrument data structure, a means of representing a set of questions, answer restrictions, and question-answer-validation rules;

b) providing, within said assessment-instrument data structure, a means of representing a plurality of assessment instruments, storing, for each said assessment instrument, an assessment-instrument key and a plurality of links to associated questions;

c) providing, within a subject data structure, a means of representing a plurality of subjects, said data structure storing, for a each said subject, a subject key and a plurality of subject attributes;

d) providing, within an assessment-event data structure, a means of representing a plurality of said assessment events, said data structure storing, for each said assessment event, an associated said assessment-event key, associated said subject and assessment-instrument keys, as well as a reference to a point in time;

e) storing, within said assessment-event data structure, for each said assessment event, the assessment results including of a plurality of validated answers to a plurality of associated said questions; and

f) deriving, for any group of subjects, assessment results for said group by aggregating said assessment results from subjects within said group and utilizing linked data within said data structures.

2. The method of claim 1 wherein a fixed and minimal number of flexible general-purpose data structures are used to represent an unlimited number of assessment instruments, and assessment-instrument definition is data-driven and user-maintained.

3. The method of claim 1 further including an assessment-instrument-definition user interface that permits users to create answers, questions and other assessment-definition elements, such as question types, answer weights, and attributes such as multi-response and required, the end result being the creation of assessment instruments whose answers can be analyzed in aggregate.

4. The method of claim 1 further including an assessment-event user interface, the method comprising:

a) using the interface to administer a plurality of said assessment events;

b) using the assessment-instrument definition information residing in the said assessment-instrument data structure to enforce question-answer integrity;

c) using, for a given assessment event, said assessment-event user interface to display a plurality of associated said questions, to capture and validate associated said subject'"'"'s given answer(s), and to store said answers in said assessment-event data structure.

5. The method of claim 4 wherein the process for answering a given question during an assessment event uses a form whose controls for collecting said subject'"'"'s answer vary by question type, said controls being defined to ensure question-answer integrity

6. The method of claim 4 wherein an alternate set of forms can be used for assessment event administration, in the situation where the associated said assessment instrument is of simple construct, an example being said instruments which have only single-question question groups.

7. The method of claim 1 further including an assessment-event user interface, the method comprising:

a) utilizing the interface to administer a plurality of said assessment events; and

b) utilizing, for a given assessment event, said assessment-event user interface to display a plurality of associated said questions, to capture and validate associated said subject'"'"'s given answer(s), and to store said answers in said assessment-event data structure.

8. The method of claim 1 further comprising an aggregate-assessment user interface, the method comprising the steps of:

a) collecting scenario parameters such as subject grouping parameters, analysis-horizon parameters, assessment instruments or questions to analyze, output medium, output of graphing format; and

b) utilizing a standard set of queries to operate against the said assessment data structures, using the inputted parameters, deriving and presenting aggregate assessment results.

9. The method of claim 1 wherein subject-specific assessment results are provided by an external data source, rather than through the standard assessment-event user interface or defined within hard-coded assessment instruments

11. The method of claim 1 where special types of assessment instruments have an aggregate score associated with each assessment event that uses that instrument, the method comprising the steps of:

a) defining numeric questions that are to be used in the assessment instrument;

b) defining the upper and lower bounds that the associated answers can take on;

c) define the set of intervals that partition the range of values derived from summing the lower and upper bounds of the said instrument'"'"'s questions;

d) associating, with each said interval, an interpretation; and

e) using such aggregate-score instruments, having the system automatically assign an associated score interpretation to the associated events.

12. The method of claim 1 where static subject attributes include sex, race, date of birth, and such attributes are used for subject grouping and analysis scenarios

13. The method of claim 1 where dynamic subject attributes can be captured within assessment events, the method comprising the steps of:

a) defining dynamic subject attributes as questions, said attributes containing, for example, marital status, job info, school info, and address;

b) creating an assessment instrument which includes said variables;

c) utilizing the said assessment instrument to conduct assessment events, capturing the dynamic attributes; and

d) utilizing assessment instruments, with date stamp, to capture dynamic subject attributes, such as address, school, job, and marital status.

10. The method of claim 10 wherein criminal activity is an externally-provided assessment result that is used in measuring recidivism rates for various subject populations

14. A method for tracking program participation within programs that provide services to program participants, wherein the participation experience includes service events and said assessment events, and assessment data has been stored in assessment-instrument and assessment-event data structures, the method comprising the steps of:

a) providing a means of representing a plurality of programs having varying program components and services, storing a unique program key and a plurality of program attributes in a program-definition data structure;

b) providing a means of representing a plurality of program-participation experiences, storing, for each said program-participation experience, a unique program-participation key, an associated program key, an associated program-participant key, and a reference to a participation time period within a program-participation data structure; and

c) representing, also within said program-participation data structure, for a given said program-participation experience, a plurality of program-participation activities and events, including said assessment events, wherein each said event is linked to the said experience by said program-participation key.

15. The method of claim 14, wherein case management of program participants can be facilitated by the captured program-participation and assessment information.

16. The method of claim 15 wherein the method is used for case management purposes, the method comprising:

a) defining activities to be tracked;

b) defining the assessment instruments to be used for assessing program participant'"'"'s progress;

c) tracking, for each said program participant, the activities in which said participant was involved;

d) conducting assessments, for each program participant, assessing participant outcomes; and

e) reporting participant-specific information about program participation and assessment results to ascertain relationships between participation and participant outcomes and behavior.

17. A method for assessing the effectiveness of programs that provide services to program participants wherein assessment questions have been stored in an assessment-instrument data structure and validated assessment results for program participants have been stored as assessment events in an assessment-event data structure, and are analyzed, in aggregate, relative to program-participation experiences, the method comprising the steps of:

a) providing a means of representing a plurality of programs having varying program components and services, storing a unique program key and a plurality of program attributes in a program-definition data structure;

b) providing a means of representing a plurality of program-participation experiences, storing, for each said program-participation experience, a unique program-participation key, an associated program key, an associated program-participant key, and a reference to a participation time period within a program-participation data structure;

c) representing, also within the said program-participation data structure, for a given said program-participation experience, a plurality of program-participation activities and events, including said assessment events, wherein each said event is linked to the said experience by said program-participation key; and

d) for any said program having outcome indicators represented in said assessment instruments, selecting the set of said assessment events associated with said instrument(s) and with said program'"'"'s said participants, and deriving program-level assessment results by aggregating said assessment results of said assessment events.

18. The method of claim 17 wherein programs can utilize said assessment instruments as one measure of program effectiveness, the method further comprising the steps of:

a. determining outcome indicators that measure their program effectiveness;

b. incorporating those outcome indicators into the said assessment instruments by defining said outcome indicators as questions or as aggregate-score assessment instruments;

c. administering assessment events to its participants using the said outcome-indicator assessment instruments; and

d. aggregating the results from said assessment events to derive useful information about the outcomes of the program

19. The method of claim 18 wherein the program-level outcome measures derived from subject-specific assessment results can be further aggregated, using relationships between program and program provider, program provider and community, community and region, etc., this higher-level aggregation accomplished by the utilization of standardized instruments, containing standardized questions whose answers use standardized answers, such as DSM-IV diagnosis codes

20. The method of claim 17 further comprising an program-evaluation user interface, the method comprising the steps of:

c) collecting scenario parameters such as program to be evaluated, participant grouping parameters, analysis-horizon parameters, assessment instruments or questions to analyze, comparison parameters, output medium, output of graphing format; and

d) utilizing a standard set of queries to operate against the said assessment data structures and the program-participation data structures, using the inputted parameters, deriving and presenting aggregate assessment results.

21. A computer readable medium having computer executable instructions stored thereon for performing method steps for performing an aggregate assessment of a group of subjects given subject-specific assessments performed using assessment instruments and data-validation rules, the method steps comprising:

a) providing, within an assessment-instrument data structure, a means of representing a set of questions, answer restrictions, and question-answer-validation rules;

b) providing, within said assessment-instrument data structure, a means of representing a plurality of assessment instruments, storing, for each said assessment instrument, an assessment-instrument key and a plurality of links to associated questions;

c) providing, within a subject data structure, a means of representing a plurality of subjects, said data structure storing, for a each said subject, a subject key and a plurality of subject attributes;

d) providing, within an assessment-event data structure, a means of representing a plurality of said assessment events, said data structure storing, for each said assessment event, an associated said assessment-event key, associated said subject and assessment-instrument keys, as well as a reference to a point in time;

e) storing, within said assessment-event data structure, for each said assessment event, the assessment results including of a plurality of validated answers to a plurality of associated said questions; and

f) deriving, for any group of subjects, assessment results for said group by aggregating said assessment results from subjects within said group and utilizing linked data within said data structures.

22. A computer readable medium having computer executable instructions stored thereon for performing method steps for assessing the effectiveness of programs that provide services to program participants wherein assessment questions have been stored in an assessment-instrument data structure and validated assessment results for program participants have been stored as assessment events in an assessment-event data structure, and are analyzed, in aggregate, relative to program-participation experiences, the method steps comprising:

a) providing a means of representing a plurality of programs having varying program components and services, storing a unique program key and a plurality of program attributes in a program-definition data structure;

b) providing a means of representing a plurality of program-participation experiences, storing, for each said program-participation experience, a unique program-participation key, an associated program key, an associated program-participant key, and a reference to a participation time period within a program-participation data structure;

c) representing, also within the said program-participation data structure, for a given said program-participation experience, a plurality of program-participation activities and events, including said assessment events, wherein each said event is linked to the said experience by said program-participation key; and

d) for any said program having outcome indicators represented in said assessment instruments, selecting the set of said assessment events associated with said instrument(s) and with said program'"'"'s said participants, and deriving program-level assessment results by aggregating said assessment results of said assessment events.

23. A computer readable medium having computer-executable instructions stored thereon for assessing the effectiveness of programs that provide services to program participants wherein assessment questions have been stored in an assessment-instrument data structure and validated assessment results for program participants have been stored as assessment events in an assessment-event data structure, and are analyzed, in aggregate, relative to program-participation experiences, the method comprising the steps of:

a) providing a means of representing a plurality of programs having varying program components and services, storing a unique program key and a plurality of program attributes in a program-definition data structure;

b) providing a means of representing a plurality of program-participation experiences, storing, for each said program-participation experience, a unique program-participation key, an associated program key, an associated program-participant key, and a reference to a participation time period within a program-participation data structure;

c) representing, also within the said program-participation data structure, for a given said program-participation experience, a plurality of program-participation activities and events, including said assessment events, wherein each said event is linked to the said experience by said program-participation key; and

d) for any said program having outcome indicators represented in said assessment instruments, selecting the set of said assessment events associated with said instrument(s) and with said program'"'"'s said participants, and deriving program-level assessment results by aggregating said assessment results of said assessment events.