Item tracking, database management, and relational database system associated with multiple large scale test and assessment projects

• US 20050086257A1
• Filed: 10/15/2004
• Published: 04/21/2005
• Est. Priority Date: 10/17/2003
• Status: Abandoned Application

• Current Assignee: Measured Progress, Inc.
• Original Assignee: Measured Progress, Inc.
• Inventors: Wright, Lonnie Glen
• Application Number:
  US10/966,718
• Publication Number:
  US 20050086257A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A system and process includes: a relational database that has a core database of global rules and project data, an item tracking database, a scanning and image management database, and a scoring database; an enterprise integrated application software program and computer network; and a web interface linking host operations and users; all of which is used for generating a production order and packing schedule of test documents from the rules and project data, for tracking production, packing, shipping, return shipping, and processing of actual test documents by using unique identifiers assigned to every document and monitoring and recording every movement of every document and appending all records of movement and other changes of condition to the original project data in the relational database, for scanning all documents into image form and storing and scoring the images, and for generating reports based on the scores and demographics of the testees.

Claims

1. A process for conducting an assessment project of testing multiple testees comprising the steps of:

using a system comprising a document processing system for exposing articles and individual pages to electronic scanning, a computer network connected to the document processing system and to the internet, and a software program associated with said document processing system and said computer network;

said software program comprising global business rules, a relational database, a web browser application for host and client interface, and an image processing application associated with said document processing system;

storing in said relational database a unique project identifier and associated project data;

said project data comprising project rules, project locations and unique location identifiers, and quantities of testees at each said project location;

generating a master items production quantity requirement which includes each item of assessment materials for said project in accordance with said global business rules and said project data, said items comprising secure test materials and ancillary test materials, said secure test materials comprising sheets of answer pages susceptible of electronic scanning;

creating and storing in said relational database a unique item identifier corresponding to each respective item of secure test materials including each sheet containing at least one answer page;

producing said assessment materials corresponding to said master items production quantity requirement, including producing each said item of secure test materials with its respective said unique identifier inseparably affixed thereto;

generating a production plan for assembly of said assessment materials in accordance with said global business rules and said project data, said plan comprising using combinations of pre-selected intermediate quantities of said secure test materials and said ancillary test materials for packing shipping containers;

packaging said pre-selected intermediate quantities of said secure test materials into class packs according to said production plan, each said class pack being assigned and labeled with a unique identifier which is stored in said relational database and associated with the unique identifiers of the items of secure test materials within;

packaging said pre-selected intermediate quantities of said ancillary materials into kits according to said production plan such that at least one kit is required at each said project location, each said kit being assigned and labeled with a unique identifier which is stored in said relational database and associated with the master items within and the quantities thereof;

allocating selected said kits and said class packs to specific project locations, and associating in said relational database the unique identifiers of each said class pack and said kit with the unique identifier of the respective project location;

producing and including return shipping envelopes for each said project location for packaging and returning selected said items of secure test materials, each said envelope assigned and labeled with a unique identifier which is stored in said relational database and associated therein with the project location identifier and item identifiers of items expected to be returned therein;

packing said class packs, said return shipping envelopes and said kits by project location into said shipping containers, each said container being assigned and labeled with a unique identifier which is stored in said relational database and associated with the unique identifiers of the class packs and kits and return shipping labels within and the respective project location identifier of the project location for which it is intended;

consigning said containers to a shipper for shipping, obtaining therefrom a shipper'"'"'s identifier for each said container which is stored in said relational database and associated therein with the unique identifiers of the respective said container and the project location;

sending a notice of shipment including the shipper'"'"'s identifier for each said container to a designated POC at each respective project location;

periodically querying said shipper for status of said containers consigned, recording responses in said relational database, and associating said responses with respective said shipper'"'"'s identifier for each said container; and

generating on demand, status reports relating to said project.

2. The process according to claim 1, wherein said secure test materials comprise multiple forms of test booklets and each said form of a test booklet has a unique form identifier and each said test booklet of the same form has a unique booklet identifier, said project data requires a particular distribution of said forms of test booklets among project locations and among the group of testees at each said project location, comprising the further steps:

generating and storing in said relational database a schedule for the producing of said test booklets and packaging into said class packs wherein the associated forms identifiers are arranged and grouped into a distributed set of form identifiers and test booklet identifiers in class pack quantities, each said set being assigned a class pack identifier which is stored in said relational database and associated with its respective said form and test booklet identifiers and their distributed order therein;

producing said test booklets and said class packs according to said schedule, each said class pack being labeled with its respective unique identifier;

validating said class packs to said schedule by comparing the form and test booklet identifier of test booklets at selected locations within selected class packs to the corresponding schedule of test booklet identifiers at corresponding locations within of the respective set of class pack identifiers in said relational database.

3. The process of claim 1, said step of producing said assessment materials corresponding to said master items production quantity requirement, including producing each said item of secure test materials with its respective said unique identifier inseparably affixed thereto, comprising producing test booklets in sequence and incorporating a respective unique identifier with each said test booklet as it is produced.

6. The process of claim 1, comprising the steps of:

receiving requests for changes;

evaluating according to global business rules and project data all said requests for at least one of requestor'"'"'s authority, format of request, items authorized, and quantities authorized;

approving or refusing said requests;

appending any approved said requests in said relational database to original project data and production requirements and any prior approved requests such that original project data and original production requirements and all changes thereto are preserved in said relational database; and

updating the current in-use project data and production requirements to be the summation of the original project data and production requirements plus all approved changes thereof.

7. The process of claim 6, the step of receiving requests for changes comprising receiving online interactive requests for changes.

8. The process of claim 6, said step of refusing or approving comprising:

refusing requests for shipments of additional materials to a project location if no acknowledgement of receipt of prior shipments has been received.

10. The process of claim 6, further comprising the steps of:

permitting on-line system access and real time monitoring and audit control of said project data and said production requirements and requested changes thereto.

11. The process of claim 1, further comprising the steps:

recording, uploading and comparing the container, kit and class pack identifiers received at said project locations with corresponding identifiers in said relational database to confirm delivery of intended materials.

12. The process of claim 11, further comprising the steps:

distributing said assessment materials to said testees;

assuring that the particular secure materials used by each testee are marked with its testee identity;

conducting testing of said testees whereby testee responses to test items within its secure test materials are entered on the respective answer pages;

collecting, repacking and reshipping at least said secured test materials using said return shipping labels to the system host for processing; and

providing notice to said system of said reshipping.

13. The process of claim 1, said steps further comprising:

using a scanner at a project location,

scanning assessment materials at said project location; and

connecting said scanner to said system via said on-line internet access for uploading of scanned information.

14. The process of claim 1, said system further comprising hand held scanners.

15. The process of claim 14, said scanners being connectible to said system by wireless means.

16. A process according to claim 1, said project data further comprising testee identities by project location, comprising the further steps:

creating and storing a unique identifier in said relational database for each said testee identity in association with its respective project location;

associating in said relational database a said secure test materials identifier with each said testee identifier;

producing a testee set of said secure test materials with a respective said secure test materials identifier and associated testee identifier inseparably affixed thereto;

packaging said testee sets into said class packs, each said class pack containing testee sets for a common project location.

19. The process of claim 1, comprising the further step:

incorporating unique identifiers with items of assessment materials in the form of an RFID chip programmed with at least a unique identifier associated with the respective said item;

reading said unique identifiers by means of bringing said assessment materials within range of an RFID signal receiver and initiating an RFID chip transmission, said RFID receiver communicating received information with said system.

21. The process of claim 19, said RFID chip being further programmed with processing instructions.

27. A process according to claim 1, said unique identifiers comprising bar-codes, said scanning said identifiers comprising laser scanning, zone optical character recognition and live video encoding (LVE) of said bar-codes.

28. A process according to claim 1, comprising the further steps

associating a unique testee identifier with respective testee identity and demographic data in said relational database;

and

providing on-line internet access to selected parties for review and revision of said testee demographic data.

29. A process according to claim 28, comprising the further step

adding additional testee identities and related demographic data to said relational database;

thereby creating respective additional unique testee identifiers associated with said additional testees in said relational database.

30. A process according to claim 28, comprising the further steps:

assigning a particular test booklet to a particular testee for testing; and

associating the related test booklet identifier with the related testee identifier in said relational database.

31. A process according to claim 28, comprising the further steps:

associating a particular test booklet identifier with a particular testee identifier in said relational database; and

matching the related test booklet to the related testee for testing.

32. A process according to claim 29, said step of adding additional testee identities comprising the steps:

producing and packing a pre-selected overage of secure test materials with initial shipments to each said project location, said overage of secure test materials having respective available identifiers recorded in said rational database for later association with a testee identifier;

adding additional testees in said relational database after initial shipments have shipped;

associating each related said additional testee identifier with said available identifiers of said overage of secure test materials in said relational database;

creating at said project location a corresponding identifier label for each said additional testee; and

attaching said identifier label to the corresponding said secure test materials.

33. A process according to claim 1, said software program employing extended markup language (XML) for electronic data interchange.

4. A process for preparing multiple controlled articles of multiple forms for distribution of the articles including a pre-determined scheme of distribution of the forms of the articles among a group of recipients, comprising the steps:

generating and storing in a relational database a schedule of form identifiers, each corresponding to a different said form of the article, and a unique article identifier for each article of the same form in the quantities required, where each type of form identifier is distributed according to said scheme throughout the schedule and associated with a said article identifier for an article of the same form.

producing and labeling an arrangement of articles corresponding to said schedule;

validating the correct arrangement of said articles for proper distribution of said forms of said article by comparing the form and article identifier of articles at selected locations within said arrangement to the corresponding schedule of form and article identifiers at corresponding locations within the schedule.

5. The process of claim 4, the step of producing, arranging and labeling said articles according to said schedule comprising

producing said articles in sequence and incorporating said form and article identifiers according to said schedule with each said article as it is produced.

9. A process for interpreting whether a specific, visible target of finite area on a sheet has been manually marked to indicate an intentional response directed at that target, comprising the steps

making an electronic image of at least a portion of the sheet containing the target at a scanned pixel density of at least 150 dpi;

determining the target location within the image area;

superimposing a zone of measurement over the target location wherein the area of said zone of measurement is at least equal to the area of said target;

measuring the light density of each pixel within the zone of measurement;

calculating the sum of said light densities of all said pixels within the zone; and

comparing the sum to a pre-determined threshold light density indicating a mark.

17. A process for preparing multiple controlled articles of multiple forms for distribution of the controlled articles to identified recipients, including a pre-determined scheme of distribution of the forms of the controlled articles among the group of identified recipients, comprising the steps:

generating and storing in a relational database a schedule of form identifiers, each corresponding to a different said form of the article, and a unique article identifier for each article of each form in the quantities required, where each type of form identifier is distributed throughout the schedule and associated with a said article identifier for an article of the same form

creating and storing a unique identifier in said relational database for each intended said recipient of said articles;

associating each said form identifier and associated said article identifier in said schedule with a respective said recipient identifier;

producing and labeling an arrangement of said articles according to said schedule;

validating the arrangement of said articles for proper distribution of said forms of said article by comparing the form and article identifier of articles at selected locations within said arrangement to the corresponding schedule of form and article identifiers at corresponding locations within of the respective set of identifiers in said relational database.

18. A method for receiving, processing, scoring and reporting on secure test materials returned in an assessment project testing multiple testees, comprising the steps:

using a system comprising a document processing system for exposing documents including individual sheets and pages to electronic scanning, a computer network connected to the document processing system and to the internet, and a software program associated with said document processing system and said computer network;

said software program comprising global business rules, a relational database, a web browser application for system host and client interface to said system, and an image processing application associated with said document processing system, said relational database containing and relating a project identifier to project data and identifiers including project rules, data and identifiers for project locations, data and item identifiers and production requirements on all master items of the secure test materials including all test booklets and other documents containing a test item and all documents having an answer page, the production plan and history for assembly and shipment of said master items, and the tracking history of said secure test materials;

maintaining a record of unique identifiers of said secure test materials shipped to project locations and not yet returned;

receiving previously shipped assessment materials as returned materials;

scanning and comparing the unique identifiers affixed to said returned materials to said record of said materials shipped;

updating said record of said materials shipped;

reducing packages of said returned materials to sheets of secure test materials;

scanning each page of secure test materials and storing an image thereof in a data warehouse in association with its page identifier;

pre-selecting scanning procedures for said document processing system and said image processing application by reference to a returned item identifier;

scanning and storing an answer page image of each answer page of said secure test materials in said relational database in association with its respective said answer page identifier;

associating each used said answer page with a respective test booklet;

extracting answer information from said images of said answer pages;

converting said answer information into raw scores;

compiling said raw scores into a final score for all answer pages associated with each used said test booklet, and hence with each associated said testee;

compiling final scores for each said testee by project location; and

generating project reports based on said final scores.

20. The method of claim 18, comprising the further step:

wherein unique identifiers were incorporated with items of assessment materials in the form of an RFID chip programmed with at least a unique identifier associated with the respective said item;

reading said unique identifiers by means of bringing said assessment materials within range of an RFID signal receiver and initiating an RFID chip transmission, said RFID receiver communicating received information with said system.

22. The process of claim 20, said RFID chip being further programmed with processing instructions.

23. The method of claim 18, said steps of extracting answer information from said answer page images;

and converting said answer information into raw scores comprising;

distributing each said answer page image or an associated portion thereof among a group of scorers, each said scorer having an associated unique scorer identifier;

presenting selected said images for viewing in sequence to each scorer for scoring,

recording a raw score entered by said scorer as each image is viewed; and

associating said raw score with the said image identifier of the respective viewed image and with the scorer identifier of said scorer in said relational database.

24. A method according to claim 23, said step of presenting selected said images in sequence to each scorer comprising

caching images in said sequence subsequent to the current viewed image such that any interruption in the scorer'"'"'s station, program or network results in the dropping of cached said images.

25. The method of claim 18, said steps of extracting answer information from said images of said answer pages;

and converting said answer information into raw scores comprising;

creating clipped images by use of a template of areas of said answer page image containing designated answer targets intended for selective marking by a testee in response to instructions and test items;

determining the pattern of targets marked by automated scanning of said target zones; and

comparing said pattern of targets marked to an associated schedule of possible patterns of targets marked and related range of raw scores in said relational database.

26. The method of claim 25, said step of determining the pattern of targets marked comprising the steps:

superimposing a zone of measurement over each target location within the clipped image area wherein the area of said zone of measurement is at least equal to the area of said target;

measuring the light density of each pixel within the zone of measurement;

calculating the sum of said light densities of all said pixels within the zone; and

comparing the sum to a pre-determined threshold light density indicating a mark.

34. A method according to claim 18, said process further comprising the steps:

automated reviewing of each specific answer area of each said answer page image;

automated detecting of the absence of a testee input in said specific answer area;

automated recording in said relational database of the said absence of a testee input associated with said specific answer page area of said related answer page; and

automated withholding from said scorers of said specific answer area of said answer page image.

35. A method according to claim 18, said image processing application comprising a scalable image capture and image generation sub system.

36. A method according to claim 18, said document processing system and said software program comprising in combination capability for automated indexing in said relational database of said images according to specific attributes in order to facilitate retrieval of said images on the basis of said selected attributes.

37. A method according to claim 36, comprising the further step:

extracting said index information from said images using any of zone optical character recognition, OMR, bar code, and ICR techniques.

38. A method according to claim 18, said image processing application and said software program in combination comprising the use of pipe functions where output from a first command is the input for a subsequent command.

39. A method according to claim 18, comprising the further step of

transferring extracted said answer information directly to pre-defined sequel data files.

40. A method according to claim 18, said step of generating project reports comprising:

extracting testee scores from said relational database; and

populating a pre-defined report template in a specific sequence.

41. A process for conducting an assessment project of testing multiple testees comprising the steps of:

using a system comprising a document processing system for exposing articles and individual pages to electronic scanning, a computer network connected to the document processing system and to the internet, and a software program associated with said document processing system and said computer network;

said software program comprising global business rules, a relational database, a web browser application for host and client interface, and an image processing application associated with said document processing system;

storing in said relational database a unique project identifier and associated project data;

said project data comprising project rules, project locations and unique location identifiers, and quantities of testees at each said project location;

generating a master items production quantity requirement which includes each item of assessment materials for said project in accordance with said global business rules and said project data, said items comprising secure test materials, said secure test materials comprising test booklets and answer pages susceptible of electronic scanning;

creating and storing in said relational database a unique test booklet identifier corresponding to each respective said test booklet and a unique answer page identifier corresponding to each respective answer page;

producing said assessment materials corresponding to said master items production quantity requirement, including producing each said test booklet and answer page with their respective said unique identifiers inseparably affixed thereto;

generating a production plan for assembly of said assessment materials in accordance with said global business rules and said project data, said plan comprising using combinations of pre-selected intermediate quantities of said test booklets and said answer pages for packing shipping containers;

packaging said pre-selected intermediate quantities of said secure test materials into class packs according to said production plan, each said class pack being assigned and labeled with a unique identifier which is stored in said relational database and associated with the unique identifiers of the items of secure test materials within;

allocating selected said class packs to specific project locations, and associating in said relational database the unique identifiers of each said class pack with the unique identifier of the respective project location;

producing and including return shipping envelopes for each said project location for packaging and returning selected said items of secure test materials, each said envelope assigned and labeled with a unique identifier which is stored in said relational database and associated therein with the project location identifier and item identifiers of items expected to be returned therein;

packing said class packs said return shipping envelopes by project location into said shipping containers, each said container being assigned and labeled with a unique identifier which is stored in said relational database and associated with the unique identifiers of the class packs and return shipping labels within and the respective project location identifier of the project location for which it is intended;

consigning said containers to a shipper for shipping, obtaining therefrom a shipper'"'"'s identifier for each said container which is stored in said relational database and associated therein with the unique identifiers of the respective said container and the project location;

sending a notice of shipment including the shipper'"'"'s identifier for each said container to a designated POC at each respective project location;

periodically querying said shipper for status of said containers consigned, recording responses in said relational database, and associating said responses with respective said shipper'"'"'s identifier for each said container; and

generating on demand, status reports relating to said project.

42. A system for conducting large-scale test and assessment projects comprising

a relational database consisting of a core database of global rules and project data, a tracking database, a scanning and image management database, and a scoring database;

a software program and computer network connected to said relational database, a web interface linking said software program and computer network to host and client users, said software program and said relational database configured for generating a production requirement of master items and a packing schedule of said master items from said rules and said project data, and for tracking the packing and shipping of said master items, tracking the return and scanning of secure said master items, scanning and scoring said secure said master items, and reporting on the resulting scores.